pfSense HA (Hardware/Device Failover) Configuration using CARP

Introduction

This is a short guide on how to set up device failover configuration using pfSense. I have written this as part of a training exercise for myself after gaining the initial knowledge from the official documentation here which was invaluable in my learning. Much of this information in this blog can be gained directly from the official documentation, but perhaps it will clear some things up for you if you are having any issues!

Before you start, I recommend you read through this guide in full once before attempting to follow it.

Topology

There are various ways you could build a network to increase device redundancy, but this is a topic for another day. Here are two examples of how you may wire in two pfSense devices. The steps below would work for both of these designs.

Part 1: Device Sync Config (pfsync)

Network Interfaces

I assume you have two devices at factory configuration that will act as a master and a slave.
By default, only the WAN and LAN interfaces will be assigned. You will need to assign (Menu: Interfaces -> Assign) any OPT interfaces. In this case, OPT1 is being assigned as the sync interface. You should label this interface something clear such as “PFSYNC”

We are going to use three different local nets for this example. WAN 192.168.1.0/24, LAN 10.0.1.0/24, and SYNC 172.16.1.0/24. The WAN and LAN interfaces will take up 3 IP addresses each, the 2 physical device IPs, as well as the CARP IP address.

I recommend using the Setup Wizard to fill in as much as possible for the below details (WAN IP + GW, LAN GW). When changing the LAN IP, the wizard will update the DHCP server settings too. If you change the LAN settings manually after the Setup Wizard, remember to update the DHCP Server before applying the LAN settings as described below.

Menu: Interfaces

master:
WAN: Static IPv4 192.168.1.21/24 Gateway 192.168.1.1
LAN: Static IPv4 10.0.1.21/24
PFSYNC (OPT2): Static IPv4 172.16.1.21/24

slave:
WAN: Static IPv4 192.168.1.22/24 gw 192.168.1.1
LAN: Static IPv4 10.0.1.22/24
PFSYNC (OPT2): Static IPv4 172.16.1.22/24

Menu: Services -> DHCP Server -> LAN

Change the range to 10.0.1.100 -> 10.0.1.199 (you can fine tune this later)

Menu: Interfaces -> LAN

Apply the changes here. You will now need to get a new DHCP lease dhclient eth0 -v, and connect to the new LAN IP (10.0.1.21 for master or 10.0.1.22 for slave). If you do not get a DHCP lease, you can manually configure your local network device using ifconfig eth0 10.1.0.33 and attempt to fix. Otherwise you, may need to connect to the console and attempt to fix the config there, and possibly reset and start again!

Firewall Rules
Menu: Firewall -> Rules
(Both devices)
On the sync interface (PFSYNC), at a rule with the following parameters:
ipv4 prot:any source:any dest:any
Apply the changes. Once this part is done on both devices, you should be able to check connectivity between devices by going toMenu: Diagnostics -> Ping and input the other devices IP address.

Sync Settings
Menu: System -> High Avail Sync
State Synchronization Settings (pfsync)
Enable, and set Peer IP to 172.16.1.22 on master, and 172.16.1.21 on slave (i.e. point to each other), and set Synchronize Interface to PFSYNC

Configuration Synchronization Settings (XMLRPC Sync)
Only on master, enable, and set peer IP to slave 172.16.1.22 enter admin user/pass, and enable all sync ticks.

Part 2: CARP

Working exclusively on the master device, but check slave device to make sure config is syncing.

CARP interface

Menu: Firewall -> Virtual IPs
Add a CARP interface with the following parameters:

type:CARP if:WAN addr:192.168.1.20/24 pass:s0m3p4SS1 VHID:1 adv:1/0 desc: WAN-CARP-GW

Add a second CARP interface with the following parameters:

type:CARP if:LAN addr:10.0.1.20/24 pass:s0m3p4SS2 VHID:2 adv:1/0 desc: LAN-CARP-GW

Verify all of these changes have replicated to slave device. You will notice the skew on the slave will be set to a higher number (100) which defines this device as being a slave/backup.

NAT / Gateway

Menu: Firewall -> NAT -> Outbound
Switch to Manual Outbound NAT rule generation
Edit each rule, and change Translation to the WAN-CARP-GW (192.168.1.20)
Verify all of these changes have replicated to slave device

CARP Status
Menu: Status -> CARP (failover)
You should see the two CARP interfaces here, on the master they should both have the status of MASTER, and on slave they should have the status of BACKUP

DHCP Settings
Menu: Services -> DHCP Server -> LAN
Enable DHCP server on LAN interface
DNS servers: add the LAN-CARP-GW IP (192.168.1.20)
Gateway: add the LAN-CARP-GW IP (10.0.1.20)
Failover Peer IP: 10.0.1.22 (note, if the clocks are not synced this will break DHCP)
Review the rest of the DHCP config

DNS Resolver
Menu: Services -> DNS Resolver
TIP: Edit and save the DNS resolver settings (without making any changes), this will regenerate the config so that the DNS resolver can respond on the CARP interface.

Part 3: Test
If you are lucky, you can now refresh your interface IP dhclient eth0 -r; dhclient eth0 -v and everything will work perfectly. Regardless, you should now review, test, unplug, halt, plug in, reboot, kick, repeat. If all is well, then factory reset and do this again without reading the guide!

Comments

comments