Personal tools
Views
Multi-WAN Version 1.2.x
From PFSenseDocs
This community-contributed guide leaves out some important information and considerations. The best source of multi-WAN information is in the pfSense book.
Introduction
This setup enables pfSense to load balance traffic from your LAN to multiple internet connections (WANs).
Traffic from the LAN is shared out on a round robin basis across the available WANs.
pfSense monitors each WAN connection, using an IP address you provide, and if the monitor fails, a failover configuration is used, this typically just feeds all traffic down the other connection(s).
This example sets up 2 WANs, but 3 or more can be used by simply extending what this page describes.
Note that currently most pfSense add-on packages do NOT support multi WAN and all their traffic will use the WAN connection.
Overview
This guide helps you setup pfSense to support a local network (the LAN) and 2 connections to the internet (WAN and WAN2). Most traffic is shared out between the 2 WAN connections, but specific rules are also setup for some types of traffic to only use 1 connection (for example https), where load balancing can cause problems.
pfSense runs in a small system that uses 3 network interface cards (NICs), 1 for each of the WANs and 1 for the LAN.
pfSense can also be run in a virtual machine for testing and lightweight use, although this is not as secure or robust as a physical machine implementation.
The guide also shows how to setup access from the internet to servers on the internal network, and has guides to the setup for some specific applications.
Note that if you install servers connected to DMZ1 or DMZ2, these are not protected by pfSense, and will have to be internet hardened.
Before you start
You must have completed the basic pfSense installation.
Target network setup
This guide assumes the following network setup; you can easily do something different, but you will need to translate network addresses appropriately if you do.
- Your ISPs have assigned a single IP address for each internet connection (which could be dynamic) and you are using your modem / routers in router mode (some guidance on other variants of this are included in the details below).
- DMZ 1 is going to use the subnet 192.168.0.0/24
This means that DMZ 1 uses IP addresses between 192.168.0.0 and 192.168.0.254. - DMZ 2 is going to use the subnet 192.168.1.0/24
This means that DMZ 2 uses IP addresses between 192.168.1.0 and 192.168.1.254 - The LAN uses subnet 192.168.10.0/24
This means that the internal network uses IP addresses between 192.168.10.0 and 192.168.10.254
You should pick up the 3 interface cards. Note that if you have DHCP turned off on your WAN1 modem router, there will be a long pause here while pfSense tries to pick up an IP address.
Finishing pfSense console setup
The console will eventually give a prompt pfSense console setup. Select option 2 and setup up the LAN interface as follows:
- LAN IP Address
- 192.168.10.254
- subnet bit count
- 24 (for a class C space) - this will allow up 250 computers to be used
- DHCP
- y
- DHCP start address
- 192.168.10.10
- DHCP end address
- 192.168.10.200
You should now be able to plug a PC into the network, and it will be allocated an IP address and you will be able to access pfSense web interface (although not much else yet).
Setting up your modems / routers
Router mode setup
If you have CABLE/DSL modems that are bridge routers you may want to use them in router mode. The client ID (PPPoE) is installed on the modem/router and the modem/router maps the Public IP it receives to a Private IP on the modem/router LAN interface. How to do this is specific to each modem/router.
| setting | WAN (WAN1) modem / router | OPT1 (WAN2) modem / router |
|---|---|---|
| LAN IP address | 192.168.0.254 | 192.168.1.254 |
| Subnet mask | 255.255.255.0 | 255.255.255.0 |
| DHCP | on | on |
| DHCP address range | 192.168.0.10 - 192.168.0.100 | 192.168.1.10 - 192.168.1.100 |
Once you have set up the modem/routers you can test them by plugging a PC into their network, and accessing your favourite web site.
Or you can wait until the basic pfSense configuration is in place, and test through pfSense.
Note if you are *cheating* by running multiple subnets on one physical network, you must have DHCP turned off on all but 1 subnet.
Bridge mode setup
If you have a fixed IP address from your ISP you can also use bridged mode for some or all of your connections. (If you do not have a fixed address it makes life complicated in pfSense)
In bridged mode, the modem becomes a transparent (in IP terms) device, and your internet IP address is allocated to the pfSense interface. This makes life a bit simpler as it means there is one less NAT going on.
You can usually set up at least WAN1 to work in bridge mode (if your modem / router allows it). as this connections allows PPPoE or bigpond account information to be configured in pfSense.
If you do this, your ISP assigned address will replace the 192.168.x.y address (from the router mode setup above) in the later sections of the setup.
Using the pfSense Wizard
- Go to http://192.168.10.254 (or the address you gave pfSense if different)
- Select System - Setup Wizard from the menu
General parameters screen
- hostname
- leave as pfsense
- domain
- as you like - I use me.local at home
- Primary DNS server
- a DNS address from WAN1 DNS list
- Secondary DNS server
- a DNS address from WAN2 DNS list
- Allow DNS server list to be overridden by DHCP/PPP on WAN
- Unchecked - if this is checked you won't see the right DNS server list when you set up load balancing pools
- Click next
Note: it is important to use one from each (or use a public DNS service) or you will loose internet access when one or other connections fails.
date, time and time zone screen
- time server DNS name
- its a good idea to select a local service - either the one your ISP provides, or a local pool.ntp.org address (for example uk.pool.ntp.org if you are in the UK, or one in your time zone).
- Timezone
- pick the right entry from the time zone. Note pfSense can provide an NTP service so all your local machines pick up time from pfSense.
- click Next
WAN configuration
If have set your WAN modem router to DHCP, you can leave this set to DHCP, otherwise:
- Selected type
- Static
- IP address
- 192.168.0.1 /24 (or an address in your DMZ1 subnet)
- Gateway
- 192.168.0.254 (or the address you gave your fist modem / router
If you are using a plain modem then you can set up your ISP account information here, I can't find a wiki page about this, but there several threads in the forums that discuss this.
LAN configuration This was set up through the console so shouldn't need changing
Change your password and reboot
Put in a sensible password, then let pfSense reboot.
After Wizard general setup
These settings make it easier to access machines on your local network - you can access them by name, and if you are running Windoze you will not suffer at the vagiaries of WINS.
- Go into 'Services' - 'DNS Forwarder', turn on
- Register DHCP leases in DNS forwarder
- Register DHCP static mappings in DNS forwarder
Initial setup for Load balancing
Finishing the interfaces setup
Now it is time to finish setting up the interfaces and make sure they are setup OK.
Setting up the OPT 1 interface
From the pfSense menu select Interfaces - OPT1 and set up as follows:
- enable Optional 1 interface
- checked
- Type
- Static - assumes you are not using an address assigned by your ISP
- MAC address and MTU
- do not usually need to be set - see info on screen
- Bridge with
- None
- IP address
- 192.168.1.1 /24 - or an appropriate address in DMZ 2 if you used a different subnet
- Gateway
- 192.168.1.254 - or whatever address you gave modem / router 2 (or your ISP has assigned, if no routing being used)
Checking interfaces
From the pfsense menu select Interfaces - Assign and you should get an screen like the one of the right. Note your hex numbers (The MAC addresses) will be different.
Now to check that pfSense can see your modem routers you use Diagnostics - Ping. With WAN 1 selected, enter the IP address of your modem / router - 192.168.0.254 if you are using the guide values in this document.
If you are using using a modem / router without NAT, the check first that the WAN link is up and ping the DNS server address that you recorded earlier.
FTP helper: Check also that FTP helper is only enabled for the LAN interface. That is it should be disabled on all WAN interfaces
Setting up Load Balancing pools
Overview
This setup uses 3 pools
- One pool for load balanced use when both WANS are working
- One pool which prefers WAN 1, for use when WAN 2 has failed
- One pool which prefers WAN 2, for use when WAN 1 has failed
These pools use the 2 gateways that are already established (by the interfaces WAN and WAN 2) to load balance and support failover when a WAN link fails
Selecting a Monitor IP address
pfSense monitors each WAN connection by pinging the monitor address you specify. If the ping fails, the link is marked down and the appropriate failover configuration is used (actually if the ping fails it retries a few times to be sure, this avoids false indications of the connection going down).
Note that pfSense automatically sets up to route traffic to your monitor IP only down the link it is monitoring, so don't use a popular web site as this will force all its traffic down 1 link. Better to use a router or server in your ISP's network.
Good addresses to use your ISP's DNS server (1 from each ISP). The web interface makes it easy to pick these when setting up the pools later.
Other good monitor addresses are the default gateway your modem has assigned (if it responds to ping!), your ISP's webmail server, or a router within your ISP's network - you can find one of these by using traceroute to a public service, be careful though, larger ISPs will have networks that dynamically adapt so a router you see now may not be there an hour later!
Setting up the pools
We are going to set up 3 pools in Services - Load Balancer
Note that each pool has 2 monitors set up, when complete the 1st pool should correspond to the screenshot on the right.
| Setting | Pool 1 | Pool 2 | Pool 3 |
|---|---|---|---|
| Pool name | LoadBalance | WAN1FailsToWAN2 | WAN2FailsToWAN1 |
| Description | Round Robin load balancing | WAN 2 preferred when WAN 1 fails | WAN 1 preferred when WAN 2 fails |
| Type | Gateway | Gateway | Gateway |
| Behavior | Load Balancing | Failover | Failover |
| Port | Unused | Unused | Unused |
| 1st Monitor IP | DNS server 1 | DNS server 2 | DNS server 1 |
| 1st Interface name | WAN | WAN2 | WAN |
| 2nd Monitor IP | DNS server 2 | DNS server 1 | DNS server 2 |
| 2nd Interface name | WAN 2 | WAN | WAN2 |
This finals screenshot shows the summary you should end up with.
Setting up DNS for Load Balancing
Make sure that you have a DNS server from each ISP in the General Settings. This will ensure that you have DNS service in case one ISP goes down. You will also need to setup Static Routes for each DNS server. In this example if the DNS is on the WAN link then the static route for that DNS server will have 192.168.0.254 as the gateway. If the DNS server is on the other ISP (ie OPT1) then the static route will have have 192.168.1.254 as the gateway.
Sticky Connections
pfSense Version 1.2 introduced Sticky connections, which can be used as part of a MultiWan setup. Where Sticky connections are used, some of the firewall rules previously used are no longer required; this is noted in the information below. 'Sticky connections' are a very good where there are many active systems / users, or where your WAN connections are fast, they are not so useful for small number of users on slower connections (as the multiple requests involved in fetching a single web page will not be shared across the available connections.
Basic Firewall Rules
These are the rules you need to add to support access from your LAN to the internet. Later sections describe the rules you need to support incoming access from the internet to machines on your LAN, this includes how to support peer to peer applications.
First 3 rules
If you do not need to access any of your systems from the internet, and you use sticky connections, then these are probably the only rules you will need.
Set these rules up in Firewall - Rules, and then click the LAN tab.
| Rule | Load Balance | DMZ 1 | DMZ 2 |
|---|---|---|---|
| Position in rule list | Last | Top | Top(-1) |
| Action | Pass | Pass | Pass |
| Disabled | Unchecked | Unchecked | Unchecked |
| Interface | LAN | LAN | LAN |
| Protocol | any | any | any |
| Source | LAN subnet | LAN subnet | LAN subnet |
| Source OS | any | any | any |
| Destination | any | network: 192.168.0.0 / 24 | WAN2 subnet |
| Log | no | yes temporarily (see below) | yes temporarily (see below) |
| Schedule | none | none | none |
| Gateway | LoadBalance | default | default |
| Description | Everything else gets shared out | Make sure DMZ 1 traffic goes to right interface | Make sure DMZ 2 traffic goes to WAN2 DMZ |
Rule logging
It is always a good idea to put a new rule in with logging turned on, then check by generating some appropriate traffic, that the rule is working, then turn logging off once you know it is having the right effect.
Rule explanation - Load Balance
This rule must always be the last rule in the rule list. It catches anything else that is not special in any way, and load balances the traffic. Any rule that comes after this rule will never trigger, so may as well not be there!
Rule explanation - DMZ 1 and DMZ 2
These rules make sure that any traffic to the modem / router, (or other machines that are connected to this subnet if you are not using bridge mode), go down the right WAN connection. Without these rules you will find strange things happening when you try to access your modem / router.
These rules should always be top of the rule list as you do not want earlier rules to route this traffic elsewhere.
Testing these rules
- Testing the DMZ rules
- Use a web browser to access the administration interface on your modem / router. Then use Status - System Logs, Firewall tab to check if the rule has fired.
- Testing the load balancing rule
- Access any site on the internet, then check the firewall log (as above) to see if the rule fired.
Don't forget to turn off logging on the rules once you have checked them.
Testing failover
Now you should make sure that failover is working.
- Switch off (or unplug) one modem / router
- Check the pfSense Load balancer status screen ('Status' - 'Load Balancer')
it should show (within a few seconds) that one link has failed.- if it shows that both links have failed, it probably means you have your monitor IP's the wrong way round. Use a trace route from PC on the LAN to trace the route to each monitor IP address and if it is using the wrong WAN link, re-setup the WAN links the right way round.
- Now try accessing a internet site, it should appear without any problems.
If it fails, then check the load balancer status (see above). If one link is still marked up, check that it is not a DNS failure
Setting up for protocols that don't like load balancing
Some sites (for example banking sites) get upset when requests from a single session come from different IP addresses. To avoid this, protocols that are likely to suffer from load balancing are setup to favour 1 connection.
Note that use of the sticky bit (see above) should avoid this issue. If you are not using sticky bit, you definitely need this.
For each protocol that needs to be handled this way you need a rule on the LAN interface; the sample below is for https (port 443). The values marked in bold are the ones that change for different protocols.
These rules need to be above the final load balancing rule, and below the rules for DMZ access.
| Parameter | Value |
|---|---|
| Action | Pass |
| Disabled | unchecked |
| Interface | LAN |
| Protocol | TCP |
| Source: not | unchecked |
| Source: type | LAN subnet |
| Source OS | Any |
| Destination: not | unchecked |
| Destination: type | any |
| Destination port range | HTTPS |
| Log | checked initially; uncheck when known to be working |
| Gateway | WAN1FailsToWAN2 - or WAN2FailsToWAN1 as you prefer |
| Description | Route https through one working connection |
Other entries you are likely to need are SSH and POP3. For these just replace HTTPS in bold above with the protocol you requre, and amend the description.
Further Rules for handling outgoing traffic
Depending on usage there are likely to be other rules you will need for outgoing traffic.
Setting up rules to access specific ISPs
If you send traffic to hosts on a specific ISP (such as SMTP email) you may have to make sure that traffic goes to the right ISPs WAN connection. ISPs block mail being sent if it does not come from one of their customer's lines, so if you try to send mail through the wrong connection it will be rejected. If your WAN connections are from different ISPs and you send mail using SMTP you will need to do this. If you only use webmail (your email interface is a web browser, such as hotmail), you do not need this.
The simplest way to handle this is to route all SMTP traffic to one ISP - of course if you send SMTP mail through both ISPs you will need to handle this a different way.
For this type of use, the rule is setup to use only 1 WAN connection. This means that if the connection goes down, the traffic cannot pass, but as it would fail if it picked up the other connection this is the right behaviour.
The example below is for SMTP, change the bold parameters for other traffic
These rules should go in above both DMZ and preferred traffic rules
| Parameter | Value |
|---|---|
| Action | Pass |
| Disabled | unchecked |
| Interface | LAN |
| Protocol | TCP usually |
| Source: not | unchecked |
| Source: type | LAN subnet |
| Source OS | Any |
| Destination: not | unchecked |
| Destination: type | any |
| Destination port range | SMTP |
| Log | checked initially; uncheck when known to be working |
| Gateway | 192.168.0.254 or 192.168.1.254 or the appropriate gateway address for this traffic |
| Description | Route SMTP to the ISP that handles it |
-
This page was last modified on 23 November 2009, at 18:14.
This page has been accessed 208,881 times.