Traffic Shaping Guide
- 1 PFSense 1.2.x Traffic Shaping Guide
- 2 PFSense 2.0 Traffic Shaping Guide (Work In progress)
- 2.1 What is traffic Shaping?
- 2.2 Wizards
- 2.3 Queuing Schedulers
- 2.4 ACK Queue Size
- 2.5 Floating Rules
- 2.6 Limiter
- 2.7 Layer 7
- 2.8 Example Scenarios
- 2.9 Other Documentation
PFSense 1.2.x Traffic Shaping Guide
The 1.2.x traffic shaper will not work for more than one WAN and one LAN interface. Please look at version 2.0.x for that functionality.
It is recommened that you use the Traffic Shaping Wizard to create a default set of rules from which to start. The rules the wizard creates can sometimes cope well with VOIP traffic, but need tweaking to accomodate other traffic.
As an example, let's look at shaping P2P traffic. Assuming you used the wizard, there will be qP2Pup and qP2Pdown created already. When you launch a P2P app, you should see traffic in these queues. They are designed to carry the bulk P2P traffic, which normally slows your connection down. Other generic traffic, like web pages (HTTP), email, IM, VOIP etc will go into other queues.
Initially, the wizard sets all queues to 1% bandwidth. This is not enough. In particular, the queue qwanacks certainly needs more bandwidth if you do a lot of downloading. First, a quick note about ACK packets.
ACK Queue sizing
When you download, your computer needs to send (upload) ACK packets. These are basically saying "yep, I got that part of the download OK". If the computer you are downloading from detects that an ACK has not been received, it assumes that the data was not received and sends it again. The rate at which ACKs are sent back is also used to help determine the maximum speed that you can download data at, so it's important that ACKs get sent as soon as possible and don't get dropped in order to keep your downloads flowing fast. Also, repeatedly dropped ACKs can result in dropped connections, web page time-outs etc.
When you download, qwanacks is where the ACK packets your computer sends out go. You need to make sure this queue has enough bandwidth to maintain your downloads. To work out how much bandwidth you need, there are two options. You could simply experiment, keeping an eye on the queue while downloading as fast as your connection will allow, or you could try and work it out. As a rough starting point, an NTL 10Mb/512Kb cable connection needs about 260-270Kb/sec of ACK packets to download at full speed.
Taking the above example, we can see that ACKs can consume 60% of the available upload bandwidth. Thus, qwanacks should have at least 60% bandwidth available (I use 65% for the above). If you set qwanacks like this, you should not see any drops in that queue. However, you will see a lot in qP2Pup, but that's OK. P2P upload packets are just bulk traffic, not really important so it doesn't matter if they drop a bit. qP2Pup will now be using what is left of the available upload bandwidth, after qwanacks has used up to 65% of it. You will probably want to increase the bandwidth allowance for qwandef as well, since this is where HTTP requests and other general uploads go, which chances are you want to be higher priority than qP2Pup. Bandwidth percentages need not add up to 100%, but unless you have a very slow connection you don't need too much for qwandef since it is mainly small requests or the odd few kb of email.
PFSense 2.0 Traffic Shaping Guide (Work In progress)
What is traffic Shaping?
Traffic shaping (also known as "packet shaping") is the control of computer network traffic in order to optimize or guarantee performance, lower latency, and/or increase usable bandwidth by delaying packets that meet certain criteria. More specifically, traffic shaping is any action on a set of packets (often called a stream or a flow) which imposes additional delay on those packets such that they conform to some predetermined constraint (a contract or traffic profile).
PFSense 2.0 includes multiple wizards that setup the traffic shaping for various usage scenarios. It is recommend to start out with one of the Wizards since having a known starting point makes support easier.
Single Lan multi Wan
This will setup the traffic shaping for the case where you have one local network and multiple WAN networks. You would also use this in the case where you only have one WAN and one LAN. Just specify 1 for the number of WAN links when going through the wizard.
Single Wan multi Lan
This will setup the traffic shaping for the case where you have multiple local networks and one WAN network.
This will setup the traffic shaping for the case where you have both multiple local networks and multiple WAN networks.
Dedicated links is for when you want to manage more than 1 link which get routed to a separate different LAN in the same box. So a single firewall manages several 'virtual' links through it. For example, say you were providing services to 4 different customers in one building, and they each had their own separate internet connections. You could run all 4 internet connections through one pfsense box to each of the customers LAN networks and provide separate traffic shaping configurations to each one.
Creating new Wizard templates
Wizard configuration files are located under /usr/local/www/wizards.
Priority Queueing (PRIQ)
Priority queuing is the simplest form of traffic shaping you can select. You create a flat hierarchy of priority levels, all packets at the highest priority level are always processed first.
- Easy to configure and understand.
- Lower priority queues can be completely starved for bandwidth easily.
Class Based Queueing (CBQ)
Hierarchical Fair Service Curve (HFSC)
ACK Queue Size
The size of the ACK Queue often needs to be adjusted with asymmetrical links since by default the size is based on both up and down speed being equal.
Floating rules allow you to to set shaping rules for all interfaces at once. They are evaluated before the interface rules, and are non terminating. The last floating rule that matches a stream will be the one that applies.
- When modifying floating rules, remember to clear the firewall states before you start testing your changes. If you do not do this, you many not get the results you expect.
The limiter feature allows you to setup Dummynet pipes. Dummynet was designed to be able to simulate any kind of network connection. You can simulate a dialup connection, a T1, a T1 run through a microwave oven, or a satellite connection to the Moon. A side effect of being able to simulate any type of network connection is that you can use them to limit the amount of bandwidth a host or group of hosts have access to.
Both the packet shaper and limiters can be used at the same time so you can shape your traffic as a whole, and also limit certain traffic to a certain amount of bandwidth.
There are 2 basic steps to setting up a limiter to control bandwidth.
- Setup the limiters you will be using.
- Assign traffic to those limiters.
Limiters are setup by creating them under Firewall > Traffic Shaper, on the Limiters tab.
You can use just one pipe for both inbound and outbound traffic, but that would mean you are simulating a half-duplex connection.
The recommended method is to create 2 pipes, one for inbound traffic and one for outbound traffic. The direction is from the perspective of the interface. If using limiters on LAN, the inbound queue is your upload and the outbound queue is your download. You should name the pipes so that you will easily remember which one is which, such as InLimitLan and OutLimitLan.
Dynamic queue creation
Dummynet pipes have a feature called dynamic queue creation which allows you to have a unique queue based on the uniqueness of a connections source protocol, ip, source port, destination ip or destination port. They can also be used in combination. pfSense currently only allows setting the source address or the destination address as the mask, meaning that you can give each host behind your firewall its own set of pipes so that each node is restricted to using a certain amount of bandwidth. To do this you would give your In pipe a Source Address mask, so that each host sending packets gets it's own dynamic pipe for uploading. You would give your Out pipe a destination address mask, so that each host receiving packets gets it's own dynamic pipe for downloading.
Once you setup a limiter pipe, the next step is to assign traffic to it by setting the "in/out" option in a firewall rule. Remember that in and out are from the perspective of that interface on the firewall. If you're choosing limiters on the LAN interface, "out" is download speed (traffic from the LAN NIC into the LAN) and "in" is upload speed (traffic from the LAN into the LAN NIC).
Simply creating the limiters does not do anything, you must assign them on a firewall rule for them to be used.
The bandwidth usage and other limiter information is available under Diagnostics>Limiter Info.
Captive Portal Notes
Captive portal can automatically setup its own pipes for each logged in user, no need to set this up manually. Take a look at the captive portal setup page to set this up.
Using Limiters for Bandwidth Guarantees
If you want to use limiters to guarantee a certain amount of bandwidth instead of limit, you can do so by making four limiters.
- Bandwidth to guarantee upload
- Bandwidth to guarantee download
- Total bandwidth upload (less guaranteed above)
- Total bandwidth download (less guaranteed above)
Ensure that you do not set the Mask to anything other than "none". It must be "none" for these to work properly.
So if you have 8Mb down and 2Mb up, and you want to guarantee 512Kb/s for service X, you'd have queues sized like so:
- 512 Kb/s
- 512 Kb/s
- 1536 Kb/s
- 7680 Kb/s
Then direct the guaranteed service traffic into the first two limiters, and everything else into the "total" limiters.
Limiters on Bridges
When using limiters on bridges, you need to assign the bridge interface and put the IP address for the bridge there, and place the limiters on the member interfaces.
Visit Diagnostics > Limiter Info in the GUI, and it will show you the output of:
ipfw pipe show
Which lists all of the pipes currently configured on your system, and related information about their status.
- Dummy Net documentation: http://www.dummynet.com/
Layer 7 filtering or shaping is identifying traffic at layer 7 (Application Layer) of the OSI model. Instead of shaping/filtering based on the port and source/destination, you are identifying a stream based on its contents. This is also sometimes called deep packet inspection since it works by looking into the contents of the packets not just the headers.
You might want to use layer 7 if you need to deal with packets that use dynamic source and destination ports, or if filtering/shaping based on ports is not fine grained enough.
PFSense is using the ipfw-classifyd application to provide Layer 7 filtering capabilities.
* Initial Post announcing project. http://lists.freebsd.org/pipermail/freebsd-net/2008-July/019086.html * Research Paper by Andre Ribeiro, and Helder Pereira. http://www.di.uminho.pt/~prh/uce15-0809/g13.pdf
The protocol patterns are from the L7-filter project.
Defining Protocol Patterns
For now take a look at the L7-filter Pattern Writing Howto. http://l7-filter.sourceforge.net/Pattern-HOWTO
Patterns are stored under /usr/local/share/protocols.
Using Layer 7 with a bridging firewall
If you want to use layer 7 with a bridging firewall you will need to make the following change to a sysctl under system -> advanced.
net.link.bridge.pfil_member = 0 net.link.bridge.pfil_bridge = 1
After that change you can use the floating rules to assign traffic in a bridging environment.
If you don't make that change you would need to duplicate the assignment rules on each bridge member interface. (I think this is correct, but not sure).
Links to other useful documentation.
- PF packet flow diagram. - Notice how Altq is the last item to get processed.
- Shaper 2.0 bounty thread.
- ACK queue sizing thread.
- QoS/Traffic Shaping information and tips.
- Tips and Tricks thread.
- P2P queue sizing discussion.
- P2P queue sizing post.
- Monitoring PF article.
- PF: Packet Queueing and Prioritization (OpenBSD not FreeBSD)
- HFSC howto on Calomel.org.
- Paper on HFSC design.
- Linux HFSC description, good conceptual examples.