Realtime Remote pfSense Log Visualization with glTail

From PFSenseDocs

This article is part of the HOWTO series.

NOTE: This currently does NOT work with 2.0 snapshots based on FreeBSD 8.0. The log format has changed, and the parser has yet to catch up.

You can "watch" your logs in a visually compelling -- and actually useful -- manner using glTail.

I've written a glTail parser that will display firewall logs from pfSense in glTail's realtime log visualization on a workstation. If you haven't used glTail before, I also suggest checking it out for additional monitoring such as web and mail server logs.

glTail breaks the log output down into "blocks" such as Action (pass/block), Protocol (tcp/udp/icmp/etc), Host/Interface (fxp0, vr0, em0, vlan0, etc), source host, source port, destination host, destination port, and full source/destination pair. More active entries bubble up to the top of their respective blocks, and are displayed with a "requests per minute" counter. In some ways it is sort of like top, showing the most active entries in the firewall logs. Here is a screenshot of what it looks like from a slightly older revision of the parser. (It might also look a little funny since I blacked out some IP Addresses).

It can take a bit of CPU time and some graphics card power to run, but it is worth it. It also looks pretty enough to show PHBs/Management that the firewall is "doing stuff", especially if you see a lot of blocked packets in your logs.

I had to use a slightly older version of glTail than what the author has available in his public git repository. The current version uses a library that is not currently working on Windows. It should work fine on Ubuntu. I have the source for a version of glTail that should work fine on either Windows or Ubuntu (and probably elsewhere) available on my site Download glTail source.

What follows is a set of general instructions that should get glTail up and running, and talking to your pfSense system. It's not exactly a simple process, and some things will vary from system to system, so it might take some effort beyond what I have outlined below.

Method 1, The Easy Way (Windows only at the moment)

Step 1 - Ensure ssh is turned on in pfSense, and accessible from your workstation.

Step 2 - Download the glTail binary executable and unzip it somewhere convenient on your workstation.

Step 3 - Edit the config file (gl_tail.yaml) as needed

• Change the host and password to suit your needs. Leave the username as root. The password is the same as admin in the webgui, but this will allow glTail to run a command and not get caught in the shell menu.
  • You may leave the password blank and it should prompt you to enter it when run.
• If you are connecting to a syslog server instead of directly, comment out the first host definition and uncomment the second, and change the settings as needed.
• You might need to change the dimensions, but it does not work properly at less than about 1000px wide.
• There is also an option at the bottom to turn off/on reverse IP lookups if you want to turn them off

Step 4 - Double click the glTail.exe executable

Step 5 - Enjoy! (And read the Caveats section below)

Method 2, Using the Source and installing required dependencies: (Windows/Linux)

Step 1 - Ensure ssh is turned on in pfSense, and accessible from your workstation.

Step 2 - Download the glTail source and unzip it somewhere convenient on your workstation.

• In case you missed it above, you can get the glTail files I worked with here: Download glTail source.
• Much of the following setup is covered in the README/README.txt file included in the Zip.
• If you choose to use a different version of glTail obtained elsewhere, copy the pfsense.rb file from the lib/gl_tail/parsers directory in the zip to your local installation of glTail. If you are using the latest version of glTail from the public git repository, the pfSense parser is now included in the distribution. You will likely also need the supplied example config.yaml to work from/refer to.

Step 3 - Install Ruby and rubygems on your workstation.

• Windows:

Download and run the Ruby One-Click Installer

http://rubyforge.org/frs/download.php/29263/ruby186-26.exe

(Or most recent final from http://rubyforge.org/frs/?group_id=167&release_id=28426 )

• Ubuntu:

sudo apt-get install rubygems rake ruby1.8-dev libgl1-mesa-dev libglu1-mesa-dev libglut3-dev build-essential

• This will also install some libraries required to build the native ruby extensions needed for glTail

Step 4 - Install the prerequisite gems on your workstation:

gem install -y file-tail -r
gem install -y ruby-opengl -r

• Select 0.60.0 (i386-win32 for Windows, native for Ubuntu)

gem install -y net-ssh -v 1.1.4
  

Step 5 - Edit the config file (config.yaml) as needed

• Change the host and password to suit your needs. Leave the username as root. The password is the same as admin in the webgui, but this will allow glTail to run a command and not get caught in the shell menu.
• If you are connecting to a syslog server instead of directly, comment out the first host definition and uncomment the second, and change the settings as needed.
• You might need to change the dimensions, but it does not work properly at less than about 1000px wide.
• There is also an option at the bottom to turn off/on reverse IP lookups if you want to turn them off

Step 6 - Run glTail from a command line on your workstation Windows:

 cd <path_to_gltail>
 ruby bin\gl_tail config.yaml

Ubuntu:

 cd <path_to_gltail>
 ruby bin/gl_tail config.yaml

• If any libraries are missing, it should print out an error that tells you which ones and will even tell you how to install them.

Step 7 - Enjoy!

Caveats:

1. If you have trouble running Ruby on Windows, try adding "C:\Ruby\bin" to your path.
2. When connecting directly, there is no way to only view the end of the log. The clog program to view circular logs will dump the entire log to the parser, then will tail it showing new messages. Therefore, the parser I wrote has to run a simple time check and only view entries from the last 5 minutes, or the "future".

   The log output from pfSense does NOT include a time zone, so it is assumed that the workstaion is set to the same time as the pfSense box. If this is not the case, you may need to edit pfsense.rb and correct the time zone. You may also need to ensure that your workstation time is correct, as well as the time on the pfSense box.

   Symptoms of an incorrectly set time zone would be that either no data ever shows up, or that too much shows up all at once when glTail is started.

3. In case it wasn't obvious from the above, this monitoring/output is done remotely via ssh from a workstation. This does not happen on the server or in the webgui.
4. If you have a lot of IPs pointed at your pfSense box, and someone does a port scan, it might get a tad sluggish as it draws the deluge of new entries. This is entirely dependent on your workstation, and may not be noticeable if you have a fast CPU/GPU and proper OpenGL-aware drivers. This may also happen during the start of a run, if there are a lot of log entries in the last 5 minutes or so.
5. Every once in a while there might be a log entry that isn't properly interpreted by the parser. If I can get examples of log entries that are formatted incorrectly, I can refine the parser to either handle or ignore such entries.

Let me know if this works for you!

I have submitted the parser to the author of glTail so it might be included in future versions.

Thanks go to Erlend Simonsen for writing glTail!

Originally posted in the forum here.