Grafana: pfSense Per IP Usage Statistics Panel with NtopNG

Ever wanted to have per host usage statics with pfSense? I know I have for awhile. There are packages you can get from the package manager that will accomplish this. Both darkstat and bandwidthd will do this for you. They will host a webpage or web server on your pfSense router. They are pretty self explanatory on how to configure and access, and they work decently for a basic use case. However they have either non existent exporting mechanisms or not well documented ones. For us this is a no-go as Grafana needs exportable data. Luckily there is another option and that has multiple methods of export, NtopNG. This not only has a plugin for Grafana, but it also has a option to export to InfluxDB. For this guide, I’ll be using Influx as I am more comfortable with it, and I could not get the ntopng plugin for Grafana to work.

If you have not already setup Grafana and InfluxDB, we have a guide on how to set one up here: https://angrysysadmins.tech/index.php/2020/04/bailey/grafana-how-to-install-grafana-influxdb-on-centos-8/

InfluxDB

To get a database and user created run the following on your influx instance replacing the $variables with what you will be using.

create database $NewDatabase
CREATE USER $Username WITH PASSWORD '$Password'
grant all privileges on $NewDatabase to $Username

Ntop

NOTE: The webgui may crash during this step. This will not effect the routing of the network. You either need physical console access or ssh access to fix it. BEFORE continuing I’d suggest logging in via ssh. The fix is to restart PHP-FPM (option 16 on the console\ssh welcome screen). If the issue keeps happening just keep restarting php-fpm. This seems to only happen when accessing the ntopng settings, so in production it should not cause issues with anything else. If the ntopng webgui won’t come up restart the webconfigurator or reboot the machine

 

Logon to the pfsense webgui and go to System -> Package Manager. Click on Available Packages. In the search term field type ‘ntopng’ Then install the package. Once done with that, go back to the home page of pfsense

Next on the top menu go to Diagnostics -> ntopng Settings. Here you need to set a admin password and a interface to bind the ntopng webgui to. Choose LAN. Make sure to check enable ntopng. Hit save at the bottom.

Now we can access the ntopng webpage. go to Diagnostics -> ntopng. Log in with admin as the username and your password you just set.

Navigate to the gear at the top right and choose user preferences. Next click the timeseries button. From there change the timeseries driver from RRD to InfluxDB. Populate it with your database IP, DB, username, and password.

Grafana

Add the influxDB data source, then create a new panel. Here is an example panel I made. It will list all the active hosts with the inbound and outbound bandwidth totals.

 

Click here for the JSON
{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "gnetId": null,
  "graphTooltip": 0,
  "id": 18,
  "links": [],
  "panels": [
    {
      "cacheTimeout": null,
      "datasource": "RoutingLokTech_NtopNG",
      "fieldConfig": {
        "defaults": {
          "custom": {
            "align": null
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          },
          "unit": "bytes"
        },
        "overrides": [
          {
            "matcher": {
              "id": "byType",
              "options": "string"
            },
            "properties": []
          }
        ]
      },
      "gridPos": {
        "h": 9,
        "w": 12,
        "x": 0,
        "y": 0
      },
      "id": 2,
      "interval": null,
      "links": [],
      "options": {
        "showHeader": true,
        "sortBy": [
          {
            "desc": true,
            "displayName": "Inbound"
          }
        ]
      },
      "pluginVersion": "7.1.5",
      "targets": [
        {
          "groupBy": [
            {
              "params": [
                "host"
              ],
              "type": "tag"
            }
          ],
          "limit": "1",
          "measurement": "host:traffic",
          "orderByTime": "ASC",
          "policy": "autogen",
          "query": "SELECT \"bytes_rcvd\" AS \"Inbound\", \"bytes_rcvd\" AS \"Outbound\" FROM \"autogen\".\"host:traffic\" WHERE $timeFilter GROUP BY host LIMIT 1",
          "rawQuery": false,
          "refId": "A",
          "resultFormat": "table",
          "select": [
            [
              {
                "params": [
                  "bytes_rcvd"
                ],
                "type": "field"
              },
              {
                "params": [
                  "Inbound"
                ],
                "type": "alias"
              }
            ],
            [
              {
                "params": [
                  "bytes_rcvd"
                ],
                "type": "field"
              },
              {
                "params": [
                  "Outbound"
                ],
                "type": "alias"
              }
            ]
          ],
          "slimit": "",
          "tags": []
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "Most active hosts",
      "type": "table"
    }
  ],
  "schemaVersion": 26,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "TestingDash",
  "uid": "KMeK8SOGk",
  "version": 1
}

 

If anyone has any cool panels to add to this I’d be happy to post the JSON here and credit you for it! Just post it as a reply and I’ll add it.

 



About: Ryan Parker

I am former captain of the Cyber Defense team form Cal State San Bernardino. I also have a side job helping small to medium business with anything technology, including but not limited to servers, networking, and end user devices. One of my hobbies is building out infrastructures for myself, friends, and clients. I current maintain a VMware ESXi cluster with about 280GB of RAM, with a 10Gbit network as backbone.


Leave a Reply

Your email address will not be published. Required fields are marked *