Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

nfsen-ng is a modern, in-place replacement for the ageing NfSen web frontend. It sits on top of the existing nfdump tool suite and adds real-time SSE push, a responsive UI, and a choice of RRD or VictoriaMetrics as the storage backend — without changing how nfcapd itself captures traffic.

Graphs — the traffic dashboard, light and dark

Who this is for

Anyone already running (or migrating from) NfSen/nfdump to monitor NetFlow traffic: source/destination breakdowns, protocol and port distributions, flow search, and threshold alerting, all served from the nfcapd files nfdump already writes.

The shape of the app

nfsen-ng is a single-page application: one route (/) serves the whole UI, and every “page” the nav bar shows — Graphs, Flows, Statistics, Sankey, Settings — is a client-local view switch, not a server round-trip. The server side is PHP 8.4 running on OpenSwoole coroutines via a small in-house framework (php-via), pushing UI updates to the browser over Server-Sent Events using Datastar. There’s no separate REST API and no client-side framework build step: the server renders Twig templates, and Datastar patches the DOM.

To get it running, start with Installation and Configuration. The Architecture chapter covers how these pieces fit together; Features walks each screen; and Development covers running it locally.

OpenSwoole’s FreeBSD/other-BSD port is unmaintained (openswoole/ext-openswoole#233), so nfsen-ng currently requires Linux.

Status

This book documents the v1.0.0-beta.2 line — see the Roadmap for what’s tracked and what’s next.

Installation

nfsen-ng sits on top of an existing nfdump capture setup: nfcapd writes rotated flow files, and nfsen-ng reads, imports, and visualises them. It does not capture traffic itself — you need a working nfcapd (or an equivalent NetFlow/sFlow/IPFIX collector) writing files before nfsen-ng has anything to show.

Linux only. The backend runs on the OpenSwoole PHP extension, which has no maintained FreeBSD/other-BSD port (openswoole/ext-openswoole#233). Docker images are Linux-only.

Two ways to install: Docker (recommended) or bare-metal. For a local development setup with source-mounted auto-reload, see Getting Started instead.

Option A — Docker

Prerequisites

  • Docker and Docker Compose
  • nfcapd writing files under a directory you can bind-mount (default /var/nfdump/profiles-data), in the -S 1 subdirectory layout — see nfcapd setup below

Quick start

No clone needed — grab the compose file and edit it:

curl -O https://raw.githubusercontent.com/mbolli/nfsen-ng/master/deploy/docker-compose.yml

At minimum set NFSEN_SOURCES and NFSEN_PORTS, and make sure the bind-mount and NFSEN_NFDUMP_PROFILES point at your capture directory. For standard setups the environment variables are all you need — no settings.php. The full list of variables lives in Configuration.

When you do want a settings.php (custom filter presets, many sources, a bare-metal path layout): copy backend/settings/settings.php.dist to backend/settings/settings.php, edit it, and mount it read-only. See Configuration → Settings file. Note it must assign the global $nfsen_config array — a file that returns an array is silently ignored.

Deployment modes

The app container (nfsen) listens on port 9000 inside the Docker network only. How you expose it is the choice:

Mode 1 — bundled Caddy (simplest public setup)

deploy/docker-compose.yml ships an optional caddy service gated behind the proxy compose profile. It terminates TLS (automatic HTTPS via Let’s Encrypt, HTTP/3) and reverse-proxies everything to nfsen:9000.

# Edit deploy/Caddyfile.prod first: replace `yourdomain.com` with your domain.
docker compose -f deploy/docker-compose.yml --profile proxy up -d

Access at https://<your-domain> (ports 80, 443, and 443/udp are published).

Caddy here is a pure reverse proxy. It does not compress responses or serve static files — the app does that itself. php-via serves and Brotli-compresses /frontend/* from inside OpenSwoole (withStaticDir() / withBrotli() in backend/app.php). The provided Caddyfile.prod therefore has no encode directive on purpose: SSE streams (the live-update channel) must never be compressed or buffered. It also uses handle_errors 5xx { abort } so Datastar’s SSE client reconnects cleanly across a server restart instead of seeing an error page.

Mode 2 — behind your own reverse proxy

If you already run Traefik, nginx, or your own Caddy, start only the app:

docker compose -f deploy/docker-compose.yml up -d   # just the nfsen service

Point your proxy at http://nfsen:9000 if it shares the Docker network, or uncomment the ports entry in the compose file to publish 9000 on the host:

# deploy/docker-compose.yml — nfsen service
# ports:
#   - "9000:9000"

Requirements for any fronting proxy:

  • Do not compress proxied responses. The app already Brotli/gzip-compresses its own output; double-compression wastes CPU, and compressing the SSE stream breaks live updates. (Only ever compress at the proxy if you strip the app’s compression, which you shouldn’t.)
  • Do not buffer. Disable response buffering / set an immediate flush interval so SSE frames reach the browser as they’re written (proxy_buffering off / X-Accel-Buffering: no / flush_interval -1).
  • Pass 5xx through as a connection abort (or equivalent) so the Datastar SSE client retries on server restart rather than rendering an error page.

Mode 3 — persistent RRD volume + file-based settings

deploy/docker-compose.prod.yml is a variant that always starts Caddy (no profile), mounts the capture directory read-only, keeps RRD files on a named rrd-data volume (via NFSEN_RRD_PATH=/var/nfsen-ng/rrd), and bind-mounts a backend/settings/settings.php. Use it when you prefer a settings file over environment variables and want RRD storage decoupled from the capture mount.

Images and tags

Only one image is published: ghcr.io/mbolli/nfsen-ng, built from deploy/Dockerfile (PHP 8.4 CLI + OpenSwoole + a source-compiled nfdump 1.7.8 + the rrd, inotify, and brotli extensions). The Caddy service uses the stock caddy:latest image — there is no custom Caddy image to build.

TagTracks
latestnewest tagged release (betas included while pre-1.0)
edgenewest master build
x.y.z / x.y / xa specific release, pinnable

Useful commands

docker compose -f deploy/docker-compose.yml logs -f nfsen   # tail app logs
docker compose -f deploy/docker-compose.yml ps              # container status

Option B — Bare-metal (Ubuntu / Debian)

The Docker image is the reference install; the steps below reproduce it by hand. nfsen-ng needs PHP 8.4 with the openswoole, inotify, brotli, and rrd extensions, plus an nfdump binary.

nfcapd setup

nfsen-ng expects nfcapd files in the -S 1 subdirectory layout (YYYY/MM/DD/). A typical invocation for nfdump ≥ 1.7.x:

nfcapd -w /var/nfdump/profiles-data/live/<source> -z=lz4 -S 1 -T all -p <port> -D
FlagMeaning
-w <path>Output directory. Must be <profiles-data>/<profile>/<source> (e.g. .../live/gw1).
-z=lz4Compress capture files (also =lzo, =zstd). The legacy bare -z was removed in nfdump 1.8.x — use the explicit =<algo> form.
-S 1YYYY/MM/DD/ subdirectory structure. Required for nfsen-ng to locate files.
-T allCapture all flow extensions (recommended).
-p <port>UDP listen port (e.g. 9995).
-DDaemonize.

Timezone: nfcapd names files from the host’s local time. If nfsen-ng then runs in a container at TZ=UTC, set NFCAPD_TZ to the capture host’s timezone (e.g. Europe/Berlin) so filenames parse to the correct epoch — otherwise RRD timestamps and chart labels are off by the UTC offset. See Configuration.

The ready-to-use deploy/systemd/nfcapd.service unit already uses -z=lz4 -S 1.

Install the stack

# As root.

# --- PHP 8.4 repository ---
# Ubuntu:
add-apt-repository -y ppa:ondrej/php
# Debian:
apt install -y apt-transport-https lsb-release ca-certificates curl gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list
curl -fsSL https://packages.sury.org/php/apt.gpg | gpg --dearmor > /etc/apt/trusted.gpg.d/sury-php.gpg
apt update

# --- Packages ---
apt install -y git pkg-config brotli \
    php8.4 php8.4-dev php8.4-xml php8.4-mbstring php8.4-curl \
    rrdtool librrd-dev \
    flex bison libbz2-dev zlib1g-dev build-essential autoconf automake libtool unzip wget

# --- nfdump 1.7.8 from source (matches the Docker image) ---
wget https://github.com/phaag/nfdump/archive/refs/tags/v1.7.8.zip
unzip v1.7.8.zip && cd nfdump-1.7.8
./autogen.sh && ./configure --prefix=/usr/local/nfdump && make && make install && ldconfig
cd ..
# binary is now /usr/local/nfdump/bin/nfdump

# --- PHP extensions ---
# openswoole + inotify + brotli via PECL:
pecl install openswoole inotify
echo "extension=openswoole.so" > /etc/php/8.4/mods-available/openswoole.ini
echo "extension=inotify.so"    > /etc/php/8.4/mods-available/inotify.ini
pecl install rrd
echo "extension=rrd.so"        > /etc/php/8.4/mods-available/rrd.ini
# brotli: install ext-brotli (https://github.com/kjdev/php-ext-brotli), then:
echo "extension=brotli.so"     > /etc/php/8.4/mods-available/brotli.ini
phpenmod openswoole inotify rrd brotli mbstring curl xml

# --- nfsen-ng ---
cd /var/www
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data nfsen-ng
cd nfsen-ng

# Composer (https://getcomposer.org/download/):
php composer.phar install --no-dev --optimize-autoloader

# Optional settings file (bare-metal usually wants one):
cp backend/settings/settings.php.dist backend/settings/settings.php
$EDITOR backend/settings/settings.php   # set sources, ports, nfdump.binary, profiles-data

# Start the HTTP server (listens on port 9000):
sudo -u www-data php backend/app.php

On a bare-metal install point nfdump.binary (or NFSEN_NFDUMP_BINARY) at /usr/local/nfdump/bin/nfdump if you compiled it as above. On first start the database is empty — trigger the first import from the web UI (see First import).

Front it with a reverse proxy

The app serves and compresses its own static assets, so a fronting proxy only needs to terminate TLS and forward everything to port 9000 — no static-file root, no encode. The provided deploy/Caddyfile.prod is a complete example; replace yourdomain.com, then point Caddy at it. Any proxy must follow the SSE rules in Mode 2 above (no compression, no buffering, pass 5xx through).


systemd services

Pre-built units are in deploy/systemd/. They reference /var/www/nfsen-ng and eth0 as placeholders — sed-replace those for your host.

FilePurpose
nfsen-ng-docker.serviceRun nfsen-ng via docker compose … --profile proxy up -d (recommended)
nfsen-ng.serviceRun the app directly on bare metal (php backend/app.php, as www-data)
nfcapd.serviceNetFlow capture daemon (-z=lz4 -S 1)
softflowd.serviceSoftware NetFlow exporter for testing/dev
# Docker:
sed -i 's|/var/www/nfsen-ng|/your/path|g' deploy/systemd/nfsen-ng-docker.service
sudo cp deploy/systemd/nfsen-ng-docker.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now nfsen-ng-docker.service

# Bare-metal:
sed -i 's|/var/www/nfsen-ng|/your/path|g' deploy/systemd/nfsen-ng.service
sudo cp deploy/systemd/nfsen-ng.service /etc/systemd/system/
sudo systemctl enable --now nfsen-ng.service

# Capture on the same host (edit eth0 first):
sed -i 's/eth0/YOUR_INTERFACE/g' deploy/systemd/softflowd.service
sudo cp deploy/systemd/nfcapd.service deploy/systemd/softflowd.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now nfcapd.service softflowd.service

Unraid

deploy/unraid/ packages the same single image into two Community Applications roles (web UI + nfcapd collector) sharing one appdata directory, plus a docker-compose.unraid.yml for the Compose Manager plugin. See deploy/unraid/README.md for the walkthrough. The collector role reuses the app image with --entrypoint /usr/local/nfdump/bin/nfcapd; the UI’s NFSEN_SOURCES must include the collector’s source name (both default to flows).


First import

nfsen-ng embeds its import daemon inside the server process — there is no separate CLI binary. On a fresh install with no data yet, nothing is imported automatically: open the app, go to the Settings → Import panel, and click Trigger Import. That scans every nfcapd file back to NFSEN_IMPORT_YEARS and builds the database. Once data exists, each server start does an incremental gap-fill for the files written while it was offline. See Health & Admin for the full panel.

Troubleshooting

No data showing

  • Verify the file layout: <profiles-data>/<profile>/<source>/YYYY/MM/DD/nfcapd.*
  • Confirm the container mount and NFSEN_NFDUMP_PROFILES point at the same tree.
  • Trigger the first import: Settings → Import → Trigger Import.
  • Check RRD files were created under backend/datasources/data/<profile>/.

nfdump warnings: appendix offset error or read() error … Success

These surface in the Statistics/Flows warning toast when nfdump reads a truncated nfcapd file — usually one killed mid-write (container restart, OOM, power loss). nfdump still processes every other file in the range; nothing already imported is lost. To find corrupt files:

docker exec nfsen-ng sh -c 'find /data/nfsen-ng/live -name "nfcapd.*" | \
  xargs -P4 -I{} sh -c "nfdump -r {} -c 1 -q 2>&1 | grep -q error && echo {}"'

Delete the listed files; already-imported RRD/VictoriaMetrics data is unaffected.

SSE not connecting

  • DevTools → Network → filter _sse should show one persistent connection.
  • Confirm the proxy isn’t buffering or compressing the stream (see the proxy requirements above).

Rebuild all data from scratch

Use Settings → Import → Force Rescan in the web UI. This wipes the selected profile’s datasource and re-imports from the capture files. (There is no force-import environment variable; the rescan is a UI action.)

Configuration

nfsen-ng can be configured two ways:

  • Environment variables — the recommended path for Docker. No file needed.
  • A settings.php file — for custom filter presets, unusual layouts, or bare-metal installs.

When a settings file is present it wins: the source/port/filter/processor environment variables are ignored (a warning is logged if you set both). Everything else — nfdump paths, datasource, import depth, NetBox, etc. — is read from the settings array, which itself defaults to the matching environment variable in the shipped template.

Environment variables

Sources & data

VariableDefaultDescription
NFSEN_SOURCES(none)Comma-separated source names, e.g. gw1,router.
NFSEN_PORTS(none)Comma-separated port numbers to track, e.g. 80,443,22.
NFSEN_FILTERS(none)JSON array of nfdump filter presets, e.g. ["proto tcp","dst port 80"].

With a settings.php present, NFSEN_SOURCES, NFSEN_PORTS, NFSEN_FILTERS and NFSEN_PROCESSOR are ignored in favour of the file, and a warning is logged if both are set.

Core

VariableDefaultDescription
NFSEN_SETTINGS_FILEbackend/settings/settings.phpPath to a custom settings file (used only if it exists).
NFSEN_PREFERENCES_FILEbackend/settings/preferences.jsonPath to the persisted user-preferences overlay.
NFSEN_DATASOURCERRDDatasource: RRD or VictoriaMetrics.
NFSEN_PROCESSORNfDumpFlow processor. Only NfDump is implemented.
NFSEN_LOG_LEVELINFOLog verbosity. Accepts DEBUG, INFO, NOTICE, WARNING, ERR/ERROR, CRIT, ALERT, EMERG (and LOG_-prefixed forms). Controls both the app and the Swoole server.
NFSEN_DEV_MODEfalseEnables php-via dev mode (static assets served no-cache). Leave off in production.

nfdump / nfcapd paths

VariableDefaultDescription
NFSEN_NFDUMP_BINARY/usr/local/nfdump/bin/nfdump (env) · /usr/bin/nfdump (settings-file default)Path to the nfdump binary. The Docker image compiles nfdump to /usr/local/nfdump/bin.
NFSEN_NFDUMP_PROFILES/var/nfdump/profiles-dataRoot path to the nfcapd data tree. In Docker this must match the container-side bind-mount (the shipped compose maps it to /data/nfsen-ng).
NFSEN_NFDUMP_PROFILEliveDefault profile subfolder. See Profiles.
NFSEN_NFDUMP_MAX_PROCESSES1Max concurrent nfdump processes (floored at 1).
NFCAPD_TZ(PHP default TZ)Timezone nfcapd used when writing filenames. Set this when nfcapd ran on a non-UTC host and nfsen-ng runs at TZ=UTC — otherwise epoch timestamps are off by the UTC offset. E.g. Europe/Berlin.
TZ(system)The container/process timezone. nfsen-ng also compares it against php.ini in a health check.

Import

VariableDefaultDescription
NFSEN_IMPORT_YEARS3Years of history to scan on import; also sets the depth of the RRD daily archive.
NFSEN_SKIP_INITIAL_IMPORT(off)Set to 1 or true to skip the startup gap-fill and only set up inotify watches.
NFSEN_SKIP_DAEMON(off)Set to 1 or true to disable the embedded import daemon (and periodic alert evaluation) entirely.

Changing NFSEN_IMPORT_YEARS after the first import only affects the RRD daily-archive depth, which is fixed at creation time. To resize it, run Force Rescan from the Settings → Import panel to recreate the RRD files.

There is no NFSEN_FORCE_IMPORT variable — rebuilding is a UI action (Force Rescan). The NFSEN_IMPORT_VERBOSE line that appears commented-out in the shipped compose files is not currently read by the app.

RRD storage

VariableDefaultDescription
NFSEN_RRD_PATHbackend/datasources/dataWhere RRD files are stored. Override to keep them on a separate volume.

VictoriaMetrics

VariableDefaultDescription
VM_HOSTvictoriametricsVictoriaMetrics hostname.
VM_PORT8428VictoriaMetrics HTTP port.

See VictoriaMetrics for the full setup.

NetBox IP lookup

nfsen-ng can enrich private/reserved IPs in the Flows view with metadata from a NetBox IPAM instance. When configured, clicking such an address opens a modal with its NetBox description, tenant, VRF, role, and status. Only private/reserved ranges are looked up; public IPs fall through to the geo lookup.

VariableDefaultDescription
NFSEN_NETBOX_URL(empty)Base URL of your NetBox instance, e.g. https://netbox.example.com.
NFSEN_NETBOX_TOKEN(empty)Read-only NetBox API token.

Both are also settable as general.netbox_url / general.netbox_token in settings.php. The integration is disabled while either value is empty.

Alert email

VariableDefaultDescription
NFSEN_ALERT_EMAIL_FROM(empty)From-address for alert emails. Setting it enables the email alert channel; leaving it empty disables email delivery. Also settable as general.alert_email_from.

OpenSwoole server

VariableDefaultDescription
SWOOLE_WORKER_NUM1Worker processes. Raise toward CPU-core count for CPU-bound loads.
SWOOLE_MAX_REQUEST0Requests per worker before restart. 0 (unlimited) is correct for a long-lived SSE server.
SWOOLE_MAX_COROUTINE10000Max concurrent coroutines / SSE connections.

The comments in the shipped compose files quote different Swoole defaults (4 / 10000); the values above are what the code actually defaults to.

Settings file

For custom filter presets or a bare-metal path layout, copy the template and edit it:

cp backend/settings/settings.php.dist backend/settings/settings.php

The file must assign the global $nfsen_config array — nfsen-ng includes it and reads that global. A file that returns an array (or defines any other variable) is silently ignored and you get empty settings. The template’s own values default to getenv(), so a settings file and environment variables can coexist.

<?php
$nfsen_config = [
    'general' => [
        'sources' => ['gw1', 'router'],   // nfcapd source names
        'ports'   => [80, 443, 22],        // ports to track in RRD
        'filters' => ['proto tcp', 'dst port 80'], // nfdump filter presets
        'db'      => 'RRD',                // 'RRD' or 'VictoriaMetrics'
        'processor' => 'NfDump',
        'max_stats_window' => 0,           // seconds; 0 = unlimited
        'netbox_url'   => '',
        'netbox_token' => '',
    ],
    'nfdump' => [
        'binary'        => '/usr/local/nfdump/bin/nfdump',
        'profiles-data' => '/var/nfdump/profiles-data',
        'profile'       => 'live',
        'max-processes' => 1,
    ],
    'db' => [
        'RRD'            => ['data_path' => null, 'import_years' => 3],
        'VictoriaMetrics'=> ['host' => 'victoriametrics', 'port' => 8428, 'import_years' => 3],
    ],
    'log' => ['priority' => \LOG_INFO],
];

Key reference (the template ships more, including frontend.defaults.* UI defaults):

KeyMeaningDefault in template
general.sourcesnfcapd source names (string[])['source1','source2']
general.portsPorts to track (int[])[80, 22, 53]
general.filtersnfdump filter presets shown in the UI (string[])a starter set
general.dbDatasource class namegetenv('NFSEN_DATASOURCE') ?: 'RRD'
general.processorFlow processor'NfDump'
general.max_stats_windowStatistics query cap in seconds (0 = unlimited)0
general.netbox_url / general.netbox_tokenNetBox lookupempty
general.alert_email_fromAlert email From-address(not in template)
nfdump.binarynfdump path/usr/bin/nfdump
nfdump.profiles-dataCapture data root/var/nfdump/profiles-data
nfdump.profileDefault profilelive
nfdump.max-processesMax concurrent nfdump procs1
db.RRD.data_pathRRD storage dir (null = default)null
db.<datasource>.import_yearsYears to import/retain3
log.prioritySyslog level constant\LOG_INFO

Note the key spelling: nfdump.profiles-data and nfdump.max-processes use dashes; general.max_stats_window, general.netbox_url, and general.netbox_token use underscores. import_years is read from under the sub-key that matches general.db (e.g. db.RRD.import_years).

Timezones

nfcapd names files using the local time of the host running it. nfsen-ng parses those filenames back to epochs to place data on the timeline. If the two run in different timezones (classic case: bare-metal nfcapd in CEST, nfsen-ng in a TZ=UTC container), set NFCAPD_TZ to the capture host’s IANA timezone. When unset, nfsen-ng falls back to PHP’s effective timezone. Getting this wrong shifts every RRD timestamp and chart label by the UTC offset.

RRD data retention

RRD files are independent of nfcapd files. Deleting old capture files does not touch RRD graphs, and vice versa. See Data Sources for the conceptual split.

nfsen-ng creates one fixed-size .rrd file per source (and per tracked port). The size is set at creation and never grows. Each file holds four resolutions (each stored as both an AVERAGE and a MAX archive):

ResolutionRetention
5-minute samples45 days
30-minute samples90 days
2-hour samples1 year
1-day samplesNFSEN_IMPORT_YEARS years (default 3)

Only the daily archive’s depth changes with NFSEN_IMPORT_YEARS; the finer three are fixed.

Disk space

Because the fine-resolution archives dominate and are fixed, every .rrd file is roughly the same size — about 5 MiB — whether it’s an aggregate or a per-port file, and almost independent of NFSEN_IMPORT_YEARS (the daily archive is a small fraction of the total). Budget accordingly:

  • ~5 MiB per source (aggregate)
  • ~5 MiB per tracked port, per source

So 10 sources × 5 tracked ports ≈ 60 files ≈ ~300 MB total. This is a flat, predictable cost — RRD files are circular buffers that never grow with traffic volume.

Changing retention depth

To store more (or fewer) years of daily data, set NFSEN_IMPORT_YEARS and then recreate the RRD structure with Settings → Import → Force Rescan (the server also logs a warning on start if it detects a size mismatch):

# docker-compose.yml
environment:
  - NFSEN_IMPORT_YEARS=5   # 5 years of daily graph data
  # then run Force Rescan once from the web UI to rebuild the RRD files

RRD vs nfcapd files

RRD (.rrd)nfcapd (nfcapd.*)
PurposeGraph counters (flows/packets/bytes)Raw flow records for Flows & Statistics
SizeFixed (~5 MiB/file)Grows with traffic
Removed with nfcapd files?No — independent
Flows/Stats work without them?No — nfdump reads them directly
Retention controlNFSEN_IMPORT_YEARSManage separately (e.g. cron + find -mtime)

nfdump Profiles

nfsen-ng supports multiple nfdump profiles (e.g. live, test, backup). Each profile is an independent tree of nfcapd capture files under nfdump.profiles-data, with its own separate RRD (or VictoriaMetrics) dataset. The conceptual model is covered in Data Sources → Profiles; this page is the operational side.

Directory layout

<profiles-data>/
├── live/
│   └── <source>/
│       └── YYYY/MM/DD/nfcapd.YYYYMMDDHHMM
└── test/
    └── <source>/
        └── YYYY/MM/DD/nfcapd.YYYYMMDDHHMM

Profiles are detected automatically: Config::detectProfiles() scans <profiles-data> and treats each top-level directory that contains a source → YYYY/ tree as a profile. A directory that instead holds sub-groups of such trees becomes a nested group/child profile. The nfdump.profile setting (NFSEN_NFDUMP_PROFILE, default live) only picks the default profile shown on load. If the data root is missing or empty, nfsen-ng falls back to a single profile named after nfdump.profile.

Configuration

No extra configuration is required — detection is automatic on startup. To run a second collector writing into another profile, just point its -w at a different top-level directory:

# Live collector — port 9995 → profiles-data/live/all/
nfcapd -w /var/nfdump/profiles-data/live/all -z=lz4 -S 1 -T all -p 9995 -D

# Test/secondary collector — port 9996 → profiles-data/test/all/
nfcapd -w /var/nfdump/profiles-data/test/all -z=lz4 -S 1 -T all -p 9996 -D

Docker Compose example

nfcapd-test:
  image: ghcr.io/mbolli/nfsen-ng:latest
  entrypoint: ["/usr/local/nfdump/bin/nfcapd"]
  command: ["-w", "/data/nfsen-ng/test/all", "-z=lz4", "-S", "1", "-T", "all", "-p", "9996"]
  volumes:
    - /var/nfdump/profiles-data:/data/nfsen-ng
  ports:
    - "9996:9996/udp"

Profile selector

When more than one profile is detected, a profile selector appears in the page header. Switching profiles re-scopes the whole UI to that profile’s data range and slides the visible window to the end of its available data; the choice is persisted to preferences.json so it survives a reload.

Import daemon

The embedded import daemon watches each detected profile independently — the Settings → Import panel shows a per-profile row, and the footer daemon indicator reports how many profiles are being watched. Each profile:

  • watches its own <profiles-data>/<profile>/ subtree via inotify;
  • writes into profile-namespaced storage (data/<profile>/<source>.rrd, or a profile= label for VictoriaMetrics);
  • decides on startup whether to gap-fill based on its own last-update timestamp. A profile with no data yet is skipped — trigger its first import manually with the per-row Trigger button.

Storage layout

RRD files are stored per profile:

backend/datasources/data/<profile>/<source>.rrd
backend/datasources/data/<profile>/<source>_<port>.rrd
backend/datasources/data/<profile>/<port>.rrd        # port-only aggregate

A .rrd.first sidecar (e.g. all.rrd.first) records the Unix timestamp of the first real data point, so the date-range slider starts at the true beginning of available data rather than the RRD creation time.

Force Rescan is per profile. The Rescan action reads the targeted profile (admin_target_profile) and resets only that profile’s datasource, leaving the others untouched.

For VictoriaMetrics the profile is a Prometheus label (profile="live"). Historical data written before profiles existed has no label and is still matched via a regex selector (profile=~"live|") for backward compatibility.

Health checks

The health panel reports per-profile status: nfcapd path existence and capture freshness, daemon status and last auto-import, and datasource freshness. With a single profile the check ids and labels stay flat (rrd_data_<source>); with more than one they’re profile-suffixed (rrd_data_<profile>_<source>) so the two don’t collide. See Health & Admin.

VictoriaMetrics

VictoriaMetrics is an optional alternative to the default RRD datasource — useful when you want parallel/out-of-order imports, longer retention, or PromQL/MetricsQL for ad-hoc queries. The architectural comparison is in Data Sources; this page is the setup guide.

Trade-offs vs RRD

RRDVictoriaMetrics
Extra infrastructurenone (a PHP extension)a separate VM service
Disk footprintfixed ~5 MiB per source/port filegrows with retention (typically tens of MB+)
Parallel / out-of-order writesnoyes
Query languageRRDtoolMetricsQL (PromQL-compatible)
HTTP query APInoyes
Setupsimplemoderate

Setup

1. Start VictoriaMetrics

deploy/docker-compose.victoriametrics.yml brings up a full stack — victoriametrics, nfsen, and caddy — with the app already pointed at VM:

docker compose -f deploy/docker-compose.victoriametrics.yml up -d

VM listens on 8428; the UI is fronted by Caddy. To add VM to an existing deployment instead, run a victoriametrics/victoria-metrics container reachable from the app and set the variables below.

2. Point nfsen-ng at it

Via environment variables (recommended):

VariableDefaultDescription
NFSEN_DATASOURCERRDSet to VictoriaMetrics to activate.
VM_HOSTvictoriametricsVictoriaMetrics hostname.
VM_PORT8428VictoriaMetrics HTTP port.
NFSEN_IMPORT_YEARS3Lookback window for import and boundary queries.

Or with a settings file (backend/settings/settings.victoriametrics.dist is a ready template). Remember the file must assign the global $nfsen_config array, not return one:

<?php
$nfsen_config = [
    'general' => [
        'sources' => ['gw1', 'router'],
        'ports'   => [80, 443, 22],
        'db'      => 'VictoriaMetrics',
    ],
    'db' => [
        'VictoriaMetrics' => [
            'host' => 'victoriametrics',
            'port' => 8428,
            'import_years' => 3,
        ],
    ],
    // ... nfdump paths, log, etc.
];

3. Import

On a fresh install, run the first import from Settings → Import → Trigger Import in the web UI. To rebuild from scratch, use Force Rescan. nfcapd files are still required — VictoriaMetrics is a storage backend, not a replacement for the raw captures nfdump reads for Flows and Statistics.

Data model

nfsen-ng writes Prometheus-exposition samples to http://<host>:<port>/api/v1/import/prometheus (millisecond timestamps) and queries them back through VM’s HTTP API (/api/v1/query, /api/v1/query_range, tfirst_over_time/tlast_over_time).

Metric names are nfsen_<type> for type in flows, packets, bytes, with a protocol suffix for the per-protocol series:

nfsen_flows{source="gw",profile="live"}                 12345   # aggregate (no port label)
nfsen_flows{source="gw",port="80",profile="live"}       4321    # port-specific
nfsen_flows_tcp{source="gw",protocol="tcp",profile="live"} 8000 # TCP only
nfsen_bytes{source="gw",profile="live"}                 9876543
nfsen_packets{source="gw",profile="live"}               67890

Labels:

  • source — the source name.
  • port — port number string; absent on aggregate totals. On the read path the aggregate is selected with port="", which matches series where the label is absent (so sparse per-port series don’t shadow the dense aggregate).
  • protocoltcp/udp/icmp/other on the per-protocol series (redundant with the metric-name suffix); absent on the aggregate.
  • profile — the nfdump profile, e.g. profile="live". Data written before profiles existed has no label and is matched on read via profile=~"live|".

Real-time updates

RRD relies on inotify to detect new files directly. With VictoriaMetrics — a remote database — the embedded import daemon instead broadcasts a re-render to all SSE clients after each successful write, and a VictoriaMetricsWatcher polls VM’s health/readiness so the Admin panel can report connectivity rather than just config sanity. No browser polling is involved either way.

Migrating between RRD and VictoriaMetrics

  1. Change general.db (or NFSEN_DATASOURCE) to the target datasource.
  2. Run Force Rescan to populate it from the nfcapd files.
  3. The two datasources are independent — switching to VM does not touch existing .rrd files, and you can roll back by setting the datasource to RRD again. Remove the now-unused backend/datasources/data/*.rrd only after verifying the VM data is complete.

Seeding demo / development data

scripts/seed_vm_data.php pushes realistic synthetic NetFlow data into VM — handy for demos, screenshots, and testing without a live nfcapd source. (The RRD equivalent is scripts/seed_rrd_data.php.)

php scripts/seed_vm_data.php [--host=localhost] [--port=8428] [--source=all] [--days=90] [--ports=80,443]
OptionDefaultDescription
--hostVM_HOST env, else localhostVictoriaMetrics hostname
--portVM_PORT env, else 8428VictoriaMetrics port
--sourceallsource= label value
--days90Days of history to generate
--ports(none)Comma-separated ports to also emit, e.g. 80,443,22

It generates diurnal traffic with a realistic protocol mix (TCP 63 % / UDP 22 % / ICMP 4 % / Other 11 %), weekend dips (~30 % quieter), and ±15 % Gaussian noise — peaking around 25 000 flow events per 5-minute slot at the busiest hour — using the same metric names and labels VictoriaMetrics::write() produces, so the output shows up immediately in the graphs. Unlike the RRD seeder (which needs the app’s config and an RRD datasource, so it runs inside the container), this script only needs PHP and network reach to VM.

Warning: only run this against a development or demo VictoriaMetrics instance — it writes a large volume of fake flow events.

Upgrading from v0

v1 is a ground-up rewrite. It is not code-compatible with the v0.x (NfSen-style) releases, but it reuses the same underlying data: nfsen-ng reads the nfcapd capture files nfdump already writes, so no data migration is needed — point v1 at your existing capture tree and run an import.

What changed

Architecture

v0.xv1
Apache/nginx + PHP-FPM (per-request)OpenSwoole via php-via (one persistent process)
REST JSON API + AJAX pollingHypermedia over SSE (Datastar) — no JSON API
jQuery frontendServer-rendered Twig + Datastar signals; no client-side routing or build step
RRD onlyRRD (default) or VictoriaMetrics
No live pushinotify → SSE broadcast to every open tab

See Architecture → Overview for how the v1 pieces fit together.

Frontend

  • jQuery, ion.rangeSlider, and the old REST client are gone.
  • Replaced by Datastar, noUiSlider, and Apache ECharts for graphs (v1 migrated the Graphs tab off Dygraphs).
  • No client-side routing — the server pushes full HTML re-renders over SSE and Datastar morphs the DOM.

Backend

  • The entry point is now backend/app.php (the OpenSwoole server), not a web server document root.
  • The cli.php interface was removed. Import is driven from the web UI (Settings → Import → Trigger Import / Force Rescan) by the daemon embedded in app.php.
  • Settings still live in a PHP file that assigns the global $nfsen_config array, but the schema was expanded and reorganised (new general.db, db.<datasource>.*, frontend.defaults.*, and more). Start from the current backend/settings/settings.php.dist rather than reusing a v0 file verbatim — or skip the file entirely and configure via environment variables. See Configuration.

Docker

  • v0 ran Apache inside the container; v1 runs the OpenSwoole app and fronts it with a stock caddy:latest container (optional, behind the proxy profile) — there is no custom Caddy image.
  • Deployment layout moved under deploy/ (docker-compose.yml, docker-compose.dev.yml, …). See Installation.

Migration steps

  1. (Optional) Back up your old RRD files, in case you want to keep the v0 graph history around:

    cp -r backend/datasources/data/ /backup/rrd-$(date +%Y%m%d)/
    
  2. Deploy v1. The simplest path is the published Docker image (ghcr.io/mbolli/nfsen-ng:latest) with deploy/docker-compose.yml. If you run from source, check out a v1.0.0-* release tag (or the v1 branch) rather than the old v0 tags. Full steps: Installation.

  3. Point v1 at your existing capture tree — set NFSEN_NFDUMP_PROFILES (or nfdump.profiles-data) to the same profiles-data directory nfcapd already writes to, and list your sources in NFSEN_SOURCES.

  4. Review configuration. Map any custom v0 settings onto the current keys or environment variables (Configuration).

  5. Build the graph data. Run Settings → Import → Trigger Import (or Force Rescan) once. This rebuilds the RRD/VictoriaMetrics graph data from your nfcapd files — the v1 RRD structure differs from v0’s, so re-importing from the captures is the reliable path rather than reusing old .rrd files. Flows and Statistics work directly off the capture files and need no import.

Known differences from v0 / NfSen

  • The v0 REST API endpoints (/api/…) no longer exist — v1 has no JSON API.
  • NfSen’s alert/plugin mechanisms are not carried over; v1 has its own alerting.
  • Some v0 profile-filter behaviour differs; v1’s profile model is auto-detected from the capture tree (Profiles).

What is NetFlow?

NetFlow (and its open standard successor, IPFIX) is a way routers and switches summarize the traffic passing through them. Instead of recording every packet, a network device groups packets into flows — one flow per unique combination of source IP, destination IP, source port, destination port, and protocol — and periodically exports a compact record for each flow: how many packets, how many bytes, when it started, when it ended.

A single flow record looks roughly like:

192.168.1.50:52184 → 74.50.105.85:443, TCP, 85 packets, 10.7 KB, lasted 4m48s

That’s it — not the actual content of the traffic (no payload, no URLs, no message bodies), just the shape of the conversation. This is what makes NetFlow practical at scale: a busy link might carry gigabits of traffic per second, but the flow records summarizing it are a tiny fraction of that volume.

What problem does it solve?

Without flow data, answering basic questions about your network requires either expensive full packet capture (storage-hungry, and often legally/privacy sensitive) or nothing at all. With flow export enabled on your routers, you can answer things like:

  • “What’s using all our bandwidth right now?” — rank talkers by bytes.
  • “Did anything unusual happen last Tuesday at 3am?” — go back in time, something full packet capture rarely lets you do affordably.
  • “Is this subnet talking to that one, and how much?” — the Sankey diagram answers exactly this, visually.
  • “Alert me if ICMP traffic to this network spikes” — see Setting Up Alerts.

Where nfsen-ng fits

nfsen-ng doesn’t capture NetFlow itself — that’s nfcapd’s job (part of the nfdump suite), a small daemon that listens for NetFlow/IPFIX/sFlow packets sent by your routers and writes them to disk in 5-minute-rotated files. nfsen-ng is the web UI on top: it reads those files (via nfdump, the query tool), aggregates them into graphs, lets you search and filter individual flows, and can alert you when traffic crosses a threshold you define.

If you already have nfcapd running and writing files somewhere, nfsen-ng just needs to be pointed at that directory — see Quick Tour to get oriented, or the Developer Reference’s Getting Started page if you’re setting up nfcapd/nfdump for the first time.

Quick Tour

When you open nfsen-ng, you land on the Graphs tab — a live traffic chart for whatever’s currently being captured.

The five tabs

The navigation bar

Across the top: a home/reload icon, a dark-mode toggle (moon/sun), a reconnect spinner (only appears if your connection to the server drops — see below), and the five tabs: Graphs, Flows, Statistics, Sankey, and Settings. Whichever preset (source) you’re viewing shows in the top-left corner.

Switching tabs is instant — there’s no page reload, and whatever filters you had set on a tab are still there when you come back to it.

The shared control bar

Every data tab (Graphs, Flows, Statistics, Sankey) shares the same date range control at the top, plus a filter panel specific to that tab below it:

Date range and filter panel

  • Date range slider — drag either handle, or use the quick-select buttons (1 hour / 24 hours / Week / Month / Year) to jump to a common window. The / buttons step back/forward by the current window’s length (e.g. clicking Week then → jumps forward exactly one week); jumps to “now”.
  • Custom duration — type a number, pick h/d/w, and click Apply for anything the presets don’t cover (e.g. “6 hours”).
  • Sync now / Follow — on the Graphs tab specifically, Follow keeps the window pinned to “now” as new data arrives, so the chart scrolls automatically.

Changing the date range or any filter on the Graphs tab updates the chart immediately. Flows, Statistics, and Sankey are different — querying flow records runs the real nfdump tool, which costs real time and I/O, so those tabs only run a query when you click their Process data button. Nothing happens automatically until you do.

If you see a “Reconnecting…” banner

nfsen-ng pushes live updates to your browser over a persistent connection (Server-Sent Events). If that connection drops — your laptop slept, a proxy timed out, the server restarted — you’ll see a small spinner next to the dark-mode toggle and a banner at the top of the page. It reconnects automatically; you don’t need to reload, though a manual reload (the home/reload icon) never hurts if it seems stuck.

Where to go next

The Dashboard

The Graphs tab is a live time-series chart of your traffic — the first thing you see, and the one tab that updates itself without you clicking anything.

Graphs tab, light and dark

Reading the chart

The chart title (“traffic for source all”) and the LIVE badge above it tell you what’s plotted and confirm you’re looking at current data — the timestamp next to it is the last time new data arrived. As new nfcapd files land, the chart extends itself automatically; no refresh needed.

Choosing what to plot

The filter panel above the chart controls what you see:

ControlWhat it does
DisplayGroup the chart by Sources (one line per exporter), Protocols (one line per protocol), or Ports (one line per tracked port)
SourcesWhich exporter(s) to include — pick specific ones, or all
ProtocolsFilter to TCP / UDP / ICMP / Others, or Any
Data typeTraffic (bytes), Packets, or Flows
UnitBits or Bytes (only applies to Traffic)

Below the chart, a few display-only controls let you adjust how it’s rendered without re-querying anything: number of data points (resolution vs. render cost), linear vs. logarithmic scale, stacked vs. line series, and step vs. curve interpolation.

Zooming in

Drag across the chart itself to zoom into a specific window — the date range slider above updates to match. Use the slider’s / buttons or drag its handles to move the window instead of re-selecting on the chart every time.

What’s next

Once you spot something interesting on the graph — a spike, an unfamiliar protocol share — the natural next step is Browsing Flows or Statistics for the same time window, to see exactly which conversations made up that traffic.

Browsing Flows

The Flows tab lists individual flow records for a time window — the detail view behind the aggregate charts. Use it when you know roughly when something happened and want to see exactly what.

Flows tab with real results

Running a query

Unlike the Graphs tab, Flows doesn’t query automatically — set your date range and filters, then click Process data. This runs the real nfdump tool against your capture files, which is shown to you verbatim above the results table (handy for confirming exactly what was asked for, or for copy-pasting into a terminal if you want to run the same query outside the UI). A Kill button appears next to it while a query is running, in case you asked for more than you meant to.

Filters

ControlWhat it does
Limit flowsCap on how many records come back
SourcesWhich exporter(s) to include
nfdump filterFree-text nfdump filter syntax, e.g. proto tcp and dst port 443
Min / max bytesOnly show flows within a byte-count range

If you don’t already know nfdump’s filter syntax, start simple — proto icmp, net 192.168.1.0/24, dst port 22 — and combine with and/ or as needed. Save anything you use often as a filter preset (see Preferences) so it’s a dropdown pick next time instead of retyped text.

Aggregation & output

For summarizing rather than listing every raw flow, the aggregation panel combines matching flows together:

Aggregation & output panel

  • Global: combine both directions of a conversation into one row (Bi-directional), and/or collapse by protocol.
  • Port: collapse by source port and/or destination port.
  • IP Aggregation: collapse source/destination addresses down to a subnet (e.g. a /24) instead of listing every individual host.
  • Options: order results by start time.

These map directly onto how nfdump itself aggregates flows — if you already know nfdump’s -a/aggregation flags, this panel is that, with a form around it.

Looking up an address

Click any IP address in the results table to see where it is and who it belongs to — see Looking Up an IP.

Statistics

The Statistics tab answers “who are the top N?” — the busiest hosts, ports, or protocols for a given time window, ranked and totalled, rather than a flow-by-flow list.

Statistics tab with real results

Running a query

Same pattern as Flows: set your filters, click Process data, and the exact nfdump command used is shown above the results so you know precisely what was measured.

What can you rank by?

The Statistic for dropdown is the key control — it decides what each row of the results represents:

Statistic forAnswers
Flow RecordsTop individual flows
Any / Src / Dst IP addressBusiest hosts, overall or by direction
Any / Src / Dst portBusiest ports
Any / Src / Dst ASBusiest autonomous systems (if AS data is present)
Any / In / Out interfaceBusiest router interfaces
ProtoTraffic split by protocol
Src / Dst TOSTraffic split by ToS/DSCP marking

Combine with Order by (rank by bytes, packets, or flows) and Top records (how many rows to return) to get, say, “top 10 destination IPs by bytes in the last week.”

Same filters as Flows

Sources, nfdump filter, min/max bytes work identically to the Flows tab — anything you can filter there, you can filter here, just aggregated into ranked totals instead of a flat list.

Clicking an IP address in the results opens the same lookup modal as Flows — see Looking Up an IP.

Sankey Diagram

In development — the Sankey tab is still being built out (tracked as issue #152); expect rough edges.

The Sankey tab visualizes who talks to whom: each band is a source→destination pair, sized by bytes or packets, so the biggest conversations are immediately obvious as the widest bands.

Sankey tab with real traffic

When to reach for this instead of Statistics

Statistics ranks addresses independently — “top 10 source IPs” and “top 10 destination IPs” are two separate lists. Sankey shows the relationship between them in one picture: which sources are driving traffic to which destinations, at a glance, without cross-referencing two tables yourself.

Running a query

Same pattern as Flows and Statistics: set your date range and filters, click Process data. The extra control here is Rank / size by — Bytes or Packets — which decides how wide each band is drawn. Top pairs caps how many source→destination pairs appear; past a certain count the diagram gets crowded rather than more useful, so start small (10–20) and widen only if you need to.

Filters (sources, nfdump filter expression, min/max bytes) work the same as every other data tab.

Adding the port dimension

Sankey with the destination-port middle column enabled

Flip on Ports (Show dst port) to slot the destination L4 port between the two IP columns: src IP → dst port → dst IP. This answers what service a conversation is using, not just who talks to whom — e.g. seeing that one host’s traffic to a server is all 443 while another host’s is 22. The middle column pools traffic per port, so a busy port like 443 shows up as one thick node fed by every source and fanning out to every destination that used it. Because the port joins the aggregation key, Top pairs now caps src/port/dst tuples rather than plain pairs, so bump it up a little if a busy port thins out the view.

Looking Up an IP

Any IP address shown as a link in the Flows or Statistics tables — click it, and a popup gives you context on that address without leaving the page or opening a new tab.

IP info popup, light and dark

What you get

  • Hostname — reverse DNS, if the address resolves to one.
  • For a public address: geolocation — city, region, country, timezone — plus network ownership info (ASN, organization). Useful for a quick “is this a cloud provider, a CDN, or somewhere unexpected?” sanity check on an unfamiliar destination.
  • For a private address (RFC 1918, e.g. 192.168.x.x, 10.x.x.x) — geolocation obviously doesn’t apply, so instead you get whatever your organization’s Netbox IPAM has on record for it, if your administrator has connected one (see Preferences for how that’s configured).

A word of caution

The geolocation lookup calls an external service (ipapi.co) over the internet — it only fires for public addresses, and only when you actually click one, not automatically for every row in a table. If your nfsen-ng instance has no outbound internet access, that part of the popup will simply come back empty; reverse DNS and Netbox lookups (if configured) are unaffected.

Setting Up Alerts

Alerts watch a metric (flows, packets, or bytes) and notify you when it crosses a threshold — checked automatically every time new traffic data comes in, no need to keep the dashboard open.

Find them under Settings → Alerts:

Alerts screen

Creating a rule

Fill in the New Alert Rule form:

  1. Rule name — anything memorable, e.g. “High traffic on gw1”.
  2. Profile — which nfdump profile this rule watches (usually live).
  3. Condition — pick a Metric (Bytes/s, Packets/s, Flows/s), an Operator (>, >=, <, <=), a Threshold type, and a Value:
    • Absolute value — fire when the metric crosses this number directly.
    • Percent of average — fire when the metric is this percent above/below its own rolling average (choose the averaging window separately). Use this for “alert me when traffic is unusually high for this network” rather than a fixed number that might be normal for one link and alarming for another.
  4. Cooldown — how many 5-minute cycles to wait before this rule is allowed to fire again, so a sustained spike doesn’t flood you with repeat notifications.
  5. Notifications — an email address and/or a webhook URL. Leave both blank for an in-app-only alert (visible in the Recent Alert History list below the form).

Click Create rule when you’re done. Existing rules appear in the table above, where you can enable/disable, delete, or test-fire each one on demand with the Test button.

Scoping an alert to specific traffic

By default, a rule watches your total traffic for its metric. Often you want something narrower — “alert only on ICMP”, “alert only for this subnet.” That’s what the Traffic filter field is for:

Traffic filter field

Type any nfdump filter expression — the same syntax as the Flows tab’s filter field:

You want to watchTraffic filter
Only ICMP trafficproto icmp
Only one subnetnet 192.168.1.0/24
Traffic from one subnet, TCP onlysrc net 10.0.0.0/8 and proto tcp

When this field is set, the rule runs a real (small, fast) nfdump query over the most recent data instead of using pre-aggregated totals — so it can scope to exactly the traffic you described, at the cost of being slightly more expensive to evaluate than an unfiltered rule. Leave it blank for “total traffic,” which is cheaper and is all you need for a general high-traffic alert.

If you type something nfdump doesn’t understand, the rule simply doesn’t fire (rather than erroring loudly) — double-check the syntax with the Flows tab first if a filtered alert never seems to trigger.

Customizing the notification text

By default, email and webhook notifications use a fixed subject/title and body/message. You can override this — useful for phrasing Gotify or Apprise notifications your own way, or inserting the actual traffic numbers into the message instead of just getting a generic “an alert fired.”

There are two levels, checked in order: a rule’s own override (if set), then a global default (if set), then the built-in text shown above as each field’s placeholder.

Global defaults apply to every rule that doesn’t set its own override. Expand Default Notification Templates at the top of the Alerts page:

Default notification templates

Per-rule overrides live in each rule’s own form, collapsed behind a Customize email message / Customize webhook message toggle next to the Email address / Webhook URL fields:

Per-rule template override

Either way, click into a template field, then click one of the variable buttons to insert it at your cursor:

VariableValue
{rule}Rule name
{metric}Which metric fired (flows, packets, or bytes)
{value}That metric’s current value
{threshold}The threshold that was crossed
{operator}The comparison operator (>, >=, <, <=)
{condition}{metric} {operator} {threshold}, combined
{flows}, {packets}, {bytes}All three counters, regardless of which one the rule watches
{profile}The nfdump profile
{sources}The rule’s sources, comma-separated
{time}When the alert fired (UTC)

The Preview box below each pair of fields updates live as you type, using made-up example numbers — it’s a preview of the template, not a real fired alert. To see the real, resolved text for a specific rule, use the Test button (its result includes the actual rendered title/message alongside the fire/no-fire verdict).

Testing before you rely on it

Click Test on any rule to evaluate it immediately against current data and see whether it would fire — without waiting for the next automatic cycle, and without needing to actually breach the threshold for real. Good practice after creating or editing a rule, especially one with a traffic filter, before trusting it to notify you unattended.

Admin & Health

Two screens under Settings are for keeping nfsen-ng itself running smoothly, rather than analyzing traffic: Import (manual scan controls) and Health (a status check of the whole setup).

Health: is everything OK?

Settings → Health runs down PHP requirements, timezone configuration, the nfdump binary, your configured sources, the import process, and your storage backend — each marked ok / warning / error, with a note on what to do about anything that isn’t green.

Health screen

If something’s wrong, the entries are grouped so you can jump straight to the relevant area rather than reading the whole list. For example, the nfdump group:

nfdump health group detail

  • nfdump binary / Minimum version — confirms the tool nfsen-ng shells out to actually exists and is new enough.
  • Max processes — how many nfdump queries are allowed to run at once (see Preferences for where this is set).
  • Process inspection — a quieter but important one: this confirms the system actually can count how many nfdump processes are running (via ps/pgrep). If this shows a warning, the “Max processes” limit above it isn’t being enforced at all — worth fixing before you rely on it to keep a busy instance from running too many queries at once.

Other groups cover PHP extensions, timezone plausibility (does the most recent capture file’s timestamp look sane?), your configured capture directories, and the storage backend (RRD or VictoriaMetrics).

Import: manual control over the capture pipeline

Settings → Import shows, per profile: whether the import daemon is running, when it last auto-imported new data, and how many directories it’s watching for new files.

Import screen

Normally you never touch this — new nfcapd files are picked up and imported automatically as they’re written. Two buttons exist for when you do need to step in:

  • Trigger — re-run the catch-up import for a profile. Use this if you’ve just pointed nfsen-ng at a directory with existing historical data it hasn’t seen yet, or you suspect it missed something.
  • Rescan — resets that profile’s stored data and re-imports everything from scratch. This is destructive (it discards existing aggregated data for the profile first) and asks for confirmation — reach for it only if the data looks genuinely wrong and a normal Trigger doesn’t fix it.

Both show progress live, and can be cancelled mid-way if you change your mind.

Preferences

Settings → Preferences holds everything you can personalize day to day — as opposed to System (right next to it), which just displays how this instance was deployed and can’t be changed from the browser at all.

Preferences screen

What you can change here

  • Default view — which tab opens when you load nfsen-ng.
  • Log level — how chatty the server’s own logs are. Leave on the default unless you’re troubleshooting something and were asked to turn it up.
  • Graph defaults — the Display/Datatype/Protocols the Graphs tab starts with, so you don’t have to reselect them every visit.
  • Flow & statistics defaults — default row limit and sort order for Flows/Statistics.
  • Date & time display — show timestamps in your browser’s local timezone, or the server’s. Handy if you’re monitoring a network in a different timezone than the one you’re sitting in.
  • Filter presets — a saved list of nfdump filter expressions, offered as quick picks in the Flows/Statistics/Sankey filter panels instead of retyping the same filter every time. One per line.

Click Save at the bottom of the section you changed — each section saves independently.

Everything else (System)

The System sub-tab is read-only: configured sources and ports, which storage backend is active, how many years of history are imported, the nfdump binary path, and so on. These come from environment variables or a config file at deploy time and need a container/service restart to change — this screen exists so you (or whoever you ask for help) can see exactly what an instance is configured with, without needing shell access to it.

Overview

nfsen-ng has three moving layers, all inside a single long-running PHP process:

nfcapd (external)              → writes rotated capture files to disk
      │  inotify
      ▼
ImportDaemon (backend/common)  → parses new files, writes to the datasource,
      │                           evaluates alert rules
      ▼
Datasource (Rrd|VictoriaMetrics)
      │
      ▼
Signals + Actions + SSE (php-via / Datastar) → browser

Stack

LayerTechnology
RuntimePHP 8.4 on OpenSwoole coroutines
Web frameworkphp-via — signals, actions, SSE, in-house
ReactivityDatastar — server-driven DOM patching over SSE
TemplatesTwig
Flow decodingnfdump CLI, invoked as a subprocess
StorageRRD (default) or VictoriaMetrics, pluggable per the Datasource interface
ChartsApache ECharts (time-series graphs and Sankey diagram)

Why one long-running process

Unlike classic PHP-FPM, backend/app.php is started once and stays running: OpenSwoole’s HTTP server, the SSE broadcaster, and the import daemon’s inotify watch all live in the same process’s memory for as long as it’s up. This is what makes the reactive loop cheap — signals and their subscribers are plain PHP objects, not something serialized to a session store between requests — but it also means the process holds real state: alert cooldowns, the import daemon’s file-watch handles, per-session signal stores. A deploy is a process restart, not just a new request.

In development, deploy/docker-compose.dev.yml runs this same process under entr, which kills and restarts it whenever a watched .php/.twig/.js/ .css file changes — so there’s no build step, but there’s also no hot-module-reload: a restart means every open browser tab’s Datastar session re-connects the SSE stream and gets a resynced page.

See Reactive Loop for how signals/actions/SSE fit together, Data Sources for the storage backends, and Import Pipeline for how nfcapd files become RRD/VM data.

Reactive Loop: Signals, Actions & SSE

There is no REST API and no client-side state store. Every piece of UI state is a signal living on the server; every user interaction that needs server logic is an action; every update reaches the browser as an SSE-pushed DOM patch. This is the Datastar model, implemented server-side by php-via.

Signals

$graphSources = $c->signal(Config::$settings->sources, 'graph_sources', clientWritable: true);

A signal has a default value, a human-readable name, and a scope:

  • TAB scope (the default) — private to one browser tab/context. Most form state (alert_form_*, graph_*, flows_*) is TAB-scoped.
  • Shared scopes (ROUTE, SESSION, GLOBAL, or a custom string like rrd:live) — one signal instance shared across every context in that scope, so a write in one tab can broadcast to every other subscribed tab.
  • clientWritable: true lets the browser’s own POST update the signal (through an action); server-owned signals omit it and can only change from PHP.

Client-local signals — prefixed with _, e.g. $_currentView, $_darkMode — never round-trip to the server at all. Every “page” nfsen-ng appears to have (Graphs, Flows, Statistics, Sankey, the Settings sub-tabs) is actually one of these: a client-local signal toggling data-show on a <div> that’s already in the DOM. There’s exactly one server route (/).

Actions

$c->action(function (Context $c) use (&$flowTableHtml): void {
    $filter = $c->getSignal('flows_filter');
    // ... run nfdump, build $flowTableHtml ...
    $c->sync();
}, 'flow-actions');

Actions are closures registered with a name; the client calls them via @post('{{ action_name.url() }}') in a data-on:click attribute, which POSTs to /_action/{id} with the current signal values as the JSON body. The handler reads whatever signals it needs, does its work (frequently shelling out to nfdump — see Nfdump Integration), and calls $c->sync(), which re-renders and pushes the diff to that context’s SSE connection.

The $c->sync() / broadcast split

  • $c->sync() updates the calling context only.
  • $app->broadcast($scope) (used e.g. after an import completes) pushes to every context subscribed to that scope — this is how a new nfcapd file landing updates every open browser tab’s graph without any of them having clicked anything.

Practical consequences

  • No client build step. The frontend is server-rendered Twig + hand-written Web Components (frontend/js/components/) for the pieces that need real client-side behaviour (charts, the date-range slider, the flow table). There’s nothing to bundle.
  • Signal names are wire keys. A signal’s rendered data-bind id is a hash of its name plus a per-context salt; the human name is only a server-side lookup key ($c->getSignal('name')), not what’s transmitted.
  • Actions read signals, not $_POST. $c->input() exists for the rare case an action needs a plain query/form parameter (e.g. delete-alert taking ?id=), but the normal path is signals in, $c->sync() out.

Data Sources: RRD vs. VictoriaMetrics

Storage is pluggable behind one interface, Datasource (backend/datasources/Datasource.php), selected by general.db in config (RRD or VictoriaMetrics) via Settings::datasourceClass(). Both implementations live in backend/datasources/.

The contract

Every datasource implements:

MethodUsed for
write()Persist one source’s per-5-minute-slot counters after import
get_graph_data()Time-series for the Graphs tab (by source/protocol/port)
reset()Wipe data for a rescan
date_boundaries() / last_update()First/last timestamps for a source
get_data_path()Where this source’s data physically lives
healthChecks()Storage-specific entries in the Admin health panel
fetchLatestSlot() / fetchRollingAverage()Aggregate metrics for alert evaluation

fetchLatestSlot()/fetchRollingAverage() are what AlertManager reads by default; an alert rule with a traffic filter bypasses them entirely and runs nfdump directly instead (see Alerts).

RRD (default)

One .rrd file per source (Rrd::get_data_path()), nested under a profile subdirectory: {data_path}/{profile}/{source}[_{port}].rrd. A companion .rrd.first sidecar file tracks the true first-write timestamp, since rrd_first() isn’t reliable for that. RRD trades flexibility for simplicity — it’s a single PHP extension (ext-rrd), no separate service to run, and it’s what classic NfSen used.

VictoriaMetrics

Writes Prometheus-exposition-format samples over HTTP to a VictoriaMetrics instance (deploy/docker-compose.victoriametrics.yml), queried back via its PromQL-compatible HTTP API (query_range, tfirst_over_time, tlast_over_time). This trades the extra moving part for a real time-series database: longer retention, PromQL for ad-hoc queries, and no per-source file to manage. VictoriaMetricsWatcher polls VM’s own health/ readiness so the Admin health panel can report connectivity, not just config sanity.

Profiles

Both datasources are profile-aware: Config::detectProfiles() scans nfdump.profiles-data for source subdirectories (or nested groups of them) and returns the list. With exactly one profile, paths/health-check ids stay flat (rrd_data_gw); with more than one, everything gets profile-suffixed (rrd_data_live_gw) so the two don’t collide. This is how “live” vs. “test” (or any nfdump profile split) shows up throughout the UI and health checks without special-casing.

Import Pipeline

nfsen-ng never talks to nfcapd directly — it only reads the rotated nfcapd.YYYYMMDDHHMM files nfcapd writes to {profiles-data}/{profile}/{source}/YYYY/MM/DD/. Everything from there is backend/common/ImportDaemon.php and backend/common/Import.php, wired up once at boot in AppStartup.php.

Two passes

  1. Catch-up import, run once per profile at process start (ImportDaemon: running initial import (last N years)). Scans nfdump.importYears() worth of history, writing anything not already reflected in the datasource. This is what makes a freshly (re)started server come up already showing historical graphs instead of an empty dashboard.
  2. Ongoing inotify watch, registered per source directory (inotify_add_watch(..., IN_CREATE | IN_MOVED_TO)) and polled every second by an OpenSwoole setInterval timer (AppStartup::pollOnce()). nfcapd rotates a fresh file roughly every 5 minutes by default; each new file triggers:
    • the file’s counters written into the active datasource,
    • $app->broadcast('rrd:live') — every open tab’s graph updates without a reload,
    • AlertManager::runPeriodic() for that profile (see Alerts).

Why polling, not a blocking inotify read

OpenSwoole coroutines cooperate; a blocking inotify_read() call inside a long-running coroutine would stall the whole worker. Polling on a 1-second timer keeps the check non-blocking and cheap (inotify_read with no events pending returns immediately).

Manual controls

The Admin sub-tab (Settings → Import) exposes Force rescan and Trigger import, both of which lock the profile’s ImportDaemon (isLocked()) for the duration so the ongoing inotify poll doesn’t race a manual scan and advance the datasource’s last-update watermark ahead of where the manual scan has actually reached.

Environment caveat

Cross-container inotify (nfcapd writing into a bind-mounted directory that a different container watches) doesn’t reliably propagate on every host — notably WSL2. If the ongoing-watch path never seems to fire in a dev environment, that’s the first thing to check, not the daemon code itself.

Nfdump Integration

backend/processor/Nfdump.php is the one place that shells out to the real nfdump binary. Every other consumer — the Flows tab, Statistics, Sankey, alert traffic filters — goes through it.

Command construction

{binary} {flattened options} {escapeshellarg(filter)}

Options (-R, -M, -o json, -a, …) are set via setOption() and flattened in registration order; the filter expression, if any, is appended as a single trailing, shell-escaped, bare argument — not -f. That flag is reserved by nfdump for “read the filter from a file”; passing a filter string to -f fails with a path does not exist error rather than a filter syntax error, which is easy to misdiagnose if you’re testing a filter by hand outside the app.

-M must be registered before -R. Unlike the other options, -R’s handler doesn’t just store its value — it calls convert_date_to_path() immediately, which resolves the requested time range to actual nfcapd file paths by scanning the sources -M’s handler has recorded so far. Register -R first and that scan sees zero sources, silently finds no files, and throws — which callers that swallow the exception (as Alerts’s filtered-evaluation path does, to fail safe on a malformed filter) will misread as “filter matched nothing” instead of “options were set in the wrong order.” This is exactly how a filtered alert rule ended up unable to ever fire (#153): -R was registered before -M.

Execution

execute() runs the command via proc_open (prefixed with exec so proc_get_status()['pid'] is nfdump’s own PID, not a wrapping shell’s — otherwise kill-nfdump would kill the shell and leave nfdump running), separates stdout/stderr, and:

  • treats known exit codes specially (127 = binary missing, 254 = filter syntax error, 255 = init failure, 250 = internal error),
  • treats "No matching flows" as a normal empty result, not an error,
  • decodes JSON (array or newline-delimited, depending on query type),
  • logs the exact constructed command at LOG_DEBUG — the fastest way to confirm what a given UI action actually asked nfdump for.

The concurrency guard

Config::$settings->nfdumpMaxProcesses caps how many nfdump processes may run at once; execute() checks Misc::countProcessesByName('nfdump') before starting a new one and throws if the cap is already hit, rather than piling up parallel scans on a system that’s likely I/O-bound already.

That counter needs ps or pgrep on PATH — if neither is present (e.g. a minimal container image missing procps), it silently returns 0 and the guard never trips. That failure mode is exactly why there’s a “Process inspection” entry in the Admin health panel’s nfdump group: it flags a missing ps/pgrep explicitly, rather than leaving the guard’s silence to look like “no other nfdump running” (see Health Checks & Admin).

Getting Started

Requires Linux — OpenSwoole has no maintained FreeBSD/other-BSD port (openswoole/ext-openswoole#233).

Quick start (production-ish)

curl -O https://raw.githubusercontent.com/mbolli/nfsen-ng/master/deploy/docker-compose.yml
# edit NFSEN_SOURCES, NFSEN_NFDUMP_PROFILES, etc. in the compose file
docker compose --profile proxy up -d   # bundled Caddy, ports 80/443
# or: docker compose up -d             # app only, port 9000, behind your own proxy

Development

git clone https://github.com/mbolli/nfsen-ng
cd nfsen-ng
docker compose -f deploy/docker-compose.dev.yml up -d
docker compose -f deploy/docker-compose.dev.yml logs -f nfsen

The dev image runs the app under entr: any .php/.twig/.js/.css change under the mounted source kills and restarts the server automatically — no manual restart, no build step.

The compose file’s commented-out nfcapd/nfcapd-test services can inject real (or softflowd-generated) traffic on ports 9995/9996 for local testing; without them the app still runs, just against whatever nfcapd files already exist under the mounted profiles-data volume.

Useful commands

composer install        # PHP deps
composer test            # Pest test suite
composer test-phpstan    # static analysis, level 5
composer fix              # auto-format PHP (php-cs-fixer)
composer before-commit   # fix + phpstan — run this before every PHP commit

pnpm install              # JS deps (also vendors datastar/nouislider/echarts into frontend/js/)
pnpm run lint              # Biome lint
pnpm run format            # Biome format --write

See Project Structure for where things live, Testing for the test suite in more depth, and Environment Notes for sandbox-specific gotchas that have nothing to do with the app itself but will otherwise cost you an hour.

Project Structure

backend/
  app.php                  entry point — every signal, action, and view is defined here
  common/                  Config, Settings, HealthChecker, AlertManager, ImportDaemon, Misc, ...
  actions/                 one file per feature area: GraphActions, FlowActions, StatsActions,
                            SankeyActions, AlertActions, SettingsActions, ImportActions, UtilityActions
  datasources/             Datasource interface + Rrd, VictoriaMetrics implementations
  processor/               Nfdump — the nfdump subprocess wrapper
  templates/               Twig: layout.html.twig (shell) + partials/ (one per tab/section)
  settings/                settings.php(.dist) (deployment config) + preferences.json (user-saved)
frontend/
  js/components/            Web Components: nfsen-chart, nfsen-table, nfsen-daterange, nfsen-sankey, ...
  js/datastar.js, nouislider.min.js, echarts.min.js
                            vendored, copied in by `pnpm install`'s postinstall (see package.json)
tests/
  Unit/                    Pest unit tests — one file per class, roughly
  Feature/                 tests that exercise real I/O (RRD file creation, etc.)
deploy/
  Dockerfile, Dockerfile.dev, docker-compose*.yml, Caddyfile*
.github/workflows/
  release.yml              version bump + tag, manually triggered
  docker-publish.yml       builds/pushes the app image to GHCR (bundled Caddy uses the stock image)
  mdbook.yml               builds this book and deploys it to GitHub Pages
                            on every push to master that touches book/**
book/
  book.toml, src/          this book
  _capture.mjs             screenshot driver — see the Introduction of this chapter's source

Adding a feature end to end

  1. Signal(s) in app.php, inside the $app->page('/', ...) closure.
  2. Action in the relevant *Actions.php (or a new one, registered from app.php) — reads signals, does the work, calls $c->sync().
  3. Template in backend/templates/partials/ — bind the signal ({{ bind(signal) }}), wire the action (data-on:click="@post(...)").
  4. Test in tests/Unit/ — Pest, following the patterns already there.

AGENTS.md at the repo root has the exact Datastar attribute syntax (data-on:click, not data-on:click="${...}"; signal binding via {{ bind(signal) }}; ${{ signal.id() }} for direct assignment inside an event expression) and a list of common footguns — worth reading before your first template edit.

Testing

composer test              # everything
composer test-coverage     # with coverage
docker compose exec nfsen vendor/bin/pest   # from inside the dev container

Tests are Pest PHP, split into tests/Unit/ (pure logic, one file roughly per class) and tests/Feature/ (real I/O — actual RRD file creation, for example).

Patterns worth knowing

  • Offline test doubles for I/O-bound classes. VictoriaMetricsTest.php declares a class VictoriaMetricsTest extends VictoriaMetrics at the top of the file that overrides httpGet()/sendToVM()/tcpConnect() with in-memory stubs, so the whole suite runs without a real VictoriaMetrics instance. If you add a new abstract-ish method to VictoriaMetrics, this double’s override signature has to stay in lockstep — PHP fatals on a parent/child signature mismatch, not a soft warning.
  • Env-var isolation. A test asserting “defaults when no env vars are set” has to putenv('NFSEN_SOURCES') etc. itself; getenv() sees the real ambient environment, which in a dev container may already have NFSEN_SOURCES/NFSEN_PORTS exported for the running app.
  • Profile-aware paths. Rrd::get_data_path()/create() nest files under {data_path}/{profile}/..., not flat — a test that hardcodes a flat path or a cleanup helper that only globs the top level will drift the moment that changes. write() also drops a .rrd.first sidecar file alongside the .rrd — a cleanup glob for *.rrd alone won’t catch it.
  • Misc::countProcessesByName() needs real ps/pgrep. A container image missing procps makes this (and the “finds running php processes” test) silently return 0 rather than fail loudly — see Nfdump Integration for why that matters beyond tests.

End-to-end tests

npm run test-e2e            # runs everything in tests/e2e/ against BASE
BASE=http://localhost:8080 npm run test-e2e   # override the target (default shown)
node tests/e2e/graphs.test.mjs                # run one file directly

tests/e2e/ drives a real headless Chrome against a running app instance — actual clicks, actual nfdump queries, actual SSE-pushed DOM updates. This catches classes of bug the Pest suite structurally can’t: JS runtime errors, races between client-side state and server-pushed patches, and the client/server wire contract for actions (signal names vs. hashed ids, query params vs. JSON body — see below).

There’s one file per tab (smoke, graphs, flows, statistics, sankey, settings, alerts), plus lib/cdp.mjs, the shared driver, and run.mjs, which runs every *.test.mjs file in the directory sequentially and exits non-zero on any failure. Each test file is also directly runnable/standalone (node tests/e2e/<name>.test.mjs) via an import.meta.url check at the bottom.

No Playwright/Puppeteer dependency. lib/cdp.mjs talks raw Chrome DevTools Protocol over Node 22’s native WebSocket, resolving whatever Playwright-managed Chromium build is newest under ~/.cache/ms-playwright (override with CHROME=/path/to/chrome). If none is cached yet: npx playwright install chromium. This mirrors the pattern book/_capture.mjs already established for screenshotting the book.

Patterns worth knowing

  • data-show only toggles CSS visibility — every settings sub-section’s markup is in the DOM at once, whichever tab is “active”. An unscoped query like document.querySelector('table tbody') silently grabs whichever table happens to come first in document order, not the one in the visible panel. Scope queries to the active container instead, e.g. [data-show="$_settingsSection == 'alerts'"] table tbody (alerts.test.mjs) — and prefer page.waitForPanel(signal, value) over a substring [data-show*="..."] match, since hashed signal ids can contain the target string as a substring (import_running____<hash> matches *="import").
  • alerts.test.mjs is the one test that mutates real, persisted state (backend/settings/preferences.json). It creates a uniquely-named rule, toggles it, deletes it, and asserts the rule-count badge returns to its pre-test baseline — so a run never leaves a stray rule behind even if an earlier assertion in the same run fails partway through.
  • Don’t assume query results have rows. This sandbox’s RRD data only covers a recent window and one datatype (bits/bytes) is often gappier than flows/packets (see Import Pipeline). graphs.test.mjs tries Flows → Packets → Traffic in order and falls back to asserting the empty-state message if none have data; flows/statistics assert on the always-present result notification rather than a non-empty table.
  • This suite already found two real bugs, not just test-authoring issues: a startViewTransition race in nfsen-chart.js where a deferred callback could call setOption on an already-disposed chart, and the Alerts Delete/Enable/Test buttons using @post(url, {id: '...'}) — the second argument to Datastar’s @post is a request-options object, not a body payload, so id never reached $c->input('id') and every click was a silent no-op. Both are fixed in the app, not papered over in the test.

Static analysis

composer test-phpstan     # phpstan analyse backend -l 5

In a memory-constrained container, PHPStan’s default 128M can OOM before it finishes; run with an explicit override if that happens:

php -d memory_limit=1G vendor/bin/phpstan analyse backend -l 5 -a backend/settings/settings.php --memory-limit=1G

composer before-commit runs fix (php-cs-fixer) then test-phpstan — the convention is to run it after any PHP change, before committing.

Environment Notes

Sandbox/dev-environment quirks that have bitten while working on this project — none of them are bugs in nfsen-ng itself, but each cost real time to diagnose the first time.

Driving the app without a browser

There’s no separate API to curl — every interaction is the same Datastar action protocol the browser uses:

  1. GET / with a cookie jar to get a session + context id (via_ctx appears in the response HTML).
  2. Every signal’s wire id is a hash (name____<hash>), not its human name — scrape it from the response rather than guessing.
  3. POST /_action/<action-id> (also scraped from the HTML — action ids are randomized per context) with a JSON body of {"via_ctx": "...", "<hashed signal id>": <value>, ...}.
  4. Send an Origin header matching the request host, or expect 403 Forbidden: untrusted origin — curl sends none by default.
  5. Actions that take an id (delete-alert, test-alert, …) read it via $c->input('id'), not a signal — pass it as a query string on the POST URL.

Known flakiness

  • A dev container can restart on its own (observed with no corresponding file edit), which wipes every in-memory context. A previously-scraped via_ctx/action id will then 400 with Invalid context — just re-fetch GET /. Anything persisted to backend/settings/preferences.json (alert rules, saved settings) survives fine, since it’s re-read from disk at boot.
  • Cross-container inotify (a sibling nfcapd container writing into a bind-mounted directory a different container watches) doesn’t reliably propagate on some hosts, notably WSL2. If the import daemon’s ongoing watch never seems to fire, check that before suspecting the daemon code — see Import Pipeline.
  • git inside a container whose bind-mounted repo is owned by a different uid refuses to run (“dubious ownership”). Run git from the host instead of patching the container’s global git config.

nfdump filter syntax

nfdump’s -f flag reads a filter from a file, not from an inline string — a filter expression is a trailing, shell-escaped, bare positional argument (nfdump [options] "proto icmp"). Passing a filter to -f by hand gives a misleading path does not exist: <filter> error. See Nfdump Integration for how the app itself constructs the command correctly.

Traffic Graphs

The default landing tab: time-series traffic graphs rendered with Apache ECharts, driven by the shared date-range control (top of every data tab) and a filter panel.

Graphs tab

Filters

ControlEffect
DisplayGroup series by Sources, Protocols, or Ports
SourcesAny configured source, all, or a specific subset
ProtocolsAny / TCP / UDP / ICMP / Others
Data typeTraffic / Packets / Flows
UnitBits or Bytes

Data comes from Datasource::get_graph_data() — RRD or VictoriaMetrics, whichever is configured (see Data Sources). The graph re-renders live: every nfcapd import broadcasts rrd:live to every open tab (see Import Pipeline), so the “LIVE last update” badge and the plotted series update without a manual refresh.

Resolution & display controls

Below the chart: a data-points slider (points to render, trading resolution for render cost), linear/logarithmic scale, stacked/line series display, and step/curve plot style — all client-side ECharts options, no server round-trip.

Flow Browser

Raw flow-record search: runs nfdump (see Nfdump Integration) over the selected date range and renders the matching records as a sortable table.

Flows tab

Filters

ControlEffect
Limit flowsCap on returned records
SourcesAny / all / a specific source
nfdump filterRaw nfdump filter syntax (e.g. proto tcp and dst port 443), free text
Min / max bytesByte-count bounds

Aggregation & output

Optional bidirectional and per-protocol aggregation; port aggregation by source or destination; IP aggregation with none/exact-IP/IPv4-prefix/ IPv6-prefix granularity per direction; and an “order by tstart” toggle. These map directly onto nfdump’s own -a/aggregation flags — the filter panel is essentially a form over nfdump’s CLI surface, not a reimplementation of it.

IP details

Clicking an IP address cell opens a modal with reverse-DNS plus either public-IP geolocation or (for private IPs) Netbox data — see IP Info Lookup.

Statistics

Top-N breakdowns — nfdump’s -s statistics mode — over any of the dimensions nfdump itself supports: flow records, any/src/dst IP, any/src/dst port, interfaces, AS numbers, next-hop IP, router IP, protocol, direction, or TOS byte.

Statistics tab

Filters

Shares the same date-range, source, nfdump-filter, and min/max-bytes controls as Flows, plus:

ControlEffect
Top recordsHow many ranked rows to return
Statistic forWhich dimension to rank by (see the list above)

Each row links its IP-shaped values into the same IP Info Lookup modal as the Flows tab.

Sankey Diagram

A flow-volume Sankey diagram (source → destination, ranked by bytes or packets) rendered with ECharts, backed by a new nfdump aggregation query (SankeyActions.php) rather than reusing the Flows/Statistics query shapes.

Sankey tab

Filters

Same shape as Statistics — date range, sources, nfdump filter, min/max bytes — plus:

ControlEffect
Top pairsHow many src→dst pairs to render
Rank / size byBytes or packets
PortsAdd the destination L4 port as a middle column (src IP → dst port → dst IP)

With Ports on, the aggregation key gains the destination port (-A srcip,dstport,dstip) and the payload becomes three columns: the middle port: nodes carry a per-port total, so all traffic src→port and port→dst is summed into shared ribbons.

Sankey with the destination-port middle column enabled

Alerts

Threshold-based alert rules, evaluated automatically after every nfcapd import (Settings → Alerts). Managed by AlertManager (backend/common/AlertManager.php) and AlertActions.php.

Alerts sub-tab

Rule shape

FieldNotes
Metricflows / packets / bytes
Operator>, >=, <, <=
Threshold typeAbsolute value, or percent of a rolling average window (10m–24h)
Cooldown5-minute slots to suppress re-firing after a fire
Traffic filterOptional raw nfdump filter expression
NotificationsEmail and/or webhook (HTTP POST, JSON payload)

Percent-of-average rules return an unreachable threshold (PHP_FLOAT_MAX) while the rolling average is still zero — a genuine cold-start guard, not a bug: a rule can’t fire against a baseline that doesn’t exist yet.

Traffic filter

By default a rule’s current value comes from the active datasource’s pre-aggregated totals (fetchLatestSlot() — cheap, but only ever “all traffic for this source”). Setting a traffic filter — any nfdump filter expression, e.g. proto icmp, net 192.168.1.0/24, or a combination — switches that rule to AlertManager::fetchCurrentSlot(), which runs a real nfdump query over the latest 5-minute slot with that filter and sums the matching flows/packets/bytes instead. This is what lets a rule watch “ICMP only” or “this one subnet” rather than just aggregate totals.

fetchCurrentSlot() is the single source of truth for “should this rule use the filtered or aggregate path” — both the periodic evaluation loop and the manual Test button call it, so testing a rule and actually evaluating it behave identically. (They didn’t always: see the note in Nfdump Integration’s neighbourhood about how easy it is for a manual “preview” action to drift from the real evaluation path if it’s implemented as a separate code path instead of a shared one.)

A malformed filter expression fails safe: fetchCurrentSlot() catches the resulting exception and returns zero values rather than propagating the error, so a typo in a filter suppresses that rule’s firing rather than crashing evaluation for every other rule.

Cooldown & history

A fired rule’s cooldown ticks down once per evaluation cycle regardless of whether it fires again; Recent Alert History (bottom of the tab) shows the last 50 dispatches across all rules, newest first.

Notification templates

Email subject/body and webhook title/message are built from {token} templates, not hardcoded strings. Resolution is a 3-tier fallback, checked in AlertManager::resolveTemplate():

  1. The rule’s own override (AlertRule::$emailSubjectTemplate / $emailBodyTemplate / $webhookTitleTemplate / $webhookMessageTemplate — nullable, null = unset, following the same convention as $nfdumpFilter).
  2. A global default, stored in UserPreferences/Settings ($defaultEmailSubjectTemplate etc. — plain string, '' = unset, matching that class’s existing convention for fields like $alertEmailFrom) and saved via the existing save-settings action, not a new one.
  3. AlertManager’s built-in DEFAULT_EMAIL_SUBJECT / DEFAULT_EMAIL_BODY / DEFAULT_WEBHOOK_TITLE / DEFAULT_WEBHOOK_MESSAGE constants — themselves just {token}-templated strings, so the “hardcoded” behavior from before this feature existed is really just tier 3 with tiers 1–2 both empty. A golden test in AlertManagerTest.php asserts the resolved+rendered output for an empty-override rule matches that pre-existing hardcoded text byte-for-byte.

AlertManager::buildTemplateVars() builds the substitution map (12 tokens — see the user guide for the list); substitution itself is a plain strtr(). Notably, {flows}/{packets}/{bytes} are always all three populated regardless of the rule’s own metric, since fetchCurrentSlot() already computes all three together in one nfdump/datasource round-trip — no extra query cost to expose them all as template variables.

The Settings UI’s live preview (frontend/js/components/alert-template-preview.js) is a client-side mirror of this same resolve/substitute logic, using fake example numbers instead of real fired-alert data — it has no server round-trip, so it works for a brand-new, unsaved rule. The preview <pre> elements carry data-ignore-morph, same reasoning as #series/#legend in graph-view.html.twig: they’re empty in the server-rendered HTML and filled in entirely by data-effect, so without it an SSE-pushed catch-up sync shortly after page load morphs them back to the server’s (empty) version, wiping the computed preview text — confirmed against Datastar’s actual morphNode()/ morphChildren() source, which reconciles an element’s children toward the freshly-rendered version whenever isEqualNode() says they differ, deleting anything client JS added that the server doesn’t know about.

Health Checks & Admin

Two adjacent Settings sub-tabs cover operations: Import (manual scan controls + daemon status) and Health (a structured environment/config audit). Both are populated by HealthChecker::run() (backend/common/HealthChecker.php), re-run on every view render with a 30-second throttle.

Health sub-tab

Health check groups

GroupCovers
PHP ExtensionsPHP version, ext-openswoole, ext-rrd (only required for the RRD datasource), ext-inotify
TimezonePHP timezone, NFCAPD_TZ validity, and a plausibility check comparing the most recent nfcapd filename’s timestamp against “now”
nfdumpBinary presence/version (minimum 1.7.2 — the JSON field-name scheme changed from 1.6.x), max-processes config, and process inspection (below)
SourcesAt least one configured
Import DaemonRunning / initializing / watching N directories, last auto-import age
nfcapd Pathsprofiles-data reachable; per-profile, per-source directory presence, flat-vs-nested layout, and capture freshness (warns past ~12 minutes — 2.4× nfcapd’s default 5-minute rotation)
RRD Storage / VictoriaMetricsDelegated to the active Datasource::healthChecks()

Every entry is ok / warning / error, sorted errors-first within its group, with an optional hint line explaining what to actually do about it.

Process inspection

Misc::countProcessesByName() — the function backing the nfdump concurrency guard (see Nfdump Integration) — needs ps or pgrep on PATH. Without either, it silently returns 0, which makes the guard permanently believe no nfdump process is running. The Process inspection health entry exists specifically to surface that condition as a warning rather than let it look like a healthy, quiet system.

Import (Admin)

Import sub-tab

Trigger import re-runs the catch-up scan for the selected profile; Force rescan additionally resets the datasource for that profile first (a destructive re-import, guarded by a confirmation signal). Both lock the profile’s ImportDaemon for the duration so the ongoing inotify poll doesn’t advance the datasource ahead of where the manual scan has reached.

Preferences & Timezones

Two more Settings sub-tabs: Preferences (user-editable, persisted to backend/settings/preferences.json) and System (a read-only dump of deployment config for reference).

Preferences sub-tab

Preferences

SectionFields
GeneralDefault view (which tab opens on load), log level
Graph defaultsDefault display mode, datatype, protocols
Flow & statistics defaultsDefault flow limit, default statistics order-by
Date & time displayBrowser-local vs. server timezone for displayed timestamps
Filter presetsA saved list of nfdump filter strings, offered as quick picks across Flows/Statistics/Sankey

Saving persists through SettingsActions::register()’s single save-settings action, which merges the new values with whatever’s already in preferences.json — including the alert rules, so saving preferences never touches alert state.

Timezones

The container itself runs TZ=UTC; nfcapd filenames are parsed in NFCAPD_TZ if set (falls back to the PHP timezone otherwise), independent of the browser display setting above. If nfcapd runs in a different timezone than the container, set NFCAPD_TZ explicitly — the Health sub-tab’s “nfcapd file time” check will flag an implausible (future or very stale) timestamp if this is wrong.

System

System sub-tab

Read-only: sources, ports, active datasource, import-years, nfdump binary path, profiles-data path, and the preferences file location — everything that comes from environment variables or settings.php and needs a container restart to change. Useful as a “what is this instance actually configured with” reference without grepping env vars.

IP Info Lookup

Clicking an IP address in the Flows or Statistics tables (rendered as <a class="ip-link"> by TableFormatter.php) triggers the ip-info action (UtilityActions.php), which renders a modal fragment (ip-info-modal.html.twig) with:

  • Reverse DNSgethostbyaddr(), falling back to shelling out to host and then to a “could not be resolved” label rather than showing the raw IP back.
  • Geolocation, for public IPs only — a live lookup against ipapi.co (city, region, country, coordinates, timezone, ASN, org, currency — whatever it returns), with a 5-second timeout so a slow/unreachable external API can’t hang the modal.
  • Netbox data, for private IPs only, if NFSEN_NETBOX_URL/ NFSEN_NETBOX_TOKEN are configured — whatever IPAM record Netbox has for that address (IpLookup::netbox()).

Private vs. public is decided once (IpLookup::isPrivate()) and picks exactly one of geolocation or Netbox — a private (RFC 1918) address is never sent to the public geolocation API, and a public address never triggers a Netbox lookup.

The modal is a native <dialog> element (no Bootstrap JS/Popper dependency) pushed by the server as a Datastar patch — the same pattern as any other action, just targeting a fragment instead of the whole page.

Actions Reference

There is no separate HTTP/REST API — see Reactive Loop for why. Every server-side operation is one of these named actions, each reachable at POST /_action/{action-id} (the id is randomized per browser tab; the Twig templates always resolve it via {{ action_name.url() }}, never a hardcoded path).

ActionFileDoes
save-alertAlertActions.phpCreate/update an alert rule
delete-alertAlertActions.phpRemove an alert rule (?id=)
toggle-alertAlertActions.phpEnable/disable a rule (?id=)
test-alertAlertActions.phpEvaluate a rule once, on demand, without waiting for the next cycle
flow-actionsFlowActions.phpRun the Flows tab’s nfdump query and re-render the table
stats-actionsStatsActions.phpRun the Statistics tab’s -s query
dismiss-notificationStatsActions.phpDismiss a flow/stats-panel notification
count-filesStatsActions.phpRecount matching nfcapd files (feeds the Statistics/Sankey “may take a moment” query-time warning, not the import progress bar)
sankey-actionsSankeyActions.phpRun the Sankey tab’s aggregation query
dismiss-sankey-notificationSankeyActions.phpDismiss a Sankey-panel notification
change-profileGraphActions.phpSwitch the active nfdump profile
refresh-graphsGraphActions.phpRe-render the Graphs tab for the current filter set
save-settingsSettingsActions.phpPersist Preferences (merges with existing alert rules)
trigger-importImportActions.phpManual catch-up import for a profile
force-rescanImportActions.phpReset + re-import a profile (destructive, confirmation-gated)
cancel-importImportActions.phpCancel an in-progress manual import
ip-infoUtilityActions.phpReverse-DNS lookup modal, plus geo-IP (public IPs) or Netbox (private IPs)
kill-nfdumpUtilityActions.phpSend SIGTERM to the currently-running nfdump subprocess

Reading an action’s exact contract

The fastest way to see what signals an action actually reads/writes is the action closure itself — they’re short, and every one starts by pulling its inputs via $c->getSignal('name') before doing anything. There’s intentionally no separate schema/OpenAPI layer to keep in sync with it.

Roadmap

Tracked as GitHub issues.

Open

  • #143 — v1 broken on FreeBSD. OpenSwoole’s C extension doesn’t build on FreeBSD (openswoole/ext-openswoole#233), and php-via is built on OpenSwoole, so a bare-metal FreeBSD install currently can’t run this at all. Short-term (documenting the limitation in the README/book) is done. Medium-term: whether php-via can be made runtime-agnostic (OpenSwoole vs. Swoole vs. an alternative server) is an open question that needs a php-via-side answer, not just an nfsen-ng one.

Recently shipped

  • #152 — Sankey diagram. The Sankey tab (see Sankey Diagram) aggregates flows into top-N src/dst pairs and renders them with ECharts.
  • #150 — Alerts improvement. Host/subnet and protocol-scoped alerting, implemented as a freeform nfdump traffic filter on a rule (see Alerts) rather than separate host/protocol fields — covers the same use case (an ICMP-only or single-subnet rule) with one field instead of two.
  • Migrated the Graphs tab from Dygraphs to Apache ECharts, so the whole app now shares one charting library instead of two.
  • Custom-duration date range input (presets weren’t granular enough for every use case).
  • Column visibility/sort-order persistence across SSE table re-renders.
  • Docker env var for the SSE reload interval.
  • End-to-end browser test suite covering every tab (see Testing).