Stars: 192
Forks: 34
Pull Requests: 16
Issues: 75
Watchers: 26
Last Updated: 2023-04-13 23:22:11
Responsive NetFlow visualizer built on top of nfdump tools.
License: Apache License 2.0
Languages: PHP, CSS, HTML, JavaScript
nfsen-ng is an in-place replacement for the ageing nfsen.
Used components
Ubuntu 18.04 LTS:
# run following commands as root
# enable universe repository
add-apt-repository universe && sudo apt update
# install packages
apt install apache2 php7.2 php7.2-dev libapache2-mod-php7.2 pkg-config nfdump rrdtool librrd-dev
# enable apache modules
a2enmod rewrite deflate headers expires
# install rrd library for php
pecl install rrd
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php/7.2/mods-available/rrd.ini
# enable php mod
phpenmod rrd
# configure virtual host to read .htaccess files
vim /etc/apache2/apache2.conf # set AllowOverride All for /var/www
# restart httpd
service apache2 restart
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
Ubuntu 20.04 LTS:
# run following commands as root
# install packages
apt install apache2 git nfdump pkg-config php7.4 php7.4-dev libapache2-mod-php7.4 rrdtool librrd-dev
# enable apache modules
a2enmod rewrite deflate headers expires
# install rrd library for php
pecl install rrd
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php/7.4/mods-available/rrd.ini
# enable php mod
phpenmod rrd
# configure virtual host to read .htaccess files
vi /etc/apache2/apache2.conf # set AllowOverride All for /var/www
# restart apache web server
systemctl restart apache2
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
Ubuntu 22.04 LTS:
# run following commands as root
# install packages
apt install apache2 git nfdump pkg-config php8.1 php8.1-dev libapache2-mod-php8.1 rrdtool librrd-dev
# enable apache modules
a2enmod rewrite deflate headers expires
# install rrd library for php
pecl install rrd
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php/8.1/mods-available/rrd.ini
# enable php mod
phpenmod rrd
# configure virtual host to read .htaccess files
vi /etc/apache2/apache2.conf # set AllowOverride All for /var/www
# restart apache web server
systemctl restart apache2
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
Debian 11 :
# run following commands as root
# install packages
apt install apache2 git nfdump pkg-config php php-dev libapache2-mod-php rrdtool librrd-dev
# enable apache modules
a2enmod rewrite deflate headers expires
# install rrd library for php
pecl install rrd
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php/7.4/mods-available/rrd.ini
# enable php mod
phpenmod rrd
# configure virtual host to read .htaccess files
vi /etc/apache2/apache2.conf # set AllowOverride All for /var/www
# restart apache web server
systemctl restart apache2
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R www-data:www-data .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
CentOS 7:
# run following commands as root
# update packages
yum update
# enable EPEL repo
yum -y install epel-release
# install yum utils
yum install yum-utils
# install remi release
yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm
# enable the repository for PHP 7.2
yum-config-manager --enable remi-php72
# install packages
yum install git httpd mod_php nfdump php72 php72-php-devel php-devel php-pear php-pecl-rrd rrdtool rrdtool-devel
# configure virtual host to read .htaccess files
vim /etc/httpd/conf/httpd.conf # set AllowOverride All for /var/www/html
# start httpd service
systemctl start httpd
# enable httpd service
systemctl enable httpd
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R apache:apache .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
CentOS 8:
# run following commands as root
# update packages
dnf update
# enable EPEL repo and update epel-release package
dnf -y install epel-release && dnf -y update epel-release
# install dnf-utils
dnf -y install dnf-utils
# enable PowerTools repo
dnf config-manager --set-enabled PowerTools
# install packages
dnf -y install git httpd make mod_php nfdump php php-devel php-json php-pear rrdtool rrdtool-devel
# install rrd library for php
pecl install rrd
# create rrd library mod entry for php
echo "extension=rrd.so" > /etc/php.d/rrd.ini
# configure virtual host to read .htaccess files
vim /etc/httpd/conf/httpd.conf # set AllowOverride All for /var/www/html
# start httpd service
systemctl start httpd
# enable httpd service
systemctl enable httpd
# install nfsen-ng
cd /var/www/html # or wherever
git clone https://github.com/mbolli/nfsen-ng
chown -R apache:apache .
chmod +x nfsen-ng/backend/cli.php
# next step: configuration
Note: nfsen-ng expects the profiles-data folder structure to be
PROFILES_DATA_PATH/PROFILE/SOURCE/YYYY/MM/DD/nfcapd.YYYYMMDDHHII
, e.g./var/nfdump/profiles_data/live/source1/2018/12/01/nfcapd.201812010225
.
The default settings file is backend/settings/settings.php.dist
. Copy it to backend/settings/settings.php
and start modifying it. Example values are in italic:
The command line interface is used to initially scan existing nfcapd.* files, or to administer the daemon.
Usage:
./cli.php [ options ] import
or for the daemon
./cli.php start|stop|status
Options:
Commands:
Examples:
./cli.php -f import
Imports fresh data for sources
./cli.php -f -p -ps import
Imports all data
./cli.php start
Starts the daemon
The API is used by the frontend to retrieve data.
URL
/api/config
Method:
GET
URL Params none
Success Response:
{
"sources": [ "gate", "swi6" ],
"ports": [ 80, 22, 23 ],
"stored_output_formats": [],
"stored_filters": [],
"daemon_running": true
}
Error Response:
{"code": 400, "error": "400 - Bad Request. Probably wrong or not enough arguments."}
OR
{"code": 404, "error": "400 - Not found. "}
Sample Call:
curl localhost/nfsen-ng/api/config
URL
/api/graph?datestart=1490484000&dateend=1490652000&type=flows&sources[0]=gate&protocols[0]=tcp&protocols[1]=icmp&display=sources
Method:
GET
URL Params
datestart=[integer]
Unix timestampdateend=[integer]
Unix timestamptype=[string]
Type of data to show: flows/packets/bytessources=[array]
protocols=[array]
ports=[array]
display=[string]
can be sources
, protocols
or ports
There can't be multiple sources and multiple protocols both. Either one source and multiple protocols, or one protocol and multiple sources.
Success Response:
{"data": {
"1490562300":[2.1666666667,94.396666667],
"1490562600":[1.0466666667,72.976666667],...
},"start":1490562300,"end":1490590800,"step":300,"legend":["swi6_flows_tcp","gate_flows_tcp"]}
Error Response:
json {"code": 400, "error": "400 - Bad Request. Probably wrong or not enough arguments."}
OR
json {"code": 404, "error": "400 - Not found. "}
Sample Call:
curl -g "http://localhost/nfsen-ng/api/graph?datestart=1490484000&dateend=1490652000&type=flows&sources[0]=gate&protocols[0]=tcp&protocols[1]=icmp&display=sources"
URL
/api/flows?datestart=1482828600&dateend=1490604300&sources[0]=gate&sources[1]=swi6&filter=&limit=100&aggregate=srcip&sort=&output[format]=auto
Method:
GET
URL Params
datestart=[integer]
Unix timestampdateend=[integer]
Unix timestampsources=[array]
filter=[string]
pcap-syntaxed filterlimit=[int]
max. returned rowsaggregate=[string]
can be bidirectional
or a valid nfdump aggregation string (e.g. srcip4/24, dstport
), but not both at the same timesort=[string]
(will probably cease to exist, as ordering is done directly in aggregation) e.g. tstart
output=[array]
can contain [format] = auto|line|long|extended
and [IPv6]
Success Response:
[["ts","td","sa","da","sp","dp","pr","ipkt","ibyt","opkt","obyt"],
["2017-03-27 10:40:46","0.000","85.105.45.96","0.0.0.0","0","0","","1","46","0","0"],
...
Error Response:
json {"code": 400, "error": "400 - Bad Request. Probably wrong or not enough arguments."}
OR
json {"code": 404, "error": "400 - Not found. "}
Sample Call:
curl -g "http://localhost/nfsen-ng/api/flows?datestart=1482828600&dateend=1490604300&sources[0]=gate&sources[1]=swi6&filter=&limit=100&aggregate[]=srcip&sort=&output[format]=auto"
URL
/api/stats?datestart=1482828600&dateend=1490604300&sources[0]=gate&sources[1]=swi6&for=dstip&filter=&top=10&limit=100&aggregate[]=srcip&sort=&output[format]=auto
Method:
GET
URL Params
datestart=[integer]
Unix timestampdateend=[integer]
Unix timestampsources=[array]
filter=[string]
pcap-syntaxed filtertop=[int]
return top N rowsfor=[string]
field to get the statistics for. with optional ordering field as suffix, e.g. ip/flows
limit=[string]
limit output to records above or below of limit
e.g. 500K
output=[array]
can contain [IPv6]
Success Response:
[
["Packet limit: > 100 packets"],
["ts","te","td","pr","val","fl","flP","ipkt","ipktP","ibyt","ibytP","ipps","ipbs","ibpp"],
["2017-03-27 10:38:20","2017-03-27 10:47:58","577.973","any","193.5.80.180","673","2.7","676","2.5","56581","2.7","1","783","83"],
...
]
Error Response:
json {"code": 400, "error": "400 - Bad Request. Probably wrong or not enough arguments."}
OR
json {"code": 404, "error": "400 - Not found. "}
Sample Call:
curl -g "http://localhost/nfsen-ng/api/stats?datestart=1482828600&dateend=1490604300&sources[0]=gate&sources[1]=swi6&for=dstip&filter=&top=10&limit=100&aggregate[]=srcip&sort=&output[format]=auto"
More endpoints to come:
/api/graph_stats