diff --git a/INSTALL.md b/INSTALL.md index 5cadabf..f365dc2 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -4,7 +4,6 @@ These instructions install nfsen-ng on a fresh Ubuntu 20.04/22.04 LTS or Debian **Note that setup of nfcapd is not covered here, but nfsen-ng requires data captured by nfcapd to work.** - ## Ubuntu 20.04/22.04 LTS ```bash diff --git a/README.md b/README.md index 421ea2f..298c9b9 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,31 @@ # nfsen-ng + [![GitHub license](https://img.shields.io/github/license/mbolli/nfsen-ng.svg?style=flat-square)](https://github.com/mbolli/nfsen-ng/blob/master/LICENSE) [![GitHub issues](https://img.shields.io/github/issues/mbolli/nfsen-ng.svg?style=flat-square)](https://github.com/mbolli/nfsen-ng/issues) [![Donate a beer](https://img.shields.io/badge/paypal-donate-yellow.svg?style=flat-square)](https://paypal.me/bolli) nfsen-ng is an in-place replacement for the ageing nfsen. -
+![nfsen-ng dashboard overview](https://github.com/mbolli/nfsen-ng/assets/722725/c3df942e-3d3c-4ef9-86ad-4e5780c7b6d8) -**Used components** +## Used components - * Front end: [jQuery](https://jquery.com), [dygraphs](http://dygraphs.com), [FooTable](http://fooplugins.github.io/FooTable/), [ion.rangeSlider](http://ionden.com/a/plugins/ion.rangeSlider/en.html) - * Back end: [RRDtool](http://oss.oetiker.ch/rrdtool/), [nfdump tools](https://github.com/phaag/nfdump) +* Front end: [jQuery](https://jquery.com), [dygraphs](http://dygraphs.com), [FooTable](http://fooplugins.github.io/FooTable/), [ion.rangeSlider](http://ionden.com/a/plugins/ion.rangeSlider/en.html) +* Back end: [RRDtool](http://oss.oetiker.ch/rrdtool/), [nfdump tools](https://github.com/phaag/nfdump) ## TOC - * [nfsen-ng](#nfsen-ng) - * [Installation](#installation) - * [Configuration](#configuration) - * [CLI](#cli) - * [API](#api) - * [/api/config](#apiconfig) - * [/api/graph](#apigraph) - * [/api/flows](#apiflows) - * [/api/stats](#apistats) +* [nfsen-ng](#nfsen-ng) + * [Installation](#installation) + * [Configuration](#configuration) + * [CLI](#cli) + * [CLI as a service](#cli-as-a-service) + * [Logs](#logs) + * [API](#api) + * [/api/config](./API_ENDPOINTS.md#apiconfig) + * [/api/graph](./API_ENDPOINTS.md#apigraph) + * [/api/flows](./API_ENDPOINTS.md#apiflows) + * [/api/stats](./API_ENDPOINTS.md#apistats) ## Installation @@ -39,15 +42,15 @@ Software packages required: Apache modules required: - * mod_rewrite - * mod_deflate - * mod_headers - * mod_expires +* mod_rewrite +* mod_deflate +* mod_headers +* mod_expires PHP modules required: - * mbstring - * rrd +* mbstring +* rrd ## Configuration @@ -55,20 +58,65 @@ PHP modules required: The default settings file is `backend/settings/settings.php.dist`. Copy it to `backend/settings/settings.php` and start modifying it. Example values are in *italic*: - * **general** - * **ports:** (_array(80, 23, 22, ...)_) The ports to examine. _Note:_ If you use RRD as datasource and want to import existing data, you might keep the number of ports to a minimum, or the import time will be measured in moon cycles... - * **sources:** (_array('source1', ...)_) The sources to scan. - * **db:** (_RRD_) The name of the datasource class (case-sensitive). - * **frontend** +* **general** + * **ports:** (*array(80, 23, 22, ...)*) The ports to examine. *Note:* If you use RRD as datasource and want to import existing data, you might keep the number of ports to a minimum, or the import time will be measured in moon cycles... + * **sources:** (*array('source1', ...)*) The sources to scan. + * **db:** (*RRD*) The name of the datasource class (case-sensitive). + * **frontend** * **reload_interval:** Interval in seconds between graph reloads. - * **nfdump** - * **binary:** (_/usr/bin/nfdump_) The location of your nfdump executable - * **profiles-data:** (_/var/nfdump/profiles_data_) The location of your nfcapd files - * **profile:** (_live_) The profile folder to use - * **max-processes:** (_1_) The maximum number of concurrently running nfdump processes. _Note:_ Statistics and aggregations can use lots of system resources, even to aggregate one week of data might take more than 15 minutes. Put this value to > 1 if you want nfsen-ng to be usable while running another query. - * **db** If the used data source needs additional configuration, you can specify it here, e.g. host and port. - * **log** - * **priority:** (_LOG_INFO_) see other possible values at [http://php.net/manual/en/function.syslog.php] + * **nfdump** + * **binary:** (*/usr/bin/nfdump*) The location of your nfdump executable + * **profiles-data:** (*/var/nfdump/profiles_data*) The location of your nfcapd files + * **profile:** (*live*) The profile folder to use + * **max-processes:** (*1*) The maximum number of concurrently running nfdump processes. *Note:* Statistics and aggregations can use lots of system resources, even to aggregate one week of data might take more than 15 minutes. Put this value to > 1 if you want nfsen-ng to be usable while running another query. + * **db** If the used data source needs additional configuration, you can specify it here, e.g. host and port. + * **log** + * **priority:** (*LOG_INFO*) see other possible values at [http://php.net/manual/en/function.syslog.php] + +### Nfdump + +Nfsen-ng uses nfdump to read the nfcapd files. You can specify the location of the nfdump binary in `backend/settings/settings.php`. The default location is `/usr/bin/nfdump`. + +you should also have a look at the nfdump configuration file `/etc/nfdump.conf` and make sure that the `nfcapd` files are written to the correct location. The default location is `/var/nfdump/profiles_data`. + +here is an example of the nfdump configuration file: + +```ini +options='-z -S 1 -T all -l /var/nfdump/profiles-data/live/ -p ' +``` + +where + +* `-z` is used to compress the nfcapd files +* `-S 1` is used to specify the nfcapd directory structure +* `-T all` is used to specify the extension of the nfcapd files +* `-l` is used to specify the destination location of the nfcapd files +* `-p` is used to specify the port of the nfcapd files. + +#### Nfcapd x Sfcapd + +One might use sfcapd instead of nfcapd. In this case, you should change the `nfdump` configuration file to use `sfcapd` instead of `nfcapd`. + +In this case, you should change `/lib/systemd/system/nfdump@.service` to use `sfcapd` instead of `nfcapd`: + +```ini +[Unit] +Description=netflow capture daemon, %I instance +Documentation=man:sfcapd(1) +After=network.target auditd.service +PartOf=nfdump.service + +[Service] +Type=forking +EnvironmentFile=/etc/nfdump/%I.conf +ExecStart=/usr/bin/sfcapd -D -P /run/sfcapd.%I.pid $options +PIDFile=/run/sfcapd.%I.pid +KillMode=process +Restart=no + +[Install] +WantedBy=multi-user.target +``` ## CLI @@ -82,29 +130,58 @@ or for the daemon `./cli.php start|stop|status` +* **Options:** + * **-v** Show verbose output + * **-p** Import ports data as well *Note:* Using RRD this will take quite a bit longer, depending on the number of your defined ports. + * **-ps** Import ports per source as well *Note:* Using RRD this will take quite a bit longer, depending on the number of your defined ports. + * **-f** Force overwriting database and start fresh - * **Options:** - * **-v** Show verbose output - * **-p** Import ports data as well _Note:_ Using RRD this will take quite a bit longer, depending on the number of your defined ports. - * **-ps** Import ports per source as well _Note:_ Using RRD this will take quite a bit longer, depending on the number of your defined ports. - * **-f** Force overwriting database and start fresh - - * **Commands:** - * **import** Import existing nfdump data to nfsen-ng. _Note:_ If you have existing nfcapd files, better do this overnight. - * **start** Start the daemon for continuous reading of new data - * **stop** Stop the daemon - * **status** Get the daemon's status + * **Commands:** + * **import** Import existing nfdump data to nfsen-ng. *Note:* If you have existing nfcapd files, better do this overnight. + * **start** Start the daemon for continuous reading of new data + * **stop** Stop the daemon + * **status** Get the daemon's status - * **Examples:** - * `./cli.php -f import` + * **Examples:** + * `./cli.php -f import` Imports fresh data for sources - * `./cli.php -f -p -ps import` + * `./cli.php -f -p -ps import` Imports all data - * `./cli.php start` + * `./cli.php start` Starts the daemon +### CLI as a service + +You can use the CLI as a service. To do so, you can use the provided systemd service file below. You can copy it to `/etc/systemd/system/nfsen-ng.service` and then start it with `systemctl start nfsen-ng`. + +```ini +[Unit] +Description=nfsen-ng +After=network-online.target + +[Service] +Type=simple +RemainAfterExit=yes +restart=always +startLimitIntervalSec=0 +restartSec=2 +ExecStart=su - www-data --shell=/bin/bash -c '/var/www/html/nfsen-ng/backend/cli.php start' +ExecStop=su - www-data --shell=/bin/bash -c '/var/www/html/nfsen-ng/backend/cli.php stop' + +[Install] +WantedBy=multi-user.target +``` + +Now, you should reload and enable the service to start on boot with `systemctl daemon-reload` and `systemctl enable nfsen-ng`. + +## Logs + +Nfsen-ng logs to syslog. You can find the logs in `/var/log/syslog` or `/var/log/messages` depending on your system. Some distribuitions might register it in `journalctl`. To access the logs, you can use `tail -f /var/log/syslog` or `journalctl -u nfsen-ng` + +You can change the log priority in `backend/settings/settings.php`. ## API + The API is used by the frontend to retrieve data. The API endpoints are documented in [API_ENDPOINTS.md](./API_ENDPOINTS.md).