From 33641a2b9b7fb6ac91f8b80a8ae986612cc59cde Mon Sep 17 00:00:00 2001 From: marco Date: Tue, 23 Apr 2024 23:28:42 +0200 Subject: [PATCH 1/9] update opnsense install --- .../docs/getting_started/install_opnsense.md | 64 +++--- crowdsec-docs/sidebarsUnversioned.js | 2 + .../getting_started/installation/opnsense.mdx | 26 ++- .../getting_started/installation/pfsense.mdx | 217 +++++++++++++++++- 4 files changed, 275 insertions(+), 34 deletions(-) diff --git a/crowdsec-docs/docs/getting_started/install_opnsense.md b/crowdsec-docs/docs/getting_started/install_opnsense.md index c046a919..b8c40a5c 100644 --- a/crowdsec-docs/docs/getting_started/install_opnsense.md +++ b/crowdsec-docs/docs/getting_started/install_opnsense.md @@ -4,45 +4,53 @@ title: OPNsense sidebar_position: 1 --- -## The OPNsense plugin +# OPNsense plugin -OPNsense, the widely known firewall and routing software, is based on FreeBSD. The standard packages that we provide for FreeBSD (agent+lapi and bouncer) can also run on OPNsense, but need more integration with the rest of the system (for example, to feed the firewall tables). +The CrowdSec plugin for OPNsense is installed from the official repositories. +It includes a Log Processor, LAPI service, and Remediation Component. This allows you +to: -By installing the `CrowdSec` plugin, available through the OPNsense repositories, you can: - - - use the OPNsense server as LAPI for other agents and bouncers - - deploy an agent on OPNsense and scan its logs for attacks - - block attackers from your whole network with a single firewall bouncer + - block attacking traffic from entering the network (protect machines that don't have CrowdSec) + - deploy a log processor on OPNsense and scan its logs for attacks + - use the OPNsense server as LAPI for other log processors and remediation components - list the hub plugins (parsers, scenarios..) and decisions on the OPNsense admin interface -### Plugin installation +## Plugin installation -:::caution -With OPNsense 22.1, if you are using a RAM filesystem for `/var` (you can verify `System > Settings > Miscellaneous > Disk/Memory Settings`) you need to disable it before proceeding, because the `Security Engine` keeps a small persistent database in `/var/db`. When the `/var` directory is in RAM, the database is re-created from scratch at each reboot. The ram disk was changed to `/var/log` for 22.7, which does not interfere with the database. -::: +Click `System > Firmware > Plugins` menu. Select os-crowdsec. It will deploy three packages: -With OPNsense 22.1: + - `os-crowdsec`, the plugin itself + - `crowdsec` + - `crowdsec-firewall-bouncer` -```console -# pkg install os-crowdsec-devel -``` +Do not enable/start the services from the terminal like you would on a standard freebsd system, because the plugin takes care of that. -With OPNsense >= 22.7: +Refresh the page and go to `Services > CrowdSec > Overview` to verify the running services and installed configurations. -Download CrowdSec from the `System > Firmware > Plugins` menu. It will deploy three packages: +Great, you now have CrowdSec installed on your system. Have a look at the [post installation steps](/getting_started/next_steps.md) to see how to to configure and optimize it, +these recommendations are valid for any system. - - `os-crowdsec`, the plugin itself - - `crowdsec` - - `crowdsec-firewall-bouncer` -On the Settings tab, select the first three checkboxes: IDS, LAPI and IPS. Click Apply. +## Plugin Configuration -Do not enable/start the agent and bouncer services with `sysrc` or `/etc/rc.conf` like you would on a standard freebsd system, because the plugin takes care of that (in a different way). +You will find some options under `Services > CrowdSec > Settings`. You will see the first three are enabled by default: Log Processor (previously known as IDS), LAPI and Remediation Component (previously known as IPS). +You can disable them for testing or if you have special requirements. The parsers, scenarios and all objects from the [CrowdSec Hub](https://hub.crowdsec.net/) are periodically upgraded. The [crowdsecurity/freebsd](https://hub.crowdsec.net/author/crowdsecurity/collections/freebsd) and [crowdsecurity/opnsense](https://hub.crowdsec.net/author/crowdsecurity/collections/opnsense) collections are installed by default. -### Testing the plugin +Since crowdsec 1.6.3, private IP networks are whitelisted by default as well. This means for example an IP from a LAN or WAN which is on 192.168.x.y won't get blocked by a local decision +(cummunity blocklists don't contain private IPs). +If you want to revert to the previous behavior, to block private IPs as well, you can remove the related parser. + +```console +[root@OPNsense ~]# cscli parsers remove crowdsecurity/whitelists +``` + +If on the other hand you upgrade from a version before 1.6.3, you need to install the lists yourself. + + +## Testing the remediation component A quick way to test that everything is working correctly is to execute the following command. Your ssh session should freeze and you should be kicked out from the firewall. You will not be able to connect to it (from the same IP address) for two minutes. It might be a good idea to have a secondary IP from which you can connect, should anything go wrong. @@ -54,7 +62,7 @@ This is a more secure way to test than attempting to brute-force yourself: the d You can find a list of all available flags with `cscli decisions add --help`. -#### How do I find my connecting IP address to test? +### How do I find my connecting IP address to test? We have provided some examples below to help you find your connecting IP address. Depending on your shell / environment, you may need to use a different command. @@ -65,8 +73,7 @@ We have provided some examples below to help you find your connecting IP address ## Remote LAPI setup (optional) -You may have a need to put the LAPI service on a different machine, possibly because you already have it, or opnsense is running on a slow machine or... -well - your servers, your right to choose. But you'll have to manually tweak the configuration (thanks [Jarno Rankinen](https://github.com/0ranki)). +If you don't want to run the LAPI service on the OPNsense machine (because it's small/slow or you already have LAPI somewhere) then you'll have to manually tweak the configuration (thanks [Jarno Rankinen](https://github.com/0ranki)). Be aware: the list of machines and bouncers shown in the Overview tab will be incorrect. In the current version, the crowdsec instance on OPNsense has no way (and no permission) to retrieve the list of machines and bouncers from the LAPI if it resides on another server, so it displays the local (and outdated) information. @@ -122,8 +129,3 @@ For more information on the topic: - [Improve The CrowdSec Multi-Server Installation With HTTPS Between Agents](https://www.linuxjournal.com/content/improve-crowdsec-multi-server-installation-https-between-agents) (Linux Journal) -## Enrolling your instance - -The next step is to enroll your instance with the [CrowdSec Console](https://app.crowdsec.net/security-engines?enroll-engine=true). - -For the benefits, please visit the [Console section](/u/console/intro). diff --git a/crowdsec-docs/sidebarsUnversioned.js b/crowdsec-docs/sidebarsUnversioned.js index d3a81a8c..5530e84e 100644 --- a/crowdsec-docs/sidebarsUnversioned.js +++ b/crowdsec-docs/sidebarsUnversioned.js @@ -425,6 +425,8 @@ module.exports = { items: [ "getting_started/installation/linux", "getting_started/installation/freebsd", + "getting_started/installation/opnsense", + "getting_started/installation/pfsense", "getting_started/installation/windows", "getting_started/installation/macos", "getting_started/installation/docker", diff --git a/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx b/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx index e2d5e9bf..9735d813 100644 --- a/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx +++ b/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx @@ -5,4 +5,28 @@ pagination_prev: getting_started/pre_requisites pagination_next: getting_started/next_steps --- -## TODO BY @MMETC \ No newline at end of file +# OPNsense plugin + +The CrowdSec plugin for OPNsense is installed from the official repositories. +It includes a Log Processor, LAPI service, and Remediation Component. This allows you +to: + + - block attacking traffic from entering the network (protect machines that don't have CrowdSec) + - deploy a log processor on OPNsense and scan its logs for attacks + - use the OPNsense server as LAPI for other log processors and remediation components + - list the hub plugins (parsers, scenarios..) and decisions on the OPNsense admin interface + +## Plugin installation + +Click `System > Firmware > Plugins` menu. Select os-crowdsec. It will deploy three packages: + + - `os-crowdsec`, the plugin itself + - `crowdsec` + - `crowdsec-firewall-bouncer` + +Do not enable/start the services from the terminal like you would on a standard freebsd system, because the plugin takes care of that. + +Refresh the page and go to `Services > CrowdSec > Overview` to verify the running services and installed configurations. + +Great, you now have CrowdSec installed on your system. Have a look at the [post installation steps](/getting_started/next_steps.md) to see how to to configure and optimize it, +or keep reading [the rest of the plugin documentation](/getting_started/install_opnsense.md) diff --git a/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx b/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx index 79e4666e..6f065d1c 100644 --- a/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx +++ b/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx @@ -1,8 +1,221 @@ --- id: pfsense -title: PFsense +title: pfSense pagination_prev: getting_started/pre_requisites pagination_next: getting_started/next_steps --- -## TODO BY @MMETC \ No newline at end of file +We have created a [pfSense package](https://docs.netgate.com/pfsense/en/latest/packages/list.html) with +a simple UI to configure the Security Engine and the Firewall Remediation Component (bouncer). + +Three types of setup are supported: + +**Small** (remediation only) - the pfSense machine receives blocklists from a CrowdSec +Security Engine that you are running on a different machine. +Incoming connections are blocked at the firewall by (configurable) pfSense rules. + +**Medium** (small+log processor) - in addition to enforcing blocklists, the pfSense +machine can detect attacks directed at the firewall itself, for example port scans. +The data about the attacks is sent (for analysis and possibly sharing) to a Security Engine +that you are running on a different machine. + +**Large** (medium+LAPI) - deploy a fully autonomous Security Engine on the firewall +system and allow other Log Processors to connect to it. Requires a persistent `/var` +directory (no RAM disk) and a slightly larger pfSense machine, depending on the amount +of data to be processed. + +If you are not already a CrowdSec user, the Large setup is the easiest: just leave the default +values to enable remediation, log processor and Local API. + +:::info +The CrowdSec configuration is not transferred when you restore a pfSense backup, and you'll need +to reconfigure it or backup separately. +::: + +## Installing the package + +We have submitted the package to the pfSense developers for review and inclusion in the official repositories. + +If you find `crowdsec` under `System/Package Manager`, we recommend you to install it from there. + +If the package is not available yet, you'll have to install it manually. + + * Choose [the release you want to install](https://github.com/crowdsecurity/pfSense-pkg-crowdsec/releases), + click `Assets` for the list of packages to install. + + * Open an ssh connection to your pfSense box and run the following commands in the right order. Do not activate or run the services, + because pfSense will take care of it. + +```console +# setenv IGNORE_OSVERSION yes +# pkg add -f +# pkg add -f +# pkg add -f +# pkg add -f +# pkg add -f +``` + +## Configuration + +Once the package and its dependencies are installed, go to `Service/CrowdSec`. The options *Remediation Component*, +*Log Processor* and *Local API* should be enabled. Click Save. + +With the size analogy, the default is a "Large", autonomous installation. For a "Medium", disable *Local API* and fill the fields in the *Remote LAPI* section. For a "Small", disable *Log Processor* too. + +CrowdSec on pfSense is fully functional from the command line but the web interface is read-only, with the exception of decision revocation (unban). +Most actions require the shell or the [CrowdSec Console](https://app.crowdsec.net). +For simple things, `Diagnostics/Command Prompt` works as well as ssh. +You are free to edit the files in `/usr/local/etc/crowdsec`, although some setting may be overwritten by the pfSense package if they are mandatory. + +:::caution +*Ram Disk*: unless you disable Local API, ensure that you are [not using a RAM disk](https://docs.netgate.com/pfsense/en/latest/config/advanced-misc.html#ram-disk-settings) +for the /var directory. The persistent CrowdSec database and GeoIP tables are in `/var/db`. +If you really need a RAM disk, you can still use the log processor and remediation but you will +need to connect them to a remote CrowdSec instance. +::: + + +## Service Status + +In the page `Status/CrowdSec` you can see + + - registered log processors and bouncers + - installed hub items (collections, scenarios, parsers, postoverflows) + - alerts and local decisions + +All tables are read-only with an exception: you can delete single decisions, to unban an IP for example. + +All hub objects are periodically upgraded with a cron job. + + +## Detecting attacks + +If a Log Processor is running, the following scenarios are enabled by default: + + - portscan + - ssh brute-force + - pfSense admin UI brute-force + - HTTP vulnerability probing + +These will trigger a ban on the attacking IP (4 hours by default) and report it to the CrowdSec Central API +(meaning [timestamp, scenario, attacking IP](https://docs.crowdsec.net/docs/concepts/), for inclusion in the +Community Blocklist. + +You can add scenarios to detect other types of attack on the pfSense server, or +[connect several log processors](https://doc.crowdsec.net/docs/next/user_guides/multiserver_setup) +to the same LAPI node. +Other types of remediation are possible (ex. captcha test for scraping attempts). + +If disk space is not an issue, you can [increase the maximum size](https://docs.netgate.com/pfsense/en/latest/monitoring/logs/size.html) +of log files before they are compressed and rotated. This will help us in case you report +acquisition issues and we need to match the application behavior with the content of the acquired logs. + +We recommend you to [register to the Console](https://app.crowdsec.net/), especially if you protect several machines. + + +## Processing logs + +If a collection, parser or scenario can be applied to a software that you are running on pfSense, +you add it with `cscli collections install ...`, then you need to configure where CrowdSec will find the logs. + +New acquisition files should go in `/usr/local/etc/crowdsec/acquis.d/`. See `pfsense.yaml` for an example. +The option `poll_without_inotify: true` is required if the log sources are symlinks. +Make sure to reload or restart CrowdSec when you add new data sources. + + +## Diagnostics + +Under `Diagnostics/CrowdSec` you can check if the logs are acquired and the +events are triggered correctly. For real monitoring, you can fetch the same metrics with +[Prometheus](https://docs.crowdsec.net/docs/observability/prometheus/) (Grafana dashboard included) +Telegraf or your favorite solution. + +If you are not running a LAPI or a Log Processor, some metrics are always empty. + + +## Logs + +You can see the Security Engine logs in `Status/System Logs/Packages/crowdsec`. + +Other logs not shown in the UI are in `/var/log/crowdsec/crowdsec_api.log` +and `crowdsec-firewall-bouncer.log`. + + +## Service Management + +Both services, Security Engine (crowdsec) and Remediation (crowdsec-firewall-bouncer) can be +controlled from [Status/Services](/status_services.php). +The equivalent shell commands are `service crowdsec.sh start/stop/restart` and `service crowdsec_firewall.sh start/stop/restart`. Note the ending **.sh**! + +## Viewing blocked IPs + +You can see the tables of the blocked IPs in Diagnostics/Tables or from the shell, with the commands +`pfctl -T show -t crowdsec_blacklists` (IPv4) and `pfctl -T show -t crowdsec6_blacklists` (IPv6). + +To show the same data with more context, use `cscli decisions list -a`. + +## Testing + +A quick way to test that everything is working correctly end-to-end is to +execute the following command. + +Your ssh session should freeze and you should be kicked out from +the firewall. You will not be able to connect to it (from the same +IP address) for two minutes. + +It might be a good idea to have a secondary IP from which you can +connect, should anything go wrong. + +```console +# cscli decisions add -t ban -d 2m -i +```` + +You may have to disable the *Anti-lockout* rule in +`System/Advanced/Admin Access` for the time of the test. + +This is a more secure way to test than attempting to brute-force +yourself: the default ban period is 4 hours, and CrowdSec reads the +logs from the beginning, so it could ban you even if you failed ssh +login 10 times in 30 seconds two hours before installing it. + +It must be noted that the [Login Protection](https://docs.netgate.com/pfsense/en/latest/config/advanced-admin.html#login-protection) service which is enabled by default on pfSense can be triggered - and block a brute force attacker - before CrowdSec does, because it's more sensitive. Still, some attacks that are not detected by Login Protection will be detected by CrowdSec and shared. +If you need more CrowdSec tests you may want to temporarily disable Login Protection, depending on the scenario. + +## LAN / private networks whitelist + +By default the FreeBSD version of CrowdSec does not install any whitelist. +If you trust your `10.0.0.0/8`, `192.168.0.0/16` and `172.16.0.0/12` +networks, you can use `cscli parsers install crowdsecurity/whitelists` to whitelist them. + +## Uninstalling + +In most cases, just remove the `crowdsec` package from +`System/Package Manager/Installed Packages`. +This won't remove the database or configuration files, just in case +you want to reinstall CrowdSec later. + +If you need to make sure you removed all traces of CrowdSec, you can run the following commands: + +```console +# pkg remove pfSense-pkg-crowdsec crowdsec crowdsec-firewall-bouncer +# rm -rf /usr/local/etc/crowdsec /usr/local/etc/rc.conf.d/crowdsec* +# rm -rf /var/db/crowdsec /var/log/crowdsec* /var/run/crowdsec* +``` + +For testing purposes, you may want to remove the <crowdsec> section +from `/conf/config.xml` as well. + + +## The blocklist mirror + +Before releasing the official package, one way to integrate pfSense and CrowdSec +was to install a blocklist mirror and connect it to pfBlockerNG. While this is still +a viable solution, it has slower performance than the method described above, +especially in terms of latency when receiving decision updates. +It also required pfBlockerNG in addition to CrowdSec. + +## Enrolling your instance + +The next step is to enroll your instance with the [CrowdSec Console](https://app.crowdsec.net/security-engines?enroll-engine=true). + +For the benefits, please visit the [Console section](/u/console/intro). From ebca9ee74d88a866669d384847e60c88443a3937 Mon Sep 17 00:00:00 2001 From: marco Date: Thu, 25 Jul 2024 14:40:52 +0200 Subject: [PATCH 2/9] update pfsense install --- .../docs/getting_started/install_pfsense.md | 54 ++--- .../getting_started/installation/pfsense.mdx | 214 +----------------- 2 files changed, 27 insertions(+), 241 deletions(-) diff --git a/crowdsec-docs/docs/getting_started/install_pfsense.md b/crowdsec-docs/docs/getting_started/install_pfsense.md index eb2756cc..98a28666 100644 --- a/crowdsec-docs/docs/getting_started/install_pfsense.md +++ b/crowdsec-docs/docs/getting_started/install_pfsense.md @@ -3,14 +3,13 @@ id: install_crowdsec_pfsense title: pfSense --- -We have created a [pfSense package](https://docs.netgate.com/pfsense/en/latest/packages/list.html) with -a simple UI to configure the Security Engine and the Firewall Remediation Component (bouncer). +The CrowdSec package for pfSense requires some manual installation steps, as it is not yet available in the official repositories. Three types of setup are supported: -**Small** (remediation only) - the pfSense machine receives blocklists from a CrowdSec +**Small** (remediation only) - the pfSense machine receives blocklists from a Security Engine that you are running on a different machine. -Incoming connections are blocked at the firewall by (configurable) pfSense rules. +Attacking traffic is blocked at the firewall by (configurable) pfSense rules. **Medium** (small+log processor) - in addition to enforcing blocklists, the pfSense machine can detect attacks directed at the firewall itself, for example port scans. @@ -27,17 +26,13 @@ values to enable remediation, log processor and Local API. :::info The CrowdSec configuration is not transferred when you restore a pfSense backup, and you'll need -to reconfigure it or backup separately. +to reconfigure it or backup separately. Major pfSense upgrades may also require you to re-install +or re-configure CrowdSec so please verify that it's running afterwards. We have submitted the package +for inclusion in the official repository which should smooth out these issues. ::: ## Installing the package -We have submitted the package to the pfSense developers for review and inclusion in the official repositories. - -If you find `crowdsec` under `System/Package Manager`, we recommend you to install it from there. - -If the package is not available yet, you'll have to install it manually. - * Choose [the release you want to install](https://github.com/crowdsecurity/pfSense-pkg-crowdsec/releases), click `Assets` for the list of packages to install. @@ -53,6 +48,10 @@ If the package is not available yet, you'll have to install it manually. # pkg add -f ``` +The direct links are for the most popular Community Edition of pfSense, architecture amd64. If you run on ARM or a different base version +of FreeBSD, you will find .tar files in the release assets containing the packages for the possible platforms. + + ## Configuration Once the package and its dependencies are installed, go to `Service/CrowdSec`. The options *Remediation Component*, @@ -61,7 +60,7 @@ Once the package and its dependencies are installed, go to `Service/CrowdSec`. T With the size analogy, the default is a "Large", autonomous installation. For a "Medium", disable *Local API* and fill the fields in the *Remote LAPI* section. For a "Small", disable *Log Processor* too. CrowdSec on pfSense is fully functional from the command line but the web interface is read-only, with the exception of decision revocation (unban). -Most actions require the shell or the [CrowdSec Console](https://app.crowdsec.net). +Most other actions require the shell or the [CrowdSec Console](https://app.crowdsec.net). For simple things, `Diagnostics/Command Prompt` works as well as ssh. You are free to edit the files in `/usr/local/etc/crowdsec`, although some setting may be overwritten by the pfSense package if they are mandatory. @@ -81,7 +80,8 @@ In the page `Status/CrowdSec` you can see - installed hub items (collections, scenarios, parsers, postoverflows) - alerts and local decisions -All tables are read-only with an exception: you can delete single decisions, to unban an IP for example. +All tables are read-only with an exception: you can delete decisions one by one, to unban an IP for example. +An IP may have been banned for several reasons, which counts as separate decisions. All hub objects are periodically upgraded with a cron job. @@ -181,9 +181,17 @@ If you need more CrowdSec tests you may want to temporarily disable Login Protec ## LAN / private networks whitelist -By default the FreeBSD version of CrowdSec does not install any whitelist. -If you trust your `10.0.0.0/8`, `192.168.0.0/16` and `172.16.0.0/12` -networks, you can use `cscli parsers install crowdsecurity/whitelists` to whitelist them. +Since crowdsec 1.6.3, private IP networks are whitelisted by default as well. This means for example an IP from a LAN or WAN which is on 192.168.x.y won't get blocked by a local decision +(cummunity blocklists don't contain private IPs). + +If you want to revert to the previous behavior, to block private IPs as well, you can remove the related parser. + +```console +[root@OPNsense ~]# cscli parsers remove crowdsecurity/whitelists +``` + +If on the other hand you upgrade from a version before 1.6.3, you need to install the lists yourself. + ## Uninstalling @@ -203,17 +211,3 @@ If you need to make sure you removed all traces of CrowdSec, you can run the fol For testing purposes, you may want to remove the <crowdsec> section from `/conf/config.xml` as well. - -## The blocklist mirror - -Before releasing the official package, one way to integrate pfSense and CrowdSec -was to install a blocklist mirror and connect it to pfBlockerNG. While this is still -a viable solution, it has slower performance than the method described above, -especially in terms of latency when receiving decision updates. -It also required pfBlockerNG in addition to CrowdSec. - -## Enrolling your instance - -The next step is to enroll your instance with the [CrowdSec Console](https://app.crowdsec.net/security-engines?enroll-engine=true). - -For the benefits, please visit the [Console section](/u/console/intro). diff --git a/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx b/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx index 6f065d1c..32e388de 100644 --- a/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx +++ b/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx @@ -5,217 +5,9 @@ pagination_prev: getting_started/pre_requisites pagination_next: getting_started/next_steps --- -We have created a [pfSense package](https://docs.netgate.com/pfsense/en/latest/packages/list.html) with -a simple UI to configure the Security Engine and the Firewall Remediation Component (bouncer). +# pfSense package -Three types of setup are supported: +The CrowdSec package for pfSense is not installed from the official repositories, at least not yet. -**Small** (remediation only) - the pfSense machine receives blocklists from a CrowdSec -Security Engine that you are running on a different machine. -Incoming connections are blocked at the firewall by (configurable) pfSense rules. +Please refer to the [detailed documentation](/getting_started/install_pfsense.md) to install or update it from a release archive. -**Medium** (small+log processor) - in addition to enforcing blocklists, the pfSense -machine can detect attacks directed at the firewall itself, for example port scans. -The data about the attacks is sent (for analysis and possibly sharing) to a Security Engine -that you are running on a different machine. - -**Large** (medium+LAPI) - deploy a fully autonomous Security Engine on the firewall -system and allow other Log Processors to connect to it. Requires a persistent `/var` -directory (no RAM disk) and a slightly larger pfSense machine, depending on the amount -of data to be processed. - -If you are not already a CrowdSec user, the Large setup is the easiest: just leave the default -values to enable remediation, log processor and Local API. - -:::info -The CrowdSec configuration is not transferred when you restore a pfSense backup, and you'll need -to reconfigure it or backup separately. -::: - -## Installing the package - -We have submitted the package to the pfSense developers for review and inclusion in the official repositories. - -If you find `crowdsec` under `System/Package Manager`, we recommend you to install it from there. - -If the package is not available yet, you'll have to install it manually. - - * Choose [the release you want to install](https://github.com/crowdsecurity/pfSense-pkg-crowdsec/releases), - click `Assets` for the list of packages to install. - - * Open an ssh connection to your pfSense box and run the following commands in the right order. Do not activate or run the services, - because pfSense will take care of it. - -```console -# setenv IGNORE_OSVERSION yes -# pkg add -f -# pkg add -f -# pkg add -f -# pkg add -f -# pkg add -f -``` - -## Configuration - -Once the package and its dependencies are installed, go to `Service/CrowdSec`. The options *Remediation Component*, -*Log Processor* and *Local API* should be enabled. Click Save. - -With the size analogy, the default is a "Large", autonomous installation. For a "Medium", disable *Local API* and fill the fields in the *Remote LAPI* section. For a "Small", disable *Log Processor* too. - -CrowdSec on pfSense is fully functional from the command line but the web interface is read-only, with the exception of decision revocation (unban). -Most actions require the shell or the [CrowdSec Console](https://app.crowdsec.net). -For simple things, `Diagnostics/Command Prompt` works as well as ssh. -You are free to edit the files in `/usr/local/etc/crowdsec`, although some setting may be overwritten by the pfSense package if they are mandatory. - -:::caution -*Ram Disk*: unless you disable Local API, ensure that you are [not using a RAM disk](https://docs.netgate.com/pfsense/en/latest/config/advanced-misc.html#ram-disk-settings) -for the /var directory. The persistent CrowdSec database and GeoIP tables are in `/var/db`. -If you really need a RAM disk, you can still use the log processor and remediation but you will -need to connect them to a remote CrowdSec instance. -::: - - -## Service Status - -In the page `Status/CrowdSec` you can see - - - registered log processors and bouncers - - installed hub items (collections, scenarios, parsers, postoverflows) - - alerts and local decisions - -All tables are read-only with an exception: you can delete single decisions, to unban an IP for example. - -All hub objects are periodically upgraded with a cron job. - - -## Detecting attacks - -If a Log Processor is running, the following scenarios are enabled by default: - - - portscan - - ssh brute-force - - pfSense admin UI brute-force - - HTTP vulnerability probing - -These will trigger a ban on the attacking IP (4 hours by default) and report it to the CrowdSec Central API -(meaning [timestamp, scenario, attacking IP](https://docs.crowdsec.net/docs/concepts/), for inclusion in the -Community Blocklist. - -You can add scenarios to detect other types of attack on the pfSense server, or -[connect several log processors](https://doc.crowdsec.net/docs/next/user_guides/multiserver_setup) -to the same LAPI node. -Other types of remediation are possible (ex. captcha test for scraping attempts). - -If disk space is not an issue, you can [increase the maximum size](https://docs.netgate.com/pfsense/en/latest/monitoring/logs/size.html) -of log files before they are compressed and rotated. This will help us in case you report -acquisition issues and we need to match the application behavior with the content of the acquired logs. - -We recommend you to [register to the Console](https://app.crowdsec.net/), especially if you protect several machines. - - -## Processing logs - -If a collection, parser or scenario can be applied to a software that you are running on pfSense, -you add it with `cscli collections install ...`, then you need to configure where CrowdSec will find the logs. - -New acquisition files should go in `/usr/local/etc/crowdsec/acquis.d/`. See `pfsense.yaml` for an example. -The option `poll_without_inotify: true` is required if the log sources are symlinks. -Make sure to reload or restart CrowdSec when you add new data sources. - - -## Diagnostics - -Under `Diagnostics/CrowdSec` you can check if the logs are acquired and the -events are triggered correctly. For real monitoring, you can fetch the same metrics with -[Prometheus](https://docs.crowdsec.net/docs/observability/prometheus/) (Grafana dashboard included) -Telegraf or your favorite solution. - -If you are not running a LAPI or a Log Processor, some metrics are always empty. - - -## Logs - -You can see the Security Engine logs in `Status/System Logs/Packages/crowdsec`. - -Other logs not shown in the UI are in `/var/log/crowdsec/crowdsec_api.log` -and `crowdsec-firewall-bouncer.log`. - - -## Service Management - -Both services, Security Engine (crowdsec) and Remediation (crowdsec-firewall-bouncer) can be -controlled from [Status/Services](/status_services.php). -The equivalent shell commands are `service crowdsec.sh start/stop/restart` and `service crowdsec_firewall.sh start/stop/restart`. Note the ending **.sh**! - -## Viewing blocked IPs - -You can see the tables of the blocked IPs in Diagnostics/Tables or from the shell, with the commands -`pfctl -T show -t crowdsec_blacklists` (IPv4) and `pfctl -T show -t crowdsec6_blacklists` (IPv6). - -To show the same data with more context, use `cscli decisions list -a`. - -## Testing - -A quick way to test that everything is working correctly end-to-end is to -execute the following command. - -Your ssh session should freeze and you should be kicked out from -the firewall. You will not be able to connect to it (from the same -IP address) for two minutes. - -It might be a good idea to have a secondary IP from which you can -connect, should anything go wrong. - -```console -# cscli decisions add -t ban -d 2m -i -```` - -You may have to disable the *Anti-lockout* rule in -`System/Advanced/Admin Access` for the time of the test. - -This is a more secure way to test than attempting to brute-force -yourself: the default ban period is 4 hours, and CrowdSec reads the -logs from the beginning, so it could ban you even if you failed ssh -login 10 times in 30 seconds two hours before installing it. - -It must be noted that the [Login Protection](https://docs.netgate.com/pfsense/en/latest/config/advanced-admin.html#login-protection) service which is enabled by default on pfSense can be triggered - and block a brute force attacker - before CrowdSec does, because it's more sensitive. Still, some attacks that are not detected by Login Protection will be detected by CrowdSec and shared. -If you need more CrowdSec tests you may want to temporarily disable Login Protection, depending on the scenario. - -## LAN / private networks whitelist - -By default the FreeBSD version of CrowdSec does not install any whitelist. -If you trust your `10.0.0.0/8`, `192.168.0.0/16` and `172.16.0.0/12` -networks, you can use `cscli parsers install crowdsecurity/whitelists` to whitelist them. - -## Uninstalling - -In most cases, just remove the `crowdsec` package from -`System/Package Manager/Installed Packages`. -This won't remove the database or configuration files, just in case -you want to reinstall CrowdSec later. - -If you need to make sure you removed all traces of CrowdSec, you can run the following commands: - -```console -# pkg remove pfSense-pkg-crowdsec crowdsec crowdsec-firewall-bouncer -# rm -rf /usr/local/etc/crowdsec /usr/local/etc/rc.conf.d/crowdsec* -# rm -rf /var/db/crowdsec /var/log/crowdsec* /var/run/crowdsec* -``` - -For testing purposes, you may want to remove the <crowdsec> section -from `/conf/config.xml` as well. - - -## The blocklist mirror - -Before releasing the official package, one way to integrate pfSense and CrowdSec -was to install a blocklist mirror and connect it to pfBlockerNG. While this is still -a viable solution, it has slower performance than the method described above, -especially in terms of latency when receiving decision updates. -It also required pfBlockerNG in addition to CrowdSec. - -## Enrolling your instance - -The next step is to enroll your instance with the [CrowdSec Console](https://app.crowdsec.net/security-engines?enroll-engine=true). - -For the benefits, please visit the [Console section](/u/console/intro). From 395477d6c2f1fa4c12c106642a5684d6768029b5 Mon Sep 17 00:00:00 2001 From: Laurence Jones Date: Tue, 30 Jul 2024 10:06:11 +0100 Subject: [PATCH 3/9] Update opnsense.mdx --- .../unversioned/getting_started/installation/opnsense.mdx | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx b/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx index 9735d813..9545a2b1 100644 --- a/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx +++ b/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx @@ -28,5 +28,7 @@ Do not enable/start the services from the terminal like you would on a standard Refresh the page and go to `Services > CrowdSec > Overview` to verify the running services and installed configurations. +## Next Steps? + Great, you now have CrowdSec installed on your system. Have a look at the [post installation steps](/getting_started/next_steps.md) to see how to to configure and optimize it, -or keep reading [the rest of the plugin documentation](/getting_started/install_opnsense.md) +or keep reading [the rest of the plugin documentation](/docs/next/getting_started/install_crowdsec_opnsense) From 0a56d4c6ede4fe35194f33457c5666ec369a63cb Mon Sep 17 00:00:00 2001 From: Laurence Jones Date: Tue, 30 Jul 2024 10:06:36 +0100 Subject: [PATCH 4/9] Update opnsense.mdx --- .../unversioned/getting_started/installation/opnsense.mdx | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx b/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx index 9545a2b1..d19c29da 100644 --- a/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx +++ b/crowdsec-docs/unversioned/getting_started/installation/opnsense.mdx @@ -30,5 +30,4 @@ Refresh the page and go to `Services > CrowdSec > Overview` to verify the runnin ## Next Steps? -Great, you now have CrowdSec installed on your system. Have a look at the [post installation steps](/getting_started/next_steps.md) to see how to to configure and optimize it, -or keep reading [the rest of the plugin documentation](/docs/next/getting_started/install_crowdsec_opnsense) +Great, you now have CrowdSec installed on your system. Have a look at the [post installation steps](/getting_started/next_steps.md) to see how to to configure and optimize it, or keep reading [the rest of the plugin documentation](/docs/next/getting_started/install_crowdsec_opnsense) From bb41db632bb8b9ce938910232bff28a44bce7e63 Mon Sep 17 00:00:00 2001 From: Laurence Jones Date: Tue, 30 Jul 2024 10:07:05 +0100 Subject: [PATCH 5/9] Update pfsense.mdx --- .../unversioned/getting_started/installation/pfsense.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx b/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx index 32e388de..4253cae0 100644 --- a/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx +++ b/crowdsec-docs/unversioned/getting_started/installation/pfsense.mdx @@ -9,5 +9,5 @@ pagination_next: getting_started/next_steps The CrowdSec package for pfSense is not installed from the official repositories, at least not yet. -Please refer to the [detailed documentation](/getting_started/install_pfsense.md) to install or update it from a release archive. +Please refer to the [detailed documentation](/docs/next/getting_started/install_crowdsec_pfsense) to install or update it from a release archive. From 90a8aea9dc6a8ce46fb9fd59b7d04d72b544ac5f Mon Sep 17 00:00:00 2001 From: Laurence Jones Date: Tue, 30 Jul 2024 10:08:41 +0100 Subject: [PATCH 6/9] Update QuickStart.js --- crowdsec-docs/src/components/QuickStart.js | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/crowdsec-docs/src/components/QuickStart.js b/crowdsec-docs/src/components/QuickStart.js index a1e1b074..4c185336 100644 --- a/crowdsec-docs/src/components/QuickStart.js +++ b/crowdsec-docs/src/components/QuickStart.js @@ -43,16 +43,16 @@ const staticData = [ text: "Kubernetes", link: "/u/getting_started/installation/kubernetes", }, - // { - // icon: opnsenseLogo, - // text: "OPNsense", - // link: "/u/getting_started/installation/opnsense", - // }, - // { - // icon: pfSenseLogo, - // text: "pfSense", - // link: "/u/getting_started/installation/pfsense", - // }, + { + icon: opnsenseLogo, + text: "OPNsense", + link: "/u/getting_started/installation/opnsense", + }, + { + icon: pfSenseLogo, + text: "pfSense", + link: "/u/getting_started/installation/pfsense", + }, { icon: whmLogo, text: "WHM", From 26198a758dee22355589e8e1dcdd2b3fe2ac2836 Mon Sep 17 00:00:00 2001 From: Laurence Date: Tue, 30 Jul 2024 10:16:28 +0100 Subject: [PATCH 7/9] enhance: Some reason sidebar was bugged, fixed --- crowdsec-docs/sidebarsUnversioned.js | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/crowdsec-docs/sidebarsUnversioned.js b/crowdsec-docs/sidebarsUnversioned.js index 5530e84e..e97f2145 100644 --- a/crowdsec-docs/sidebarsUnversioned.js +++ b/crowdsec-docs/sidebarsUnversioned.js @@ -425,14 +425,12 @@ module.exports = { items: [ "getting_started/installation/linux", "getting_started/installation/freebsd", - "getting_started/installation/opnsense", - "getting_started/installation/pfsense", "getting_started/installation/windows", "getting_started/installation/macos", "getting_started/installation/docker", "getting_started/installation/kubernetes", - // "getting_started/installation/pfsense", - // "getting_started/installation/opnsense", + "getting_started/installation/pfsense", + "getting_started/installation/opnsense", "getting_started/installation/whm", ], }, From 323d5e5a22aabf4a874007a7d1399fef63da0ef7 Mon Sep 17 00:00:00 2001 From: Laurence Jones Date: Tue, 30 Jul 2024 10:55:49 +0100 Subject: [PATCH 8/9] Update install_opnsense.md --- crowdsec-docs/docs/getting_started/install_opnsense.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/crowdsec-docs/docs/getting_started/install_opnsense.md b/crowdsec-docs/docs/getting_started/install_opnsense.md index b8c40a5c..079c475c 100644 --- a/crowdsec-docs/docs/getting_started/install_opnsense.md +++ b/crowdsec-docs/docs/getting_started/install_opnsense.md @@ -39,7 +39,7 @@ You can disable them for testing or if you have special requirements. The parsers, scenarios and all objects from the [CrowdSec Hub](https://hub.crowdsec.net/) are periodically upgraded. The [crowdsecurity/freebsd](https://hub.crowdsec.net/author/crowdsecurity/collections/freebsd) and [crowdsecurity/opnsense](https://hub.crowdsec.net/author/crowdsecurity/collections/opnsense) collections are installed by default. Since crowdsec 1.6.3, private IP networks are whitelisted by default as well. This means for example an IP from a LAN or WAN which is on 192.168.x.y won't get blocked by a local decision -(cummunity blocklists don't contain private IPs). +(community blocklists don't contain private IPs). If you want to revert to the previous behavior, to block private IPs as well, you can remove the related parser. @@ -49,7 +49,6 @@ If you want to revert to the previous behavior, to block private IPs as well, yo If on the other hand you upgrade from a version before 1.6.3, you need to install the lists yourself. - ## Testing the remediation component A quick way to test that everything is working correctly is to execute the following command. Your ssh session should freeze and you should be kicked out from the firewall. You will not be able to connect to it (from the same IP address) for two minutes. It might be a good idea to have a secondary IP from which you can connect, should anything go wrong. From 2bfe0e8be956503e708404286ce5a8a35cd9b226 Mon Sep 17 00:00:00 2001 From: Laurence Jones Date: Tue, 30 Jul 2024 10:59:00 +0100 Subject: [PATCH 9/9] Update install_pfsense.md --- crowdsec-docs/docs/getting_started/install_pfsense.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crowdsec-docs/docs/getting_started/install_pfsense.md b/crowdsec-docs/docs/getting_started/install_pfsense.md index 98a28666..7ee1ccef 100644 --- a/crowdsec-docs/docs/getting_started/install_pfsense.md +++ b/crowdsec-docs/docs/getting_started/install_pfsense.md @@ -182,7 +182,7 @@ If you need more CrowdSec tests you may want to temporarily disable Login Protec ## LAN / private networks whitelist Since crowdsec 1.6.3, private IP networks are whitelisted by default as well. This means for example an IP from a LAN or WAN which is on 192.168.x.y won't get blocked by a local decision -(cummunity blocklists don't contain private IPs). +(community blocklists don't contain private IPs). If you want to revert to the previous behavior, to block private IPs as well, you can remove the related parser.