ipynb/vagrant-snort-nfqueue-tutorial-centos7.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Description\n",
    "\n",
    "An example of `snort++` (https://www.snort.org/snort3) network Intrusion Detection and Prevention System (ID/IPS) deployed on an endpoint Apache host.\n",
    "\n",
    "`snort` belongs to the class of rule-based network IDS/IPS systems. The `snort` rules work similarly to virus signatures:\n",
    "like a virus scanner, `snort` can only detect a network traffic for which a rule is written in advance.\n",
    "\n",
    "The instance of running `snort` software is called a sensor.\n",
    "In order for `snort` to serve as an IPS the traffic must pass through the sensor.\n",
    "`snort` then applies rules in order to block and/or log selected traffic.\n",
    "\n",
    "`snort` IPS has been initially designed for sensor placing on a separate Linux machine, through which the traffic will pass\n",
    "over a network bridge [https://openmaniak.com/inline_bridge.php](https://openmaniak.com/inline_bridge.php).\n",
    "\n",
    "In order to place the sensor on an end-point host (the destination of the traffic,\n",
    "or e.g. a load balancer) the recommended method is to send all traffic to be monitored\n",
    "to `nfqueue` on this host and let `snort` apply rule based decisions to the packets present in `nfqueue`\n",
    "[https://sourceforge.net/p/snort/mailman/message/35461638/](https://sourceforge.net/p/snort/mailman/message/35461638/)\n",
    "\n",
    "In this setup the `nfqueue` (https://home.regit.org/netfilter-en/using-nfqueue-and-libnetfilter_queue/) `iptables` target is used to enable\n",
    "the intrusion prevention capability of `snort`, and the `prometheus` (https://prometheus.io/) time-series database is used for monitoring of `snort` alerts.\n",
    "\n",
    "`snort++` (or `snort3`), planned at least since 2005 (Snort Coockbook, p. 179) supports \"multiple packet processing threads\"\n",
    "and several other parts of the code base have been rewritten compared to `snort2`. This setup uses `snort++`, though it is still *alpha*. After following the steps during this setup it is rather easy to switch to the more reliable `snort2`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Configuration overview\n",
    "\n",
    "The private 192.168.17.0/24 network is associated with the Vagrant eth1 interfaces.\n",
    "eth0 is used by Vagrant.  Vagrant reserves eth0 and this cannot be currently changed\n",
    "(see https://github.com/mitchellh/vagrant/issues/2093).\n",
    "\n",
    "\n",
    "                                                    -----------------\n",
    "                                                    | monitor0      |\n",
    "                                                    |               |\n",
    "                                                    | prometheus    |\n",
    "                                                    -----------------\n",
    "                                                     |     |\n",
    "                                                 eth1| eth0|\n",
    "                                        192.168.17.20|     |\n",
    "                 -----------------------------------------------------------------\n",
    "                 |                                  Vagrant HOST                 |\n",
    "                 -----------------------------------------------------------------\n",
    "                  |     |                                         |     |         \n",
    "              eth1| eth0|                                     eth1| eth0|         \n",
    "     192.168.17.30|     |                            192.168.17.10|     |         \n",
    "                 -----------------                               -----------------\n",
    "                 | snort0        |                               | attacker0     |\n",
    "                 |               |                               |               |\n",
    "                 | snort         |                               -----------------\n",
    "                 | node_exporter |\n",
    "                 | apache        |\n",
    "                 -----------------\n",
    "\n",
    "Note that the target Apache host and the attacker machines reside on the same private network, while on the internet all addresses would be public.\n",
    "This is in order to simplify routing, and it does no affect the `snort` configuration, i.e. a host configured like **snort0** would perform it's job if put on the internet.\n",
    "\n",
    "`snort++` offers a multithreading mode. Note however that `snort`'s multithreading does not mean that a single `snort`\n",
    "instance will internally load balance the traffic and process in multiple threads http://seclists.org/snort/2015/q2/90,\n",
    "but instead multithreading means running multiple `snort` processes attached to different network interfaces, like one would do with `snort2` (is this statement correct?).\n",
    "Obviously such multithreading configuration is not applicable to this setup."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Sample Usage\n",
    "\n",
    "Install VirtualBox https://www.virtualbox.org/ and Vagrant https://www.vagrantup.com/downloads.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Retrive the repository under the `jupyter` current working directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/scratch/ubuntu/jupyter\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Cloning into 'vagrant-snort-nfqueue-tutorial-centos7'...\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "git clone https://github.com/marcindulak/vagrant-snort-nfqueue-tutorial-centos7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Copy the `snort` setup files into the `jupyter` working directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'vagrant-snort-nfqueue-tutorial-centos7/configure_node_exporter_on_snort.sh' -> './configure_node_exporter_on_snort.sh'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/configure_prometheus_on_monitor.sh' -> './configure_prometheus_on_monitor.sh'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/copr-marcindulak-snort.repo' -> './copr-marcindulak-snort.repo'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/ipynb/README.md' -> './ipynb/README.md'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/ipynb/vagrant-snort-nfqueue-tutorial-centos7.ipynb' -> './ipynb/vagrant-snort-nfqueue-tutorial-centos7.ipynb'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/ipynb/vagrant-snort-nfqueue-tutorial-centos7.md' -> './ipynb/vagrant-snort-nfqueue-tutorial-centos7.md'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/LICENSE' -> './LICENSE'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/node_exporter.json' -> './node_exporter.json'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/pytbull_config.cfg' -> './pytbull_config.cfg'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/README.md' -> './README.md'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/snort-alert-count.py' -> './snort-alert-count.py'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/snort-alert-count.service' -> './snort-alert-count.service'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/snort-test-alert.service' -> './snort-test-alert.service'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/spec/daq/daq.spec' -> './spec/daq/daq.spec'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/spec/snort-extra/snort-extra.spec' -> './spec/snort-extra/snort-extra.spec'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/spec/snort/snort.service' -> './spec/snort/snort.service'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/spec/snort/snort.spec' -> './spec/snort/snort.spec'\n",
      "'vagrant-snort-nfqueue-tutorial-centos7/Vagrantfile' -> './Vagrantfile'\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "cp -rvf vagrant-snort-nfqueue-tutorial-centos7/* ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Bring up the VMs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    " Note that the output appears only when the shell exits, be patient. It should take about 5-20 minutes on a modern hardware."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bringing machine 'attacker0' up with 'virtualbox' provider...\n",
      "Bringing machine 'monitor0' up with 'virtualbox' provider...\n",
      "Bringing machine 'snort0' up with 'virtualbox' provider...\n",
      "==> attacker0: Importing base box 'bento/centos-7.3'...\n",
      "\r",
      "\u001b[KProgress: 20%\r",
      "\u001b[KProgress: 30%\r",
      "\u001b[KProgress: 50%\r",
      "\u001b[KProgress: 60%\r",
      "\u001b[KProgress: 70%\r",
      "\u001b[KProgress: 90%\r",
      "\u001b[K==> attacker0: Matching MAC address for NAT networking...\n",
      "==> attacker0: Checking if box 'bento/centos-7.3' is up to date...\n",
      "==> attacker0: Setting the name of the VM: jupyter_attacker0_1493823063769_39738\n",
      "==> attacker0: Fixed port collision for 22 => 2222. Now on port 2201.\n",
      "==> attacker0: Clearing any previously set network interfaces...\n",
      "==> attacker0: Preparing network interfaces based on configuration...\n",
      "    attacker0: Adapter 1: nat\n",
      "    attacker0: Adapter 2: hostonly\n",
      "==> attacker0: Forwarding ports...\n",
      "    attacker0: 22 (guest) => 2201 (host) (adapter 1)\n",
      "==> attacker0: Running 'pre-boot' VM customizations...\n",
      "==> attacker0: Booting VM...\n",
      "==> attacker0: Waiting for machine to boot. This may take a few minutes...\n",
      "    attacker0: SSH address: 127.0.0.1:2201\n",
      "    attacker0: SSH username: vagrant\n",
      "    attacker0: SSH auth method: private key\n",
      "    attacker0: Warning: Remote connection disconnect. Retrying...\n",
      "    attacker0: Warning: Remote connection disconnect. Retrying...\n",
      "    attacker0: Warning: Remote connection disconnect. Retrying...\n",
      "    attacker0: \n",
      "    attacker0: Vagrant insecure key detected. Vagrant will automatically replace\n",
      "    attacker0: this with a newly generated keypair for better security.\n",
      "    attacker0: \n",
      "    attacker0: Inserting generated public key within guest...\n",
      "    attacker0: Removing insecure key from the guest if it's present...\n",
      "    attacker0: Key inserted! Disconnecting and reconnecting using new SSH key...\n",
      "==> attacker0: Machine booted and ready!\n",
      "==> attacker0: Checking for guest additions in VM...\n",
      "==> attacker0: Configuring and enabling network interfaces...\n",
      "==> attacker0: Automatic installation for Landrush IP not enabled\n",
      "==> attacker0: Mounting shared folders...\n",
      "    attacker0: /vagrant => /scratch/ubuntu/jupyter\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: \n",
      "==> attacker0: ================================================================================\n",
      "==> attacker0:  Package            Arch         Version   Repository                      Size\n",
      "==> attacker0: ================================================================================\n",
      "==> attacker0: Installing:\n",
      "==> attacker0:  epel-release       noarch       7-9       /epel-release-7-9.noarch        24 k\n",
      "==> attacker0: \n",
      "==> attacker0: Transaction Summary\n",
      "==> attacker0: ================================================================================\n",
      "==> attacker0: Install  1 Package\n",
      "==> attacker0: \n",
      "==> attacker0: Total size: 24 k\n",
      "==> attacker0: Installed size: 24 k\n",
      "==> attacker0: \n",
      "==> attacker0: Installed:\n",
      "==> attacker0:   epel-release.noarch 0:7-9                                                     \n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: net.ipv6.conf.default.disable_ipv6 = 1\n",
      "==> attacker0: net.ipv6.conf.all.disable_ipv6 = 1\n",
      "==> attacker0: net.ipv6.conf.lo.disable_ipv6 = 1\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: \n",
      "==> attacker0: ================================================================================\n",
      "==> attacker0:  Package             Arch       Version                       Repository   Size\n",
      "==> attacker0: ================================================================================\n",
      "==> attacker0: Installing:\n",
      "==> attacker0:  ntp                 x86_64     4.2.6p5-25.el7.centos.2       updates     547 k\n",
      "==> attacker0: Installing for dependencies:\n",
      "==> attacker0:  autogen-libopts     x86_64     5.18-5.el7                    base         66 k\n",
      "==> attacker0:  ntpdate             x86_64     4.2.6p5-25.el7.centos.2       updates      86 k\n",
      "==> attacker0: \n",
      "==> attacker0: Transaction Summary\n",
      "==> attacker0: ================================================================================\n",
      "==> attacker0: Install  1 Package (+2 Dependent packages)\n",
      "==> attacker0: \n",
      "==> attacker0: Total download size: 699 k\n",
      "==> attacker0: Installed size: 1.6 M\n",
      "==> attacker0: Public key for ntpdate-4.2.6p5-25.el7.centos.2.x86_64.rpm is not installed\n",
      "==> attacker0: warning: /var/cache/yum/x86_64/7/updates/packages/ntpdate-4.2.6p5-25.el7.centos.2.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY\n",
      "==> attacker0: Public key for autogen-libopts-5.18-5.el7.x86_64.rpm is not installed\n",
      "==> attacker0: Importing GPG key 0xF4A80EB5:\n",
      "==> attacker0:  Userid     : \"CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>\"\n",
      "==> attacker0:  Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5\n",
      "==> attacker0:  Package    : centos-release-7-3.1611.el7.centos.x86_64 (@anaconda)\n",
      "==> attacker0:  From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7\n",
      "==> attacker0: \n",
      "==> attacker0: Installed:\n",
      "==> attacker0:   ntp.x86_64 0:4.2.6p5-25.el7.centos.2                                          \n",
      "==> attacker0: \n",
      "==> attacker0: Dependency Installed:\n",
      "==> attacker0:   autogen-libopts.x86_64 0:5.18-5.el7  ntpdate.x86_64 0:4.2.6p5-25.el7.centos.2 \n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> attacker0: Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.\n",
      "==> attacker0: Running provisioner: shell...\n",
      "    attacker0: Running: inline script\n",
      "==> monitor0: Importing base box 'bento/centos-7.3'...\n",
      "\r",
      "\u001b[KProgress: 20%\r",
      "\u001b[KProgress: 30%\r",
      "\u001b[KProgress: 50%\r",
      "\u001b[KProgress: 70%\r",
      "\u001b[KProgress: 90%\r",
      "\u001b[K==> monitor0: Matching MAC address for NAT networking...\n",
      "==> monitor0: Checking if box 'bento/centos-7.3' is up to date...\n",
      "==> monitor0: Setting the name of the VM: jupyter_monitor0_1493823171794_75741\n",
      "==> monitor0: Fixed port collision for 22 => 2222. Now on port 2203.\n",
      "==> monitor0: Clearing any previously set network interfaces...\n",
      "==> monitor0: Preparing network interfaces based on configuration...\n",
      "    monitor0: Adapter 1: nat\n",
      "    monitor0: Adapter 2: hostonly\n",
      "==> monitor0: Forwarding ports...\n",
      "    monitor0: 22 (guest) => 2203 (host) (adapter 1)\n",
      "==> monitor0: Running 'pre-boot' VM customizations...\n",
      "==> monitor0: Booting VM...\n",
      "==> monitor0: Waiting for machine to boot. This may take a few minutes...\n",
      "    monitor0: SSH address: 127.0.0.1:2203\n",
      "    monitor0: SSH username: vagrant\n",
      "    monitor0: SSH auth method: private key\n",
      "    monitor0: Warning: Remote connection disconnect. Retrying...\n",
      "    monitor0: Warning: Remote connection disconnect. Retrying...\n",
      "    monitor0: Warning: Remote connection disconnect. Retrying...\n",
      "    monitor0: \n",
      "    monitor0: Vagrant insecure key detected. Vagrant will automatically replace\n",
      "    monitor0: this with a newly generated keypair for better security.\n",
      "    monitor0: \n",
      "    monitor0: Inserting generated public key within guest...\n",
      "    monitor0: Removing insecure key from the guest if it's present...\n",
      "    monitor0: Key inserted! Disconnecting and reconnecting using new SSH key...\n",
      "==> monitor0: Machine booted and ready!\n",
      "==> monitor0: Checking for guest additions in VM...\n",
      "==> monitor0: Configuring and enabling network interfaces...\n",
      "==> monitor0: Automatic installation for Landrush IP not enabled\n",
      "==> monitor0: Mounting shared folders...\n",
      "    monitor0: /vagrant => /scratch/ubuntu/jupyter\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: \n",
      "==> monitor0: ================================================================================\n",
      "==> monitor0:  Package            Arch         Version   Repository                      Size\n",
      "==> monitor0: ================================================================================\n",
      "==> monitor0: Installing:\n",
      "==> monitor0:  epel-release       noarch       7-9       /epel-release-7-9.noarch        24 k\n",
      "==> monitor0: \n",
      "==> monitor0: Transaction Summary\n",
      "==> monitor0: ================================================================================\n",
      "==> monitor0: Install  1 Package\n",
      "==> monitor0: \n",
      "==> monitor0: Total size: 24 k\n",
      "==> monitor0: Installed size: 24 k\n",
      "==> monitor0: \n",
      "==> monitor0: Installed:\n",
      "==> monitor0:   epel-release.noarch 0:7-9                                                     \n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Detected operating system as centos/7.\n",
      "==> monitor0: Checking for curl...\n",
      "==> monitor0: Detected curl...\n",
      "==> monitor0: Downloading repository file: https://packagecloud.io/install/repositories/prometheus-rpm/release/config_file.repo?os=centos&dist=7&source=script\n",
      "==> monitor0: done.\n",
      "==> monitor0: Installing pygpgme to verify GPG signatures...\n",
      "==> monitor0: Importing GPG key 0x7457CCD1:\n",
      "==> monitor0:  Userid     : \"https://packagecloud.io/prometheus-rpm/centos (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>\"\n",
      "==> monitor0:  Fingerprint: d7df c575 af4b 92c6 e62a d3c5 f721 6757 7457 ccd1\n",
      "==> monitor0:  From       : https://packagecloud.io/prometheus-rpm/release/gpgkey\n",
      "==> monitor0: Package pygpgme-0.3-9.el7.x86_64 already installed and latest version\n",
      "==> monitor0: Installing yum-utils...\n",
      "==> monitor0: Public key for libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm is not installed\n",
      "==> monitor0: warning: /var/cache/yum/x86_64/7/base/packages/libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY\n",
      "==> monitor0: Importing GPG key 0xF4A80EB5:\n",
      "==> monitor0:  Userid     : \"CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>\"\n",
      "==> monitor0:  Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5\n",
      "==> monitor0:  Package    : centos-release-7-3.1611.el7.centos.x86_64 (@anaconda)\n",
      "==> monitor0:  From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7\n",
      "==> monitor0: Generating yum cache for prometheus-rpm_release...\n",
      "==> monitor0: Importing GPG key 0x7457CCD1:\n",
      "==> monitor0:  Userid     : \"https://packagecloud.io/prometheus-rpm/centos (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>\"\n",
      "==> monitor0:  Fingerprint: d7df c575 af4b 92c6 e62a d3c5 f721 6757 7457 ccd1\n",
      "==> monitor0:  From       : https://packagecloud.io/prometheus-rpm/release/gpgkey\n",
      "==> monitor0: \n",
      "==> monitor0: The repository is setup! You can now install packages.\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: net.ipv6.conf.default.disable_ipv6 = 1\n",
      "==> monitor0: net.ipv6.conf.all.disable_ipv6 = 1\n",
      "==> monitor0: net.ipv6.conf.lo.disable_ipv6 = 1\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: \n",
      "==> monitor0: ================================================================================\n",
      "==> monitor0:  Package             Arch       Version                       Repository   Size\n",
      "==> monitor0: ================================================================================\n",
      "==> monitor0: Installing:\n",
      "==> monitor0:  ntp                 x86_64     4.2.6p5-25.el7.centos.2       updates     547 k\n",
      "==> monitor0: Installing for dependencies:\n",
      "==> monitor0:  autogen-libopts     x86_64     5.18-5.el7                    base         66 k\n",
      "==> monitor0:  ntpdate             x86_64     4.2.6p5-25.el7.centos.2       updates      86 k\n",
      "==> monitor0: \n",
      "==> monitor0: Transaction Summary\n",
      "==> monitor0: ================================================================================\n",
      "==> monitor0: Install  1 Package (+2 Dependent packages)\n",
      "==> monitor0: \n",
      "==> monitor0: Total download size: 699 k\n",
      "==> monitor0: Installed size: 1.6 M\n",
      "==> monitor0: \n",
      "==> monitor0: Installed:\n",
      "==> monitor0:   ntp.x86_64 0:4.2.6p5-25.el7.centos.2                                          \n",
      "==> monitor0: \n",
      "==> monitor0: Dependency Installed:\n",
      "==> monitor0:   autogen-libopts.x86_64 0:5.18-5.el7  ntpdate.x86_64 0:4.2.6p5-25.el7.centos.2 \n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> monitor0: Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.\n",
      "==> monitor0: Running provisioner: shell...\n",
      "    monitor0: Running: inline script\n",
      "==> snort0: Importing base box 'bento/centos-7.3'...\n",
      "\r",
      "\u001b[KProgress: 20%\r",
      "\u001b[KProgress: 30%\r",
      "\u001b[KProgress: 50%\r",
      "\u001b[KProgress: 70%\r",
      "\u001b[KProgress: 90%\r",
      "\u001b[K==> snort0: Matching MAC address for NAT networking...\n",
      "==> snort0: Checking if box 'bento/centos-7.3' is up to date...\n",
      "==> snort0: Setting the name of the VM: jupyter_snort0_1493823276880_94113\n",
      "==> snort0: Fixed port collision for 22 => 2222. Now on port 2204.\n",
      "==> snort0: Clearing any previously set network interfaces...\n",
      "==> snort0: Preparing network interfaces based on configuration...\n",
      "    snort0: Adapter 1: nat\n",
      "    snort0: Adapter 2: hostonly\n",
      "==> snort0: Forwarding ports...\n",
      "    snort0: 22 (guest) => 2204 (host) (adapter 1)\n",
      "==> snort0: Running 'pre-boot' VM customizations...\n",
      "==> snort0: Booting VM...\n",
      "==> snort0: Waiting for machine to boot. This may take a few minutes...\n",
      "    snort0: SSH address: 127.0.0.1:2204\n",
      "    snort0: SSH username: vagrant\n",
      "    snort0: SSH auth method: private key\n",
      "    snort0: Warning: Remote connection disconnect. Retrying...\n",
      "    snort0: Warning: Remote connection disconnect. Retrying...\n",
      "    snort0: Warning: Remote connection disconnect. Retrying...\n",
      "    snort0: \n",
      "    snort0: Vagrant insecure key detected. Vagrant will automatically replace\n",
      "    snort0: this with a newly generated keypair for better security.\n",
      "    snort0: \n",
      "    snort0: Inserting generated public key within guest...\n",
      "    snort0: Removing insecure key from the guest if it's present...\n",
      "    snort0: Key inserted! Disconnecting and reconnecting using new SSH key...\n",
      "==> snort0: Machine booted and ready!\n",
      "==> snort0: Checking for guest additions in VM...\n",
      "==> snort0: Configuring and enabling network interfaces...\n",
      "==> snort0: Automatic installation for Landrush IP not enabled\n",
      "==> snort0: Mounting shared folders...\n",
      "    snort0: /vagrant => /scratch/ubuntu/jupyter\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: \n",
      "==> snort0: ================================================================================\n",
      "==> snort0:  Package            Arch         Version   Repository                      Size\n",
      "==> snort0: ================================================================================\n",
      "==> snort0: Installing:\n",
      "==> snort0:  epel-release       noarch       7-9       /epel-release-7-9.noarch        24 k\n",
      "==> snort0: \n",
      "==> snort0: Transaction Summary\n",
      "==> snort0: ================================================================================\n",
      "==> snort0: Install  1 Package\n",
      "==> snort0: \n",
      "==> snort0: Total size: 24 k\n",
      "==> snort0: Installed size: 24 k\n",
      "==> snort0: \n",
      "==> snort0: Installed:\n",
      "==> snort0:   epel-release.noarch 0:7-9                                                     \n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Detected operating system as centos/7.\n",
      "==> snort0: Checking for curl...\n",
      "==> snort0: Detected curl...\n",
      "==> snort0: Downloading repository file: https://packagecloud.io/install/repositories/prometheus-rpm/release/config_file.repo?os=centos&dist=7&source=script\n",
      "==> snort0: done.\n",
      "==> snort0: Installing pygpgme to verify GPG signatures...\n",
      "==> snort0: Importing GPG key 0x7457CCD1:\n",
      "==> snort0:  Userid     : \"https://packagecloud.io/prometheus-rpm/centos (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>\"\n",
      "==> snort0:  Fingerprint: d7df c575 af4b 92c6 e62a d3c5 f721 6757 7457 ccd1\n",
      "==> snort0:  From       : https://packagecloud.io/prometheus-rpm/release/gpgkey\n",
      "==> snort0: Package pygpgme-0.3-9.el7.x86_64 already installed and latest version\n",
      "==> snort0: Installing yum-utils...\n",
      "==> snort0: Public key for libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm is not installed\n",
      "==> snort0: warning: /var/cache/yum/x86_64/7/base/packages/libxml2-python-2.9.1-6.el7_2.3.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY\n",
      "==> snort0: Importing GPG key 0xF4A80EB5:\n",
      "==> snort0:  Userid     : \"CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>\"\n",
      "==> snort0:  Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5\n",
      "==> snort0:  Package    : centos-release-7-3.1611.el7.centos.x86_64 (@anaconda)\n",
      "==> snort0:  From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7\n",
      "==> snort0: Generating yum cache for prometheus-rpm_release...\n",
      "==> snort0: Importing GPG key 0x7457CCD1:\n",
      "==> snort0:  Userid     : \"https://packagecloud.io/prometheus-rpm/centos (https://packagecloud.io/docs#gpg_signing) <support@packagecloud.io>\"\n",
      "==> snort0:  Fingerprint: d7df c575 af4b 92c6 e62a d3c5 f721 6757 7457 ccd1\n",
      "==> snort0:  From       : https://packagecloud.io/prometheus-rpm/release/gpgkey\n",
      "==> snort0: \n",
      "==> snort0: The repository is setup! You can now install packages.\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: net.ipv6.conf.default.disable_ipv6 = 1\n",
      "==> snort0: net.ipv6.conf.all.disable_ipv6 = 1\n",
      "==> snort0: net.ipv6.conf.lo.disable_ipv6 = 1\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: \n",
      "==> snort0: ================================================================================\n",
      "==> snort0:  Package             Arch       Version                       Repository   Size\n",
      "==> snort0: ================================================================================\n",
      "==> snort0: Installing:\n",
      "==> snort0:  ntp                 x86_64     4.2.6p5-25.el7.centos.2       updates     547 k\n",
      "==> snort0: Installing for dependencies:\n",
      "==> snort0:  autogen-libopts     x86_64     5.18-5.el7                    base         66 k\n",
      "==> snort0:  ntpdate             x86_64     4.2.6p5-25.el7.centos.2       updates      86 k\n",
      "==> snort0: \n",
      "==> snort0: Transaction Summary\n",
      "==> snort0: ================================================================================\n",
      "==> snort0: Install  1 Package (+2 Dependent packages)\n",
      "==> snort0: \n",
      "==> snort0: Total download size: 699 k\n",
      "==> snort0: Installed size: 1.6 M\n",
      "==> snort0: \n",
      "==> snort0: Installed:\n",
      "==> snort0:   ntp.x86_64 0:4.2.6p5-25.el7.centos.2                                          \n",
      "==> snort0: \n",
      "==> snort0: Dependency Installed:\n",
      "==> snort0:   autogen-libopts.x86_64 0:5.18-5.el7  ntpdate.x86_64 0:4.2.6p5-25.el7.centos.2 \n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n",
      "==> snort0: Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.\n",
      "==> snort0: Running provisioner: shell...\n",
      "    snort0: Running: inline script\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "While waiting look at https://github.com/marcindulak/vagrant-snort-tutorial-centos7/blob/master/Vagrantfile. Notice that only a base system installation is performed on the VMs. The actual configuration of the machines is performed below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Configure `prometheus` on **monitor0**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Detected operating system as centos/7.\n",
      "Checking for curl...\n",
      "Detected curl...\n",
      "Downloading repository file: https://packagecloud.io/install/repositories/prometheus-rpm/release/config_file.repo?os=centos&dist=7&source=script\n",
      "done.\n",
      "Installing pygpgme to verify GPG signatures...\n",
      "Package pygpgme-0.3-9.el7.x86_64 already installed and latest version\n",
      "Installing yum-utils...\n",
      "Package yum-utils-1.1.31-40.el7.noarch already installed and latest version\n",
      "Generating yum cache for prometheus-rpm_release...\n",
      "\n",
      "The repository is setup! You can now install packages.\n",
      "\n",
      "================================================================================\n",
      " Package       Arch      Version                Repository                 Size\n",
      "================================================================================\n",
      "Installing:\n",
      " prometheus    x86_64    1.6.1-1.el7.centos     prometheus-rpm_release     10 M\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 10 M\n",
      "Installed size: 49 M\n",
      "\n",
      "Installed:\n",
      "  prometheus.x86_64 0:1.6.1-1.el7.centos                                        \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Created symlink from /etc/systemd/system/multi-user.target.wants/prometheus.service to /usr/lib/systemd/system/prometheus.service.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'sh /vagrant/configure_prometheus_on_monitor.sh'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'install -m 644 /vagrant/node_exporter.json /etc/prometheus'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "This configures the `prometheus` server on **monitor0**.\n",
    "\n",
    "The `prometheus` server periodically pulls the metrics exposed by the nodes defined in `/etc/prometheus/node_exporter.json`.\n",
    "Node exporter (https://github.com/prometheus/node_exporter) allows one to expose an arbitrary metric from a machine running the `node_exporter` service, using the functionality of Textfile Collector."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Configure `snort` on **snort0**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'install -m 644 /vagrant/copr-marcindulak-snort.repo /etc/yum.repos.d'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package          Arch   Version                   Repository              Size\n",
      "================================================================================\n",
      "Installing:\n",
      " snort            x86_64 3.0.0-0.232.a4.el7.centos copr-marcindulak-snort 1.8 M\n",
      " snort-extra      x86_64 1.0.0-0.232.a4.el7.centos copr-marcindulak-snort  75 k\n",
      "Installing for dependencies:\n",
      " daq              x86_64 2.2.1-1.el7.centos        copr-marcindulak-snort  86 k\n",
      " daq-modules      x86_64 2.2.1-1.el7.centos        copr-marcindulak-snort  37 k\n",
      " hwloc-libs       x86_64 1.11.2-1.el7              base                   1.5 M\n",
      " libdnet          x86_64 1.12-13.1.el7             base                    31 k\n",
      " libnetfilter_queue\n",
      "                  x86_64 1.0.2-2.el7               epel                    23 k\n",
      " libtool-ltdl     x86_64 2.4.2-22.el7_3            updates                 49 k\n",
      " luajit           x86_64 2.0.4-3.el7               epel                   343 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  2 Packages (+7 Dependent packages)\n",
      "\n",
      "Total download size: 3.9 M\n",
      "Installed size: 10 M\n",
      "Public key for libnetfilter_queue-1.0.2-2.el7.x86_64.rpm is not installed\n",
      "Public key for daq-modules-2.2.1-1.el7.centos.x86_64.rpm is not installed\n",
      "\n",
      "Installed:\n",
      "  snort.x86_64 0:3.0.0-0.232.a4.el7.centos                                      \n",
      "  snort-extra.x86_64 0:1.0.0-0.232.a4.el7.centos                                \n",
      "\n",
      "Dependency Installed:\n",
      "  daq.x86_64 0:2.2.1-1.el7.centos                                               \n",
      "  daq-modules.x86_64 0:2.2.1-1.el7.centos                                       \n",
      "  hwloc-libs.x86_64 0:1.11.2-1.el7                                              \n",
      "  libdnet.x86_64 0:1.12-13.1.el7                                                \n",
      "  libnetfilter_queue.x86_64 0:1.0.2-2.el7                                       \n",
      "  libtool-ltdl.x86_64 0:2.4.2-22.el7_3                                          \n",
      "  luajit.x86_64 0:2.0.4-3.el7                                                   \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "warning: /var/cache/yum/x86_64/7/epel/packages/libnetfilter_queue-1.0.2-2.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY\n",
      "warning: /var/cache/yum/x86_64/7/copr-marcindulak-snort/packages/daq-modules-2.2.1-1.el7.centos.x86_64.rpm: Header V3 RSA/SHA1 Signature, key ID 37627d2f: NOKEY\n",
      "Importing GPG key 0x37627D2F:\n",
      " Userid     : \"marcindulak_snort (None) <marcindulak#snort@copr.fedorahosted.org>\"\n",
      " Fingerprint: 987e e485 5793 87ee 3581 3bfb a919 6b3b 3762 7d2f\n",
      " From       : https://copr-be.cloud.fedoraproject.org/results/marcindulak/snort/pubkey.gpg\n",
      "Importing GPG key 0x352C64E5:\n",
      " Userid     : \"Fedora EPEL (7) <epel@fedoraproject.org>\"\n",
      " Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5\n",
      " Package    : epel-release-7-9.noarch (installed)\n",
      " From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'yum -y -d 1 -e 0 install snort snort-extra'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "unknown\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "! vagrant ssh snort0 -c \"sudo su - -c 'systemctl is-active snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The exclamation mark above is used in order to assert that the command return with non-zero exit status.\n",
    "We want a non-zero status because `snort` service should not be active yet. Note that `jupyter` cell does not \"care\" about the exit status, but if you run the command in the terminal or as part of as shell script the exit status does matter. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Disable affixing of unified2.log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The instance of `snort` is configured to log alerts and payloads which triggered the alerts in the `unified2` format (https://www.snort.org/faq/readme-unified2).\n",
    "\n",
    "At every `snort` restart or when the maximum size of the log file is exceeded `snort` will create a `/var/log/snort/unified2.log.`date +%s` file, i.e. `/var/log/snort/unified2.log.` followed by the number of seconds elapsed since 1970-01-01 00:00:00 UTC (https://en.wikipedia.org/wiki/Unix_time).\n",
    "\n",
    "This is inconvenient for the purpose of demonstrations and therefore adjust the `snort` configuration to always write to `/var/log/snort/unified2.log"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"echo unified2 = { limit = 128, units = \\'M\\', nostamp = true, mpls_event_types = true, vlan_event_types = true } >> /etc/snort/snort.lua\\\"\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Workaround for http://seclists.org/snort/2017/q1/581"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i 's|/usr/lib64/snort_extra|/usr/lib64/snort_extra/codecs|' /usr/lib/systemd/system/snort@.service\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl daemon-reload'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Start snort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl start snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "active\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl is-active snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "\"active\" means `snort` is running now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "An important comment needs to be made about the `iptables` configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Generated by iptables-save v1.4.21 on Wed May  3 14:56:57 2017\n",
      "*filter\n",
      ":INPUT ACCEPT [51:7234]\n",
      ":FORWARD ACCEPT [0:0]\n",
      ":OUTPUT ACCEPT [37:6114]\n",
      "-A INPUT -i enp0s8 -j NFQUEUE --queue-num 0 --queue-bypass\n",
      "-A OUTPUT -o enp0s8 -j NFQUEUE --queue-num 0 --queue-bypass\n",
      "COMMIT\n",
      "# Completed on Wed May  3 14:56:57 2017\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'iptables-save'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "which is performed by the `snort` service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "● snort@enp0s8.service - Snort on 'enp0s8'\n",
      "   Loaded: loaded (/usr/lib/systemd/system/snort@.service; disabled; vendor preset: disabled)\n",
      "   Active: active (running) since Wed 2017-05-03 14:56:52 UTC; 7s ago\n",
      "  Process: 5653 ExecStartPre=/bin/sh -c /usr/sbin/iptables -t filter -C OUTPUT -o %I -j NFQUEUE --queue-num 0 --queue-bypass || /usr/sbin/iptables -t filter -I OUTPUT -o %I -j NFQUEUE --queue-num 0 --queue-bypass (code=exited, status=0/SUCCESS)\n",
      "  Process: 5643 ExecStartPre=/bin/sh -c /usr/sbin/iptables -t filter -C INPUT -i %I -j NFQUEUE --queue-num 0 --queue-bypass || /usr/sbin/iptables -t filter -I INPUT -i %I -j NFQUEUE --queue-num 0 --queue-bypass (code=exited, status=0/SUCCESS)\n",
      " Main PID: 5658 (snort)\n",
      "   CGroup: /system.slice/system-snort.slice/snort@enp0s8.service\n",
      "           └─5658 /usr/sbin/snort -d -Q --daq-dir /usr/lib64/daq --daq nfq -l /var/log/snort -c /etc/snort/snort.lua -A unified2 -v --plugin-path /usr/lib64/snort_extra/codecs\n",
      "\n",
      "May 03 14:56:52 snort0 snort[5658]: MaxClientBytes: 19600\n",
      "May 03 14:56:52 snort0 snort[5658]: Portscan Detection Config:\n",
      "May 03 14:56:52 snort0 snort[5658]: Detect Protocols:  TCP UDP ICMP IP\n",
      "May 03 14:56:52 snort0 snort[5658]: Detect Scan Type:  portscan portsweep decoy_portscan distributed_portscan\n",
      "May 03 14:56:52 snort0 snort[5658]: Sensitivity Level: Medium\n",
      "May 03 14:56:52 snort0 snort[5658]: Memcap (in bytes): 1048576\n",
      "May 03 14:56:52 snort0 snort[5658]: Number of Nodes:   2052\n",
      "May 03 14:56:52 snort0 snort[5658]: Binder\n",
      "May 03 14:56:52 snort0 snort[5658]: --------------------------------------------------\n",
      "May 03 14:56:52 snort0 snort[5658]: nfq DAQ configured to inline.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl status snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In this setup all the traffic reaching `snort0` is passed to `nfqueue` for `snort` inspection. It should be **stressed** that the traffic after being processed by the `nfqueue` target does to return back to `netfilter` - `iptables` won't process any further rules https://sourceforge.net/p/snort/mailman/message/29066595/.\n",
    "On a production system one would first filter the traffic using `iptables` rules and pass the remaining, \"good\" traffic to `nfqueue` for further `snort` inspection. This would however limit `snort`'s ability to detect events which involve multiple ports, like portscans."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Configure `node_exporter` for **snort0**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Detected operating system as centos/7.\n",
      "Checking for curl...\n",
      "Detected curl...\n",
      "Downloading repository file: https://packagecloud.io/install/repositories/prometheus-rpm/release/config_file.repo?os=centos&dist=7&source=script\n",
      "done.\n",
      "Installing pygpgme to verify GPG signatures...\n",
      "Package pygpgme-0.3-9.el7.x86_64 already installed and latest version\n",
      "Installing yum-utils...\n",
      "Package yum-utils-1.1.31-40.el7.noarch already installed and latest version\n",
      "Generating yum cache for prometheus-rpm_release...\n",
      "\n",
      "The repository is setup! You can now install packages.\n",
      "\n",
      "================================================================================\n",
      " Package         Arch     Version                Repository                Size\n",
      "================================================================================\n",
      "Installing:\n",
      " node_exporter   x86_64   0.14.0-1.el7.centos    prometheus-rpm_release   2.4 M\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 2.4 M\n",
      "Installed size: 8.5 M\n",
      "\n",
      "Installed:\n",
      "  node_exporter.x86_64 0:0.14.0-1.el7.centos                                    \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Created symlink from /etc/systemd/system/multi-user.target.wants/node_exporter.service to /usr/lib/systemd/system/node_exporter.service.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'sh /vagrant/configure_node_exporter_on_snort.sh'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl daemon-reload'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package                  Arch          Version               Repository   Size\n",
      "================================================================================\n",
      "Installing:\n",
      " python-dateutil          noarch        1.5-7.el7             base         85 k\n",
      " python2-idstools         noarch        0.6.0-1.el7           epel         89 k\n",
      " python34-idstools        noarch        0.6.0-1.el7           epel         91 k\n",
      "Installing for dependencies:\n",
      " python34                 x86_64        3.4.5-3.el7           epel         50 k\n",
      " python34-dateutil        noarch        1:2.4.2-3.el7         epel         82 k\n",
      " python34-libs            x86_64        3.4.5-3.el7           epel        6.7 M\n",
      " python34-six             noarch        1.10.0-1.el7          epel         32 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  3 Packages (+4 Dependent packages)\n",
      "\n",
      "Total download size: 7.1 M\n",
      "Installed size: 29 M\n",
      "\n",
      "Installed:\n",
      "  python-dateutil.noarch 0:1.5-7.el7      python2-idstools.noarch 0:0.6.0-1.el7 \n",
      "  python34-idstools.noarch 0:0.6.0-1.el7 \n",
      "\n",
      "Dependency Installed:\n",
      "  python34.x86_64 0:3.4.5-3.el7        python34-dateutil.noarch 1:2.4.2-3.el7  \n",
      "  python34-libs.x86_64 0:3.4.5-3.el7   python34-six.noarch 0:1.10.0-1.el7      \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install python*-idstools python-dateutil'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'install -m 755 /vagrant/snort-alert-count.py /usr/local/bin'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'install -m 644 /vagrant/snort-alert-count.service /usr/lib/systemd/system'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl daemon-reload'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Created symlink from /etc/systemd/system/multi-user.target.wants/snort-alert-count.service to /usr/lib/systemd/system/snort-alert-count.service.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl enable snort-alert-count'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl start snort-alert-count'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "active\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl is-active snort-alert-count'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In the context of `snort` this means: the `snort-alert-count.py` script continuously reads the `snort` alerts unified2 file generated by `snort` on **snort0** and writes the Textfile Collector file on **snort0** under `/var/lib/prometheus`.\n",
    "The `node_exporter` service on **snort0** reads this file and exposes the metrics to the `prometheus` server.\n",
    "  \n",
    "Anyone willing to correct `snort-alert-count.py` so it writes the Textfile Collector file at regular time intervals, **independently** whether new alerts are arriving? Contributions are welcome."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Snort usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Drop ICMP traffic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The classic example presented in most `snort` tutorials when explaining `snort` rules is to alert on `ping`. Our `snort` instance runs in IPS mode so we will be able to block 'pings'.\n",
    "\n",
    "Here is a great introduction to `snort` rules \"DevNet 1126 Detection Strategies with Snort\" https://www.youtube.com/watch?v=-aeZ6OD8BPg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Verify that `ping` works between `monitor0` and `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PING snort0 (192.168.17.30) from 192.168.17.20 enp0s8: 56(84) bytes of data.\n",
      "64 bytes from snort0 (192.168.17.30): icmp_seq=1 ttl=64 time=16.4 ms\n",
      "\n",
      "--- snort0 ping statistics ---\n",
      "1 packets transmitted, 1 received, 0% packet loss, time 0ms\n",
      "rtt min/avg/max/mdev = 16.491/16.491/16.491/0.000 ms\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'ping -c 1 -W 1 -I enp0s8 snort0'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Copy the default `sample.rules` distributed by `snort` and enable them in `/etc/snort/snort_defaults.lua`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'cp -p /etc/snort/sample.rules /etc/snort/rules/snort.rules'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"echo RULE_PATH = conf_dir .. \\'/rules\\' >> /etc/snort/snort_defaults.lua\\\"\"  # http://seclists.org/snort/2017/q1/524"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"echo ips = { include = RULE_PATH .. \\'/snort.rules\\' } >> /etc/snort/snort_defaults.lua\\\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Enable dropping of incoming ICMP traffic on `snort0` by appending the appropriate rule to `/etc/snort/rules/sample.rules`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "echo 'drop icmp any any -> 192.168.17.30 any (msg:\"icmp any any -> 192.168.17.30 any\"; sid:3000001; rev:001;)' > 3000001.rules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'cat /vagrant/3000001.rules >> /etc/snort/rules/snort.rules'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The `3000001` is the first, free for a custom use, Signature ID after the ET rules (https://rules.emergingthreats.net/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Make `snort` to re-read the rules without restarting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"kill -hup $(pidof snort)\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "active\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl is-active snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Verify that the ICMP traffic is now blocked by `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PING snort0 (192.168.17.30) from 192.168.17.20 enp0s8: 56(84) bytes of data.\n",
      "\n",
      "--- snort0 ping statistics ---\n",
      "1 packets transmitted, 0 received, 100% packet loss, time 0ms\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "! vagrant ssh monitor0 -c \"sudo su - -c 'ping -c 1 -W 1 -I enp0s8 snort0'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "If at this point the packet is **not** being dropped you may have another `snort0` or `monitor0` instance active somewhere from another `vagrant up`, or some other issue."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Enable dropping of outgoing ICMP traffic on `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "echo 'drop icmp 192.168.17.30 any -> any any (msg:\"icmp 192.168.17.30 any -> any any\"; sid:3000002; rev:001;)' > 3000002.rules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'cat /vagrant/3000002.rules >> /etc/snort/rules/snort.rules'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"kill -hup $(pidof snort)\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PING monitor0 (192.168.17.20) from 192.168.17.30 enp0s8: 56(84) bytes of data.\n",
      "\n",
      "--- monitor0 ping statistics ---\n",
      "1 packets transmitted, 0 received, 100% packet loss, time 0ms\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'ping -c 1 -W 1 -I enp0s8 monitor0'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Note that the IDS functionality with `nfqueue` requires `snort` to run as `root`. In order to run `snort` in IDS mode change the user from `root` to `snort` and\n",
    "remove the `-Q` option from `/usr/lib/systemd/system/snort@.service`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Display the alerts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Until now we should have two alerts showing the dropped incoming and outgoing ICMP packets. The details of the alert and the payload can be displayed using `u2spewfoo`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "(Event)\n",
      "\tsensor id: 0\tevent id: 1\tevent second: 1493823488\tevent microsecond: 849732\n",
      "\tsig id: 3000001\tgen id: 1\trevision: 1\t classification: 0\n",
      "\tpriority: 0\tip source: 192.168.17.20\tip destination: 192.168.17.30\n",
      "\tsrc port: 0\tdest port: 0\tip_proto: 1\timpact_flag: 0\tblocked: 0\n",
      "\tmpls label: 0\tvland id: 0\tpolicy id: 0\n",
      "\n",
      "Packet\n",
      "\tsensor id: 0\tevent id: 1\tevent second: 1493823488\n",
      "\tpacket second: 1493823488\tpacket microsecond: 849732\n",
      "\tlinktype: 228\tpacket_length: 84\n",
      "[    0] 45 00 00 54 18 C0 00 00 40 01 BE 66 C0 A8 11 14  E..T....@..f....\n",
      "[   16] C0 A8 11 1E 00 00 D2 2C 19 D5 00 01 00 F0 09 59  .......,.......Y\n",
      "[   32] 00 00 00 00 3E E1 0C 00 00 00 00 00 10 11 12 13  ....>...........\n",
      "[   48] 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23  ............ !\"#\n",
      "[   64] 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33  $%&'()*+,-./0123\n",
      "[   80] 34 35 36 37                                      4567\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'u2spewfoo /var/log/snort/unified2.log'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "An alternative method of listing the contents of the `unified2` log is performed using https://github.com/jasonish/py-idstools,\n",
    "which converts `unified2` to `json` or `eve` (https://redmine.openinfosecfoundation.org/projects/suricata/wiki/_Logstash_Kibana_and_Suricata_JSON_output)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"type\": \"event\", \"event\": {\"sensor-id\": 0, \"impact\": 0, \"destination-ip\": \"192.168.17.30\", \"priority\": 0, \"protocol\": 1, \"impact-flag\": 0, \"sport-itype\": 0, \"event-microsecond\": 849732, \"signature-revision\": 1, \"dport-icode\": 0, \"source-ip\": \"192.168.17.20\", \"classification-id\": 0, \"generator-id\": 1, \"vlan-id\": 0, \"pad2\": 0, \"mpls-label\": 0, \"event-id\": 1, \"blocked\": 0, \"signature-id\": 3000001, \"event-second\": 1493823488}}\n",
      "{\"type\": \"packet\", \"packet\": {\"sensor-id\": 0, \"data\": \"RQAAVBjAAABAAb5mwKgRFMCoER4AANIsGdUAAQDwCVkAAAAAPuEMAAAAAAAQERITFBUWFxgZGhscHR4fICEiIyQlJicoKSorLC0uLzAxMjM0NTY3\", \"event-id\": 1, \"packet-second\": 1493823488, \"event-second\": 1493823488, \"packet-microsecond\": 849732, \"length\": 84, \"linktype\": 228}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Note that despite the fact that the packet has been dropped `unified2` does not provide this information as can be seen from the `u2spewfoo` output. As a consequence `idstools-u2json` assumes the packet has not been dropped. It seems worth inquiring the `snort-users` mailing list https://lists.sourceforge.net/lists/listinfo/snort-users whether the `unified2` missing *blocked* information is a bug, ar maybe a missing feature."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "There is also tool `u2boat` for converting `unified2` into `pcap`, but does not seem to work as expected http://seclists.org/snort/2017/q1/11"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Defaulting to pcap output.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'u2boat /var/log/snort/unified2.log /tmp/u2boat.pcap'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package          Arch            Version                   Repository     Size\n",
      "================================================================================\n",
      "Installing:\n",
      " tcpdump          x86_64          14:4.5.1-3.el7            base          387 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 387 k\n",
      "Installed size: 931 k\n",
      "\n",
      "Installed:\n",
      "  tcpdump.x86_64 14:4.5.1-3.el7                                                 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install tcpdump'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14:58:08.849732 00:00:40:01:be:66 (oui Unknown) > 45:00:00:54:18:c0 (oui Unknown), ethertype Unknown (0xc0a8), length 84: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "reading from file /tmp/u2boat.pcap, link-type EN10MB (Ethernet)\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'tcpdump -XX -r /tmp/u2boat.pcap'\" | grep 'ethertype Unknown'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "See \"Introduction to Packet Analysis\" https://www.youtube.com/watch?v=visrNiKIP3E&index=17 for a refresher about network protocols and\n",
    "tools like `tcpdump` or `wireshark`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The `eve` format is used when pushing `snort` logs into ELK (https://www.elastic.co/products), or for converting `unified2` into `pcap`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"python2 $(which idstools-u2eve) /var/log/snort/unified2.log > /tmp/u2eve.log\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i 's/-0600/+0000/' /usr/lib/python2.7/site-packages/idstools/scripts/eve2pcap.py\\\"\"  # https://github.com/jasonish/py-idstools/issues/37"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"python2 $(which idstools-eve2pcap) /tmp/u2eve.log --dlt RAW -o /tmp/u2eve2pcap.pcap\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14:58:08.849732 IP monitor0 > snort0: ICMP echo reply, id 6613, seq 1, length 64\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "reading from file /tmp/u2eve2pcap.pcap, link-type RAW (Raw IP)\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'tcpdump -XX -r /tmp/u2eve2pcap.pcap'\" | grep 'ICMP'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Periodic test alert"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "A periodic test alert can be setup (http://ossectools.blogspot.dk/2011/04/network-intrusion-detection-systems.html) the purpose of which\n",
    "will be to probe whether `snort` or another resource on `snort0` is able to keep up with the traffic. If the rate of the test alert\n",
    "falls below a threshold we will know that some alerts were missed by `snort`. An HTTP request on port 80 will be used as a test alert."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### In order for the rule to triggered a web server must be listening on port 80 on `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package           Arch         Version                     Repository     Size\n",
      "================================================================================\n",
      "Installing:\n",
      " httpd             x86_64       2.4.6-45.el7.centos.4       updates       2.7 M\n",
      "Installing for dependencies:\n",
      " apr               x86_64       1.4.8-3.el7                 base          103 k\n",
      " apr-util          x86_64       1.5.2-6.el7                 base           92 k\n",
      " httpd-tools       x86_64       2.4.6-45.el7.centos.4       updates        84 k\n",
      " mailcap           noarch       2.1.41-2.el7                base           31 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package (+4 Dependent packages)\n",
      "\n",
      "Total download size: 3.0 M\n",
      "Installed size: 10 M\n",
      "\n",
      "Installed:\n",
      "  httpd.x86_64 0:2.4.6-45.el7.centos.4                                          \n",
      "\n",
      "Dependency Installed:\n",
      "  apr.x86_64 0:1.4.8-3.el7                      apr-util.x86_64 0:1.5.2-6.el7   \n",
      "  httpd-tools.x86_64 0:2.4.6-45.el7.centos.4    mailcap.noarch 0:2.1.41-2.el7   \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install httpd'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl enable httpd'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### The rule must be added to detect the test alert"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The proper rule would like alert tcp 192.168.17.20 any -> any 80 (msg:\"snort-test-alert\"; flow:to_server,established; http_uri; content:\"/snort-test-alert\"; metadata:service http; classtype:not-suspicious; sid:3000003; rev:001;) but due to http://seclists.org/snort/2017/q1/635 it is modified"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "echo 'alert tcp 192.168.17.20 any -> any 80 (msg:\"snort-test-alert\"; content:\"/snort-test-alert\"; classtype:not-suspicious; sid:3000003; rev:001;)' > 3000003.rules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'cat /vagrant/3000003.rules >> /etc/snort/rules/snort.rules'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"kill -hup $(pidof snort)\"'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The HTTP request is sent using `curl -s -m 1 http://snort0/snort-test-alert` from `monitor0`, let's save it's `pcap` on `snort0`. The `pcap` will be later used to\n",
    "test how much traffic `snort0` can handle by replaying the traffic with `tcpreplay`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In order to capture the HTTP traffic Apache must be listening"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl start httpd'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package       Arch              Version                  Repository       Size\n",
      "================================================================================\n",
      "Installing:\n",
      " at            x86_64            3.1.13-22.el7            base             51 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 51 k\n",
      "Installed size: 95 k\n",
      "\n",
      "Installed:\n",
      "  at.x86_64 0:3.1.13-22.el7                                                     \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install at'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl start atd'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "job 1 at Wed May  3 14:59:00 2017\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"echo 'curl -s -m 1 http://snort0/snort-test-alert' | at now + 1 minute\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tcpdump: listening on enp0s8, link-type EN10MB (Ethernet), capture size 65535 bytes\n",
      "0 packets captured\n",
      "0 packets received by filter\n",
      "0 packets dropped by kernel\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'timeout 70 tcpdump -i enp0s8 port 80 -w /vagrant/snort-test-alert.pcap'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We expect 10 packets to be captured. Verify that the test alert has been detected by `snort`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"type\": \"event\", \"event\": {\"protocol\": 6, \"signature-revision\": 1, \"vlan-id\": 0, \"impact\": 0, \"generator-id\": 1, \"source-ip\": \"192.168.17.20\", \"pad2\": 0, \"priority\": 3, \"sport-itype\": 59996, \"classification-id\": 1, \"signature-id\": 3000003, \"destination-ip\": \"192.168.17.30\", \"event-microsecond\": 268317, \"sensor-id\": 0, \"dport-icode\": 80, \"mpls-label\": 0, \"blocked\": 0, \"impact-flag\": 0, \"event-id\": 2, \"event-second\": 1493823540}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep '\"signature-id\": 3000003'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's stop Apache for the purpose of the demonstrations in the next section"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl stop httpd'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "and restart `snort` to clear the `unified2` log."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"systemctl restart snort@enp0s8\"'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### After having verified the test alert is being detected by `snort` enable the `snort-test-alert` service"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'install -m 644 /vagrant/snort-test-alert.service /usr/lib/systemd/system'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl daemon-reload'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Created symlink from /etc/systemd/system/multi-user.target.wants/snort-test-alert.service to /usr/lib/systemd/system/snort-test-alert.service.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl enable snort-test-alert'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl start snort-test-alert'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "active\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl is-active snort-test-alert'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The test alert should keep appearing in the `tcpdump` output on `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15:00:38.484367 IP monitor0.60014 > snort0.http: Flags [S], seq 2930856475, win 29200, options [mss 1460,sackOK,TS val 158258 ecr 0,nop,wscale 6], length 0\n",
      "15:00:38.484850 IP snort0.http > monitor0.60014: Flags [R.], seq 0, ack 2930856476, win 0, length 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tcpdump: verbose output suppressed, use -v or -vv for full protocol decode\n",
      "listening on enp0s8, link-type EN10MB (Ethernet), capture size 65535 bytes\n",
      "2 packets captured\n",
      "2 packets received by filter\n",
      "0 packets dropped by kernel\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'timeout 10 tcpdump -i enp0s8 port 80'\" | grep snort0.http"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "but it is not yet detectable by `snort` (because `httpd` is not running, only `tcp` traffic is exchanged)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "! vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep '\"signature-id\": 3000003'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### After Apache is started the test alerts will be detected by `snort`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl start httpd&& sleep 10'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "(Event)\n",
      "\tsensor id: 0\tevent id: 1\tevent second: 1493823648\tevent microsecond: 494777\n",
      "\tsig id: 3000003\tgen id: 1\trevision: 1\t classification: 1\n",
      "\tpriority: 3\tip source: 192.168.17.20\tip destination: 192.168.17.30\n",
      "\tsrc port: 60016\tdest port: 80\tip_proto: 6\timpact_flag: 0\tblocked: 0\n",
      "\tmpls label: 0\tvland id: 0\tpolicy id: 0\n",
      "\n",
      "Packet\n",
      "\tsensor id: 0\tevent id: 1\tevent second: 1493823648\n",
      "\tpacket second: 1493823648\tpacket microsecond: 494777\n",
      "\tlinktype: 228\tpacket_length: 138\n",
      "[    0] 45 00 00 8A 9E 89 40 00 40 06 F8 61 C0 A8 11 14  E.....@.@..a....\n",
      "[   16] C0 A8 11 1E EA 70 00 50 A8 9C 0F CB 00 60 02 2C  .....p.P.....`.,\n",
      "[   32] 80 18 01 C9 49 CC 00 00 01 01 08 0A 00 02 91 4D  ....I..........M\n",
      "[   48] 00 00 F6 A7 47 45 54 20 2F 73 6E 6F 72 74 2D 74  ....GET /snort-t\n",
      "[   64] 65 73 74 2D 61 6C 65 72 74 20 48 54 54 50 2F 31  est-alert HTTP/1\n",
      "[   80] 2E 31 0D 0A 55 73 65 72 2D 41 67 65 6E 74 3A 20  .1..User-Agent: \n",
      "[   96] 63 75 72 6C 2F 37 2E 32 39 2E 30 0D 0A 48 6F 73  curl/7.29.0..Hos\n",
      "[  112] 74 3A 20 73 6E 6F 72 74 30 0D 0A 41 63 63 65 70  t: snort0..Accep\n",
      "[  128] 74 3A 20 2A 2F 2A 0D 0A 0D 0A                    t: */*....\n",
      "\n",
      "(Event)\n",
      "\tsensor id: 0\tevent id: 2\tevent second: 1493823658\tevent microsecond: 505243\n",
      "\tsig id: 3000003\tgen id: 1\trevision: 1\t classification: 1\n",
      "\tpriority: 3\tip source: 192.168.17.20\tip destination: 192.168.17.30\n",
      "\tsrc port: 60020\tdest port: 80\tip_proto: 6\timpact_flag: 0\tblocked: 0\n",
      "\tmpls label: 0\tvland id: 0\tpolicy id: 0\n",
      "\n",
      "Packet\n",
      "\tsensor id: 0\tevent id: 2\tevent second: 1493823658\n",
      "\tpacket second: 1493823658\tpacket microsecond: 505243\n",
      "\tlinktype: 228\tpacket_length: 138\n",
      "[    0] 45 00 00 8A 7E F6 40 00 40 06 17 F5 C0 A8 11 14  E...~.@.@.......\n",
      "[   16] C0 A8 11 1E EA 74 00 50 EB 89 C6 94 28 4D 10 E2  .....t.P....(M..\n",
      "[   32] 80 18 01 C9 CB 37 00 00 01 01 08 0A 00 02 B8 68  .....7.........h\n",
      "[   48] 00 01 1D C2 47 45 54 20 2F 73 6E 6F 72 74 2D 74  ....GET /snort-t\n",
      "[   64] 65 73 74 2D 61 6C 65 72 74 20 48 54 54 50 2F 31  est-alert HTTP/1\n",
      "[   80] 2E 31 0D 0A 55 73 65 72 2D 41 67 65 6E 74 3A 20  .1..User-Agent: \n",
      "[   96] 63 75 72 6C 2F 37 2E 32 39 2E 30 0D 0A 48 6F 73  curl/7.29.0..Hos\n",
      "[  112] 74 3A 20 73 6E 6F 72 74 30 0D 0A 41 63 63 65 70  t: snort0..Accep\n",
      "[  128] 74 3A 20 2A 2F 2A 0D 0A 0D 0A                    t: */*....\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'u2spewfoo /var/log/snort/unified2.log'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"type\": \"event\", \"event\": {\"priority\": 3, \"dport-icode\": 80, \"protocol\": 6, \"vlan-id\": 0, \"signature-id\": 3000003, \"event-microsecond\": 494777, \"generator-id\": 1, \"signature-revision\": 1, \"sport-itype\": 60016, \"source-ip\": \"192.168.17.20\", \"blocked\": 0, \"classification-id\": 1, \"pad2\": 0, \"event-id\": 1, \"destination-ip\": \"192.168.17.30\", \"event-second\": 1493823648, \"sensor-id\": 0, \"impact\": 0, \"impact-flag\": 0, \"mpls-label\": 0}}\n",
      "{\"type\": \"event\", \"event\": {\"priority\": 3, \"dport-icode\": 80, \"protocol\": 6, \"vlan-id\": 0, \"signature-id\": 3000003, \"event-microsecond\": 505243, \"generator-id\": 1, \"signature-revision\": 1, \"sport-itype\": 60020, \"source-ip\": \"192.168.17.20\", \"blocked\": 0, \"classification-id\": 1, \"pad2\": 0, \"event-id\": 2, \"destination-ip\": \"192.168.17.30\", \"event-second\": 1493823658, \"sensor-id\": 0, \"impact\": 0, \"impact-flag\": 0, \"mpls-label\": 0}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep '\"signature-id\": 3000003'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Snort alerts in Prometheus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Prometheus is not best suited for storing and analyzing `snort` alerts but the simplicity of `prometheus` configuration fits well into this tutorial.\n",
    "For an introduction to `prometheus` see \"An introduction to monitoring and alerting with time-series at scale, with Prometheus\" https://www.youtube.com/watch?v=gNmWzkGViAY\n",
    "\n",
    "The `snort-alert-count.py` script on `snort0` continuously reads the output produced by `idstools-u2json /var/log/snort/unified2.log` and\n",
    "periodically writes the `snort` alerts counter into the `/var/lib/prometheus/snort.prom`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# HELP snort_alert_count Snort alert count.\n",
      "# TYPE snort_alert_count counter\n",
      "snort_alert_count{generator_id=\"1\",signature_id=\"3000001\",blocked=\"0\",source_ip=\"192.168.17.20\",dport_icode=\"0\"}  1\n",
      "snort_alert_count{generator_id=\"1\",signature_id=\"3000003\",blocked=\"0\",source_ip=\"192.168.17.20\",dport_icode=\"80\"}  1\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'grep snort_alert_count /var/lib/prometheus/snort.prom'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "After several minutes `snort` alerts will start appearing in `prometheus`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "sleep 300"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"status\":\"success\",\"data\":{\"resultType\":\"vector\",\"result\":[{\"metric\":{\"blocked\":\"0\",\"dport_icode\":\"80\",\"generator_id\":\"1\",\"instance\":\"snort0:9100\",\"job\":\"node\",\"signature_id\":\"3000003\",\"source_ip\":\"192.168.17.20\"},\"value\":[1493823970.28,\"5.522765925925926\"]}]}}"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"curl -sgG --data-urlencode query='rate(snort_alert_count[5m]) * 60' localhost:9090/api/v1/query\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The `prometheus` query 'rate(snort_alert_count[5m]) * 60' returns the average rate of increase of `snort` alerts (per alert) per minute, during the last 5 minutes\n",
    "(https://prometheus.io/docs/querying/functions/#rate()). The `snort-test-alert` on `monitor0` performs `curl` against `snort0` every 10 seconds\n",
    "so such defined rate of the test alert should be equal to 6 per minute."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Prometheus has it's own alerting system, but one can simply query the HTTP API of `prometheus` and include such checks into traditional\n",
    "monitoring systems like `nagios`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package                Arch         Version                   Repository  Size\n",
      "================================================================================\n",
      "Installing:\n",
      " git                    x86_64       1.8.3.1-6.el7_2.1         base       4.4 M\n",
      " jq                     x86_64       1.5-1.el7                 epel       153 k\n",
      "Installing for dependencies:\n",
      " libgnome-keyring       x86_64       3.8.0-3.el7               base       109 k\n",
      " oniguruma              x86_64       5.9.5-3.el7               epel       129 k\n",
      " perl-Error             noarch       1:0.17020-2.el7           base        32 k\n",
      " perl-Git               noarch       1.8.3.1-6.el7_2.1         base        53 k\n",
      " perl-TermReadKey       x86_64       2.30-20.el7               base        31 k\n",
      " rsync                  x86_64       3.0.9-17.el7              base       360 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  2 Packages (+6 Dependent packages)\n",
      "\n",
      "Total download size: 5.2 M\n",
      "Installed size: 24 M\n",
      "Public key for jq-1.5-1.el7.x86_64.rpm is not installed\n",
      "\n",
      "Installed:\n",
      "  git.x86_64 0:1.8.3.1-6.el7_2.1              jq.x86_64 0:1.5-1.el7             \n",
      "\n",
      "Dependency Installed:\n",
      "  libgnome-keyring.x86_64 0:3.8.0-3.el7   oniguruma.x86_64 0:5.9.5-3.el7       \n",
      "  perl-Error.noarch 1:0.17020-2.el7       perl-Git.noarch 0:1.8.3.1-6.el7_2.1  \n",
      "  perl-TermReadKey.x86_64 0:2.30-20.el7   rsync.x86_64 0:3.0.9-17.el7          \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "warning: /var/cache/yum/x86_64/7/epel/packages/jq-1.5-1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY\n",
      "Importing GPG key 0x352C64E5:\n",
      " Userid     : \"Fedora EPEL (7) <epel@fedoraproject.org>\"\n",
      " Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5\n",
      " Package    : epel-release-7-9.noarch (installed)\n",
      " From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install git jq'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cloning into 'nagios_plugins'...\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"git clone https://github.com/prometheus/nagios_plugins\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK - snort-test-alert is 6: { blocked: 0, dport_icode: 80, generator_id: 1, instance: snort0:9100, job: node, signature_id: 3000003, source_ip: 192.168.17.20 }\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"bash ./nagios_plugins/check_prometheus_metric.sh -H localhost:9090 -q \\\"rate(snort_alert_count{signature_id='3000003'}[5m]) * 60\\\" -w 5 -c 4 -n snort-test-alert -m lt -t vector -i\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "This query will return the \"WARNING\" when the rate of the test alert drops below 5 and \"CRITICAL\" when it drops below 4."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Over the years several domain-specific languages (DSL) have been proposed for making the alerts produced by IDS/IPS systems more manageable.\n",
    "One can use `prometheus` as a poor man's replacement of such a system. For example, it is interesting to know whether there are any\n",
    "persistent attackers on the network, who access the system over an extended period of time using various or the same types of attacks.\n",
    "\n",
    "The query below will identify the source IP numbers which triggered an alert during the last 5 minutes, and were also triggering some alerts\n",
    "(not necessarily the same) during the preceding hour:\n",
    "\n",
    "((sum without (signature-id,generator-id,dport-icode)(rate(snort_alert_count{alert_id!=\"1:3000003\"}[5m]))) > 0 and (sum without (signature-id,generator-id,dport-icode)(rate(snort_alert_count{alert_id!=\"1:3000003\"}[1h] offset 5m))) > 0) * 60\n",
    "\n",
    "Another query will provide a complementary information about the types of alerts, which tiggered during the last 5 minutes, and were\n",
    "also triggered during the preceding hour, independently of the source IP number:\n",
    "\n",
    "((sum without (source-ip,dport-icode)(rate(snort_alert_count{alert_id!=\"1:3000003\"}[5m]))) > 0 and (sum without (source-ip,dport-icode)(rate(snort_alert_count{alert_id!=\"1:3000003\"}[1h] offset 5m))) > 0) * 60\n",
    "\n",
    "See \"PromCon 2016: Alerting in the Prometheus Universe - Fabian Reinartz\" https://www.youtube.com/watch?v=yrK6z3fpu1E for more details about alerting in `prometheus`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Manage rules with Pulledpork"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Pulledpork [https://github.com/shirkdog/pulledpork](https://github.com/shirkdog/pulledpork) writes the selected rules into `/etc/snort/rules/snort.rules`.\n",
    "\n",
    "Unfortunately certain rules options have been modified/removed from `snort++` (see https://github.com/snortadmin/snort3/blob/master/doc/differences.txt),\n",
    "which means that most `snort2` rules are unusable. Just skip this section. The instructions in this section should work with `snort2` however.\n",
    "\n",
    "`snortrules-snapshot-*.tar.gz` are available for free after registration at https://www.snort.org/downloads, and are divided into `categories`.\n",
    "On the oter hand the `community-rules.tar.gz` available without registration are not divided into `categories`, but still use `categories` labels in the `msg` field."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Pulledpork works in a slightly complicated way when `categories` are involved (I hope I got it right, if not corrections are welcome).\n",
    "The behavior is dependent on the setting of the `order` variable in `/etc/pulledpork/pulledpork.conf`, and for `order=disable,enable,drop`:\n",
    "\n",
    "- all rules that are uncommented in the files distributed within a rules tarball are written into `/etc/snort/rules/snort.rules`\n",
    "\n",
    "- a category listed in `/etc/pulledpork/enablesid.conf` will enable all rules from the category (also the commented)\n",
    "\n",
    "- a category listed in `/etc/pulledpork/disablesid.conf` will disable all rules from the category (also the commented)\n",
    "\n",
    "- a category listed in `/etc/pulledpork/dropsid.conf` will convert uncommented rules from the category to drop, and also the commented rules if the category is in `/etc/pulledpork/enablesid.conf`\n",
    "\n",
    "- `/etc/pulledpork/modifysid.conf` can be used to modify the rules, like replacing ports or other sequences of characters in the rule\n",
    "\n",
    "The list of categories and how to create the list manually is here\n",
    "[https://github.com/shirkdog/pulledpork/blob/master/doc/README.CATEGORIES](https://github.com/shirkdog/pulledpork/blob/master/doc/README.CATEGORIES)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's use `community-rules.tar.gz` using `pulledpork`, enabling few categories, disabled (commented) by default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package                     Arch       Version                  Repository\n",
      "                                                                           Size\n",
      "================================================================================\n",
      "Installing:\n",
      " pulledpork                  noarch     0.7.2-2.el7              epel      44 k\n",
      "Installing for dependencies:\n",
      " perl-Archive-Tar            noarch     1.92-2.el7               base      73 k\n",
      " perl-Business-ISBN          noarch     2.06-2.el7               base      25 k\n",
      " perl-Business-ISBN-Data     noarch     20120719.001-2.el7       base      24 k\n",
      " perl-Compress-Raw-Bzip2     x86_64     2.061-3.el7              base      32 k\n",
      " perl-Compress-Raw-Zlib      x86_64     1:2.061-4.el7            base      57 k\n",
      " perl-Crypt-SSLeay           x86_64     0.64-5.el7               base      58 k\n",
      " perl-Data-Dumper            x86_64     2.145-3.el7              base      47 k\n",
      " perl-Digest                 noarch     1.17-245.el7             base      23 k\n",
      " perl-Digest-MD5             x86_64     2.52-3.el7               base      30 k\n",
      " perl-Encode-Locale          noarch     1.03-5.el7               base      16 k\n",
      " perl-File-Listing           noarch     6.04-7.el7               base      13 k\n",
      " perl-HTML-Parser            x86_64     3.71-4.el7               base     115 k\n",
      " perl-HTML-Tagset            noarch     3.20-15.el7              base      18 k\n",
      " perl-HTTP-Cookies           noarch     6.01-5.el7               base      26 k\n",
      " perl-HTTP-Daemon            noarch     6.01-5.el7               base      20 k\n",
      " perl-HTTP-Date              noarch     6.02-8.el7               base      14 k\n",
      " perl-HTTP-Message           noarch     6.06-6.el7               base      82 k\n",
      " perl-HTTP-Negotiate         noarch     6.01-5.el7               base      17 k\n",
      " perl-IO-Compress            noarch     2.061-2.el7              base     260 k\n",
      " perl-IO-HTML                noarch     1.00-2.el7               base      23 k\n",
      " perl-IO-Socket-IP           noarch     0.21-4.el7               base      35 k\n",
      " perl-IO-Socket-SSL          noarch     1.94-5.el7               base     114 k\n",
      " perl-IO-Zlib                noarch     1:1.10-291.el7           base      51 k\n",
      " perl-LWP-MediaTypes         noarch     6.02-2.el7               base      24 k\n",
      " perl-LWP-Protocol-https     noarch     6.04-4.el7               base      11 k\n",
      " perl-Mozilla-CA             noarch     20130114-5.el7           base      11 k\n",
      " perl-Net-HTTP               noarch     6.06-2.el7               base      29 k\n",
      " perl-Net-LibIDN             x86_64     0.12-15.el7              base      28 k\n",
      " perl-Net-SSLeay             x86_64     1.55-4.el7               base     285 k\n",
      " perl-Package-Constants      noarch     1:0.02-291.el7           base      45 k\n",
      " perl-Sys-Syslog             x86_64     0.33-3.el7               base      42 k\n",
      " perl-TimeDate               noarch     1:2.30-2.el7             base      52 k\n",
      " perl-URI                    noarch     1.60-9.el7               base     106 k\n",
      " perl-WWW-RobotRules         noarch     6.02-5.el7               base      18 k\n",
      " perl-libwww-perl            noarch     6.05-2.el7               base     205 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package (+35 Dependent packages)\n",
      "\n",
      "Total download size: 2.0 M\n",
      "Installed size: 4.7 M\n",
      "\n",
      "Installed:\n",
      "  pulledpork.noarch 0:0.7.2-2.el7                                               \n",
      "\n",
      "Dependency Installed:\n",
      "  perl-Archive-Tar.noarch 0:1.92-2.el7                                          \n",
      "  perl-Business-ISBN.noarch 0:2.06-2.el7                                        \n",
      "  perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7                           \n",
      "  perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7                                  \n",
      "  perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7                                   \n",
      "  perl-Crypt-SSLeay.x86_64 0:0.64-5.el7                                         \n",
      "  perl-Data-Dumper.x86_64 0:2.145-3.el7                                         \n",
      "  perl-Digest.noarch 0:1.17-245.el7                                             \n",
      "  perl-Digest-MD5.x86_64 0:2.52-3.el7                                           \n",
      "  perl-Encode-Locale.noarch 0:1.03-5.el7                                        \n",
      "  perl-File-Listing.noarch 0:6.04-7.el7                                         \n",
      "  perl-HTML-Parser.x86_64 0:3.71-4.el7                                          \n",
      "  perl-HTML-Tagset.noarch 0:3.20-15.el7                                         \n",
      "  perl-HTTP-Cookies.noarch 0:6.01-5.el7                                         \n",
      "  perl-HTTP-Daemon.noarch 0:6.01-5.el7                                          \n",
      "  perl-HTTP-Date.noarch 0:6.02-8.el7                                            \n",
      "  perl-HTTP-Message.noarch 0:6.06-6.el7                                         \n",
      "  perl-HTTP-Negotiate.noarch 0:6.01-5.el7                                       \n",
      "  perl-IO-Compress.noarch 0:2.061-2.el7                                         \n",
      "  perl-IO-HTML.noarch 0:1.00-2.el7                                              \n",
      "  perl-IO-Socket-IP.noarch 0:0.21-4.el7                                         \n",
      "  perl-IO-Socket-SSL.noarch 0:1.94-5.el7                                        \n",
      "  perl-IO-Zlib.noarch 1:1.10-291.el7                                            \n",
      "  perl-LWP-MediaTypes.noarch 0:6.02-2.el7                                       \n",
      "  perl-LWP-Protocol-https.noarch 0:6.04-4.el7                                   \n",
      "  perl-Mozilla-CA.noarch 0:20130114-5.el7                                       \n",
      "  perl-Net-HTTP.noarch 0:6.06-2.el7                                             \n",
      "  perl-Net-LibIDN.x86_64 0:0.12-15.el7                                          \n",
      "  perl-Net-SSLeay.x86_64 0:1.55-4.el7                                           \n",
      "  perl-Package-Constants.noarch 1:0.02-291.el7                                  \n",
      "  perl-Sys-Syslog.x86_64 0:0.33-3.el7                                           \n",
      "  perl-TimeDate.noarch 1:2.30-2.el7                                             \n",
      "  perl-URI.noarch 0:1.60-9.el7                                                  \n",
      "  perl-WWW-RobotRules.noarch 0:6.02-5.el7                                       \n",
      "  perl-libwww-perl.noarch 0:6.05-2.el7                                          \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install pulledpork'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i '/IPBLACKLIST/d' /etc/pulledpork/pulledpork.conf\\\"\"  # we don't use IP blacklisting feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'echo pcre:INDICATOR-SCAN >> /etc/pulledpork/enablesid.conf'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'echo pcre:SERVER-APACHE >> /etc/pulledpork/enablesid.conf'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'echo pcre:SERVER-WEBAPP >> /etc/pulledpork/enablesid.conf'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Fetch and install the selected rules, under `/tmp/snort.rules` - on our setup we don't want to overwrite `/etc/snort/rules/snort.rules`!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " \n",
      "    https://github.com/shirkdog/pulledpork\n",
      "      _____ ____\n",
      "     `----,\\    )\n",
      "      `--==\\\\  /    PulledPork v0.7.2 - E.Coli in your water bottle!\n",
      "       `--==\\\\/\n",
      "     .-~~~~-.Y|\\\\_  Copyright (C) 2009-2016 JJ Cummings\n",
      "  @_/        /  66\\_  cummingsj@gmail.com\n",
      "    |    \\   \\   _(\")\n",
      "     \\   /-| ||'--'  Rules give me wings!\n",
      "      \\_\\  \\_\\\\\n",
      " ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n",
      "\n",
      "Checking latest MD5 for community-rules.tar.gz....\n",
      "Rules tarball download of community-rules.tar.gz....\n",
      "\tThey Match\n",
      "\tDone!\n",
      "Checking latest MD5 for opensource.tar.gz....\n",
      "Rules tarball download of opensource.tar.gz....\n",
      "\tThey Match\n",
      "\tDone!\n",
      "Prepping rules from community-rules.tar.gz for work....\n",
      "\tDone!\n",
      "Prepping rules from opensource.tar.gz for work....\n",
      "\tDone!\n",
      "Reading rules...\n",
      "Modifying Sids....\n",
      "\tDone!\n",
      "Processing /etc/pulledpork/enablesid.conf....\n",
      "\tModified 831 rules\n",
      "\tSkipped 831 rules (already disabled)\n",
      "\tDone\n",
      "Processing /etc/pulledpork/dropsid.conf....\n",
      "\tModified 0 rules\n",
      "\tSkipped 0 rules (already disabled)\n",
      "\tDone\n",
      "Processing /etc/pulledpork/disablesid.conf....\n",
      "\tModified 0 rules\n",
      "\tSkipped 0 rules (already disabled)\n",
      "\tDone\n",
      "Setting Flowbit State....\n",
      "\tDone\n",
      "Writing /tmp/snort.rules....\n",
      "\tDone\n",
      "Generating sid-msg.map....\n",
      "\tDone\n",
      "Writing v1 /etc/snort/sid-msg.map....\n",
      "\tDone\n",
      "Writing /var/log/sid_changes.log....\n",
      "\tDone\n",
      "Rule Stats...\n",
      "\tNew:-------1700\n",
      "\tDeleted:---0\n",
      "\tEnabled Rules:----1700\n",
      "\tDropped Rules:----0\n",
      "\tDisabled Rules:---0\n",
      "\tTotal Rules:------1700\n",
      "No IP Blacklist Changes\n",
      "\n",
      "Done\n",
      "Please review /var/log/sid_changes.log for additional details\n",
      "Fly Piggy Fly!\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'pulledpork -PE -c /etc/pulledpork/pulledpork.conf -o /tmp/snort.rules'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Examples of actions from the attacker machine"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "This subject is virtually unlimited, but a representative set of actions detectable by `snort` is available at https://www.aldeid.com/wiki/Suricata-vs-snort"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Portscans"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The \"SYN\" scan consists of finding out whether a TCP port is open by sending \"SYN\".\n",
    "The port is considered open when \"SYN-ACK\" or \"SYN\" is received https://nmap.org/book/man-port-scanning-techniques.html\n",
    "The rule detecting \"SYN\" (PSNG_TCP_PORTSCAN) belongs to the portscan preprocessor and is included in `snortrules-snapshot-*.tar.gz`, available for free after registration. However, at https://www.snort.org/faq/readme-sfportscan we find that the \"TCP Portscan\" corresponds to gid 122, sid 1, so we write our own version of this rule"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "echo 'alert ( msg: \"PSNG_TCP_PORTSCAN\"; sid: 1; gid: 122; rev: 1; )' > 122:1.rules"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'cat /vagrant/122:1.rules >> /etc/snort/rules/snort.rules'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In order to detect this type of portscan performed by a single host the `sense_level` needs to be set to `high`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i '/port_scan/d' /etc/snort/snort.lua\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"echo port_scan = { sense_level = \\'high\\' } >> /etc/snort/snort.lua\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'sudo su - -c \"kill -hup $(pidof snort)\"'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Perform the scan from `attacker0`, saving the `pcap` file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package            Arch            Version                 Repository     Size\n",
      "================================================================================\n",
      "Installing:\n",
      " nmap               x86_64          2:6.40-7.el7            base          4.0 M\n",
      "Installing for dependencies:\n",
      " nmap-ncat          x86_64          2:6.40-7.el7            base          201 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package (+1 Dependent package)\n",
      "\n",
      "Total download size: 4.2 M\n",
      "Installed size: 17 M\n",
      "\n",
      "Installed:\n",
      "  nmap.x86_64 2:6.40-7.el7                                                      \n",
      "\n",
      "Dependency Installed:\n",
      "  nmap-ncat.x86_64 2:6.40-7.el7                                                 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"yum -d 1 -e 0 -y install nmap\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package       Arch              Version                  Repository       Size\n",
      "================================================================================\n",
      "Installing:\n",
      " at            x86_64            3.1.13-22.el7            base             51 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 51 k\n",
      "Installed size: 95 k\n",
      "\n",
      "Installed:\n",
      "  at.x86_64 0:3.1.13-22.el7                                                     \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install at'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c \"sudo su - -c 'systemctl start atd'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "job 1 at Wed May  3 15:08:00 2017\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c \"sudo su -c \\\"echo 'nmap -sS snort0' | at now + 1 minute\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "tcpdump: listening on enp0s8, link-type EN10MB (Ethernet), capture size 65535 bytes\n",
      "1518 packets captured\n",
      "2004 packets received by filter\n",
      "486 packets dropped by kernel\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'timeout 70 tcpdump -i enp0s8 tcp and host attacker0 -w /vagrant/portscan.pcap'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "`snort` is able to detect the \"SYN\" scan in the captured `pcap` file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "05/03-15:08:00.254890 [**] [122:1:1] \"PSNG_TCP_PORTSCAN\" [**] [Priority: 0] {IP} 192.168.17.10 -> 192.168.17.30\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'SNORT_LUA_PATH=/etc/snort LUA_PATH=/usr/include/snort/lua/?.lua snort --daq-dir /usr/lib64/daq -c /etc/snort/snort.lua --plugin-path /usr/lib64/snort_extra -R /etc/snort/rules/snort.rules -r /vagrant/portscan.pcap -A alert_fast -q'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"type\": \"event\", \"event\": {\"priority\": 0, \"impact-flag\": 0, \"event-second\": 1493824080, \"signature-id\": 1, \"event-id\": 44, \"pad2\": 0, \"generator-id\": 122, \"signature-revision\": 1, \"dport-icode\": 0, \"blocked\": 0, \"event-microsecond\": 254824, \"vlan-id\": 0, \"source-ip\": \"192.168.17.10\", \"destination-ip\": \"192.168.17.30\", \"sensor-id\": 0, \"protocol\": 6, \"mpls-label\": 0, \"classification-id\": 0, \"sport-itype\": 0, \"impact\": 0}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep -v packet | grep '\"signature-id\": 1' | grep '\"generator-id\": 122' "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Tcpreplay"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Here `tcpreplay` is used from the `attacker0` machine, this is actually not an attack.\n",
    "The reason for using `attacker0` is the amount of RAM needed for caching of packets in `tcpeplay`, but also to demonstrate the purpose of `tcprewrite`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We will be pretending to be `monitor0` from `attacker0`. In order to demonstrate this let's disable the `snort-test-alert` on `monitor0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl stop snort-test-alert'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "and get the total number of tiggered test alerts until now. All of them were triggered by the traffic from `monitor0` to `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "51\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep -v packet | grep '\"signature-id\": 3000003' | grep '192.168.17.30' | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's replay the test alert 100000 times aiming at 200 Mbps (which may or may not be reached, depending on the speed of you RAM) from the original `pcap`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package            Arch            Version                 Repository     Size\n",
      "================================================================================\n",
      "Installing:\n",
      " tcpreplay          x86_64          4.2.1-1.el7             epel          304 k\n",
      "Installing for dependencies:\n",
      " tcpdump            x86_64          14:4.5.1-3.el7          base          387 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package (+1 Dependent package)\n",
      "\n",
      "Total download size: 692 k\n",
      "Installed size: 2.2 M\n",
      "Public key for tcpreplay-4.2.1-1.el7.x86_64.rpm is not installed\n",
      "\n",
      "Installed:\n",
      "  tcpreplay.x86_64 0:4.2.1-1.el7                                                \n",
      "\n",
      "Dependency Installed:\n",
      "  tcpdump.x86_64 14:4.5.1-3.el7                                                 \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "warning: /var/cache/yum/x86_64/7/epel/packages/tcpreplay-4.2.1-1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY\n",
      "Importing GPG key 0x352C64E5:\n",
      " Userid     : \"Fedora EPEL (7) <epel@fedoraproject.org>\"\n",
      " Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5\n",
      " Package    : epel-release-7-9.noarch (installed)\n",
      " From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"yum -d 1 -e 0 -y install tcpreplay\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File Cache is enabled\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"tcpreplay -K -l 100000 -M 200 --intf1 enp0s8 /vagrant/snort-test-alert.pcap\"'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Note that the `attacker0` IP is not detected by `snort0`, but instead the count for alerts from `monitor0` increases significantly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep -v packet | grep '\"signature-id\": 3000003' | grep '192.168.17.10' | wc -l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "51\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep -v packet | grep '\"signature-id\": 3000003' | grep '192.168.17.20' | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "After this test make sure the `prometheus` level of the test snort alert is restored"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl start snort-test-alert'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "active\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl is-active snort-test-alert'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "sleep 300"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK - snort-test-alert is 6: { blocked: 0, dport_icode: 80, generator_id: 1, instance: snort0:9100, job: node, signature_id: 3000003, source_ip: 192.168.17.20 }\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"bash ./nagios_plugins/check_prometheus_metric.sh -H localhost:9090 -q \\\"rate(snort_alert_count{source_ip='192.168.17.20',signature_id='3000003'}[5m]) * 60\\\" -w 5 -c 4 -n snort-test-alert -m lt -t vector -i\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's now `tcprewrite` the source in the `snort-test-alert.pcap` to honestly report that we are an `attacker0`. `tcprewrite` of the destination would be necessary if `pcap` has been captured on a different machine than `snort0`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'tcprewrite --infile=/vagrant/snort-test-alert.pcap --outfile=/tmp/1.pcap --dstipmap=0.0.0.0/0:192.168.17.30 --enet-dmac=08:00:27:00:17:30 --portmap=80:80'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'tcprewrite --infile=/tmp/1.pcap --outfile=/tmp/2.pcap --srcipmap=0.0.0.0/0:192.168.17.10 --enet-smac=08:00:27:00:17:10'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Replay it, this time with 20 times more packets, this will take about 5 minutes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "File Cache is enabled\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"tcpreplay -K -l 2000000 -M 200 --intf1 enp0s8 /tmp/2.pcap\"'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The CPU/Memory usage by `snort` on `snort0` increases significantly (100% CPU) during the replay at a modest 200 Mbps. You can always login to any of the machines, e.g. **vagrant ssh snort0**. To gain root use **sudo su -**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "There are no alerts detected on the traffic replayed by `attacker0` (because we have not define a rule for such traffic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No alert message map entries loaded.\n",
      "WARNING: No classifications loaded.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'idstools-u2json /var/log/snort/unified2.log'\" | grep -v packet | grep '\"signature-id\": 3000003' | grep '192.168.17.10' | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "but the load on `snort0` caused by `attacker0` was sufficient for our snort test alerts from `monitor0` to be missed by `snort0`. Let's quickly check the count of the test alerts, the rate should be \"WARNING\" or \"CRITICAL\" if the increase of load on `snort0` was indeed sufficient:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OK - snort-test-alert is 6: { blocked: 0, dport_icode: 80, generator_id: 1, instance: snort0:9100, job: node, signature_id: 3000003, source_ip: 192.168.17.20 }\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "! vagrant ssh monitor0 -c \"bash ./nagios_plugins/check_prometheus_metric.sh -H localhost:9090 -q \\\"rate(snort_alert_count{source_ip='192.168.17.20',signature_id='3000003'}[5m]) * 60\\\" -w 5 -c 4 -n snort-test-alert -m lt -t vector -i\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Sometimes people blame `nfqueue` for dropping packets. The sixth column of `/proc/net/netfilter/nfnetlink_queue` reports the number of packets dropped by `nfqueue` - it is zero"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    0   7340     0 2 65531     0     0     3992  1\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'cat /proc/net/netfilter/nfnetlink_queue'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "It is an interesting subject for a discussion at `snort-users` what are the factors limiting the performance of this setup."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Pytbull"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "http://pytbull.sourceforge.net/ is a `python` module which automates scans against `snort`/`suricata` IDS/IPS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package            Arch            Version                 Repository     Size\n",
      "================================================================================\n",
      "Installing:\n",
      " mercurial          x86_64          2.6.2-6.el7_2           base          2.6 M\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 2.6 M\n",
      "Installed size: 12 M\n",
      "\n",
      "Installed:\n",
      "  mercurial.x86_64 0:2.6.2-6.el7_2                                              \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"yum -d 1 -e 0 -y install mercurial\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Package tcpreplay-4.2.1-1.el7.x86_64 already installed and latest version\n",
      "Package python-paramiko is obsoleted by python2-paramiko, trying to install python2-paramiko-1.16.1-2.el7.noarch instead\n",
      "\n",
      "================================================================================\n",
      " Package               Arch       Version                     Repository   Size\n",
      "================================================================================\n",
      "Installing:\n",
      " hping3                x86_64     0.0.20051105-24.el7         epel         96 k\n",
      " httpd-tools           x86_64     2.4.6-45.el7.centos.4       updates      84 k\n",
      " ncrack                x86_64     0.4-0.11.ALPHA.el7          epel        554 k\n",
      " nikto                 noarch     1:2.1.5-10.el7              epel        263 k\n",
      " python-cherrypy       noarch     3.2.2-4.el7                 base        422 k\n",
      " python-feedparser     noarch     5.1.3-3.el7                 epel        107 k\n",
      " python2-paramiko      noarch     1.16.1-2.el7                epel        258 k\n",
      " scapy                 noarch     2.2.0-2.el7                 epel        489 k\n",
      "Installing for dependencies:\n",
      " apr                   x86_64     1.4.8-3.el7                 base        103 k\n",
      " apr-util              x86_64     1.5.2-6.el7                 base         92 k\n",
      " libtomcrypt           x86_64     1.17-23.el7                 epel        224 k\n",
      " libtommath            x86_64     0.42.0-4.el7                epel         35 k\n",
      " perl-libwhisker2      noarch     2.5-14.el7                  epel         89 k\n",
      " python-chardet        noarch     2.2.1-1.el7_1               base        227 k\n",
      " python-six            noarch     1.9.0-2.el7                 base         29 k\n",
      " python2-crypto        x86_64     2.6.1-13.el7                epel        476 k\n",
      " python2-ecdsa         noarch     0.13-4.el7                  epel         83 k\n",
      " tcl                   x86_64     1:8.5.13-8.el7              base        1.9 M\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  8 Packages (+10 Dependent packages)\n",
      "\n",
      "Total download size: 5.4 M\n",
      "Installed size: 18 M\n",
      "\n",
      "Installed:\n",
      "  hping3.x86_64 0:0.0.20051105-24.el7                                           \n",
      "  httpd-tools.x86_64 0:2.4.6-45.el7.centos.4                                    \n",
      "  ncrack.x86_64 0:0.4-0.11.ALPHA.el7                                            \n",
      "  nikto.noarch 1:2.1.5-10.el7                                                   \n",
      "  python-cherrypy.noarch 0:3.2.2-4.el7                                          \n",
      "  python-feedparser.noarch 0:5.1.3-3.el7                                        \n",
      "  python2-paramiko.noarch 0:1.16.1-2.el7                                        \n",
      "  scapy.noarch 0:2.2.0-2.el7                                                    \n",
      "\n",
      "Dependency Installed:\n",
      "  apr.x86_64 0:1.4.8-3.el7               apr-util.x86_64 0:1.5.2-6.el7          \n",
      "  libtomcrypt.x86_64 0:1.17-23.el7       libtommath.x86_64 0:0.42.0-4.el7       \n",
      "  perl-libwhisker2.noarch 0:2.5-14.el7   python-chardet.noarch 0:2.2.1-1.el7_1  \n",
      "  python-six.noarch 0:1.9.0-2.el7        python2-crypto.x86_64 0:2.6.1-13.el7   \n",
      "  python2-ecdsa.noarch 0:0.13-4.el7      tcl.x86_64 1:8.5.13-8.el7              \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"yum -d 1 -e 0 -y install python-feedparser python-cherrypy nikto hping3 tcpreplay httpd-tools python-paramiko ncrack scapy\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "destination directory: pytbull\n",
      "requesting all changes\n",
      "adding changesets\n",
      "adding manifests\n",
      "adding file changes\n",
      "added 17 changesets with 199 changes to 148 files\n",
      "updating to branch default\n",
      "139 files updated, 0 files merged, 0 files removed, 0 files unresolved\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"hg clone http://pytbull.hg.sourceforge.net:8000/hgroot/pytbull/pytbull\"'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c 'sudo su - -c \"cp -f /vagrant/pytbull_config.cfg ~/pytbull/conf/config.cfg\"'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's stop the test alert not to confuse `pytbull`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"sudo su - -c 'systemctl stop snort-test-alert'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "`pytbull` is expecting to have access to `snort0:/var/log/snort/alert_fast.txt` in the `-A fast` format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl stop snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i 's/-A unified2/-A fast/' /usr/lib/systemd/system/snort@.service\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl daemon-reload'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i 's/^unified2 =/--unified2 =/' /etc/snort/snort.lua\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"echo alert_fast = { file = true, limit = 128, units = \\'M\\' } >> /etc/snort/snort.lua\\\"\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'rm -f /var/log/snort/alert_fast.txt'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Remove our ping drops, as they will fire on some `pytbull` tests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c \\\"sed -i '/drop icmp/d' /etc/snort/rules/snort.rules\\\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl start snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "active\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'systemctl is-active snort@enp0s8'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Moreover `pytbull` wants an `ftp` server on the `snort0` sensor to access `/var/log/snort/alert_fast.txt`. This is not something one wants on a production sensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================================================================================\n",
      " Package          Arch             Version                 Repository      Size\n",
      "================================================================================\n",
      "Installing:\n",
      " vsftpd           x86_64           3.0.2-21.el7            base           169 k\n",
      "\n",
      "Transaction Summary\n",
      "================================================================================\n",
      "Install  1 Package\n",
      "\n",
      "Total download size: 169 k\n",
      "Installed size: 348 k\n",
      "\n",
      "Installed:\n",
      "  vsftpd.x86_64 0:3.0.2-21.el7                                                  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'yum -d 1 -e 0 -y install vsftpd&& systemctl start vsftpd'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'getent passwd pilou > /dev/null || useradd pilou'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Changing password for user pilou.\n",
      "passwd: all authentication tokens updated successfully.\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'echo oopsoops | passwd --stdin pilou'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'chmod go+rx /var/log/snort'\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'chmod go+r /var/log/snort/alert_fast.txt'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Test whether `snort` is still able to detect the test alert"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n",
      "<html><head>\n",
      "<title>404 Not Found</title>\n",
      "</head><body>\n",
      "<h1>Not Found</h1>\n",
      "<p>The requested URL /snort-test-alert was not found on this server.</p>\n",
      "</body></html>\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh monitor0 -c \"curl -s -m 1 http://snort0/snort-test-alert\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "05/03-15:15:54.241409 [**] [1:3000003:1] \"snort-test-alert\" [**] [Classification: Not Suspicious Traffic] [Priority: 3] {TCP} 192.168.17.20:60450 -> 192.168.17.30:80\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c \"sudo su - -c 'grep 3000003 /var/log/snort/alert_fast.txt'\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Run `pytbull`, it should takes less than 5 minutes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Starting Nmap 6.40 ( http://nmap.org ) at 2017-05-03 15:16 UTC\n",
      "Nmap scan report for snort0 (192.168.17.30)\n",
      "Host is up (0.00091s latency).\n",
      "Not shown: 65530 closed ports\n",
      "PORT     STATE SERVICE\n",
      "21/tcp   open  ftp\n",
      "22/tcp   open  ssh\n",
      "80/tcp   open  http\n",
      "111/tcp  open  rpcbind\n",
      "9100/tcp open  jetdirect\n",
      "MAC Address: 08:00:27:00:17:30 (Cadmus Computer Systems)\n",
      "\n",
      "Nmap done: 1 IP address (1 host up) scanned in 3.66 seconds\n",
      "\n",
      "Starting Nmap 6.40 ( http://nmap.org ) at 2017-05-03 15:16 UTC\n",
      "Nmap scan report for snort0 (192.168.17.30)\n",
      "Host is up (0.0090s latency).\n",
      "Not shown: 65530 closed ports\n",
      "PORT     STATE SERVICE\n",
      "21/tcp   open  ftp\n",
      "22/tcp   open  ssh\n",
      "80/tcp   open  http\n",
      "111/tcp  open  rpcbind\n",
      "9100/tcp open  jetdirect\n",
      "MAC Address: 08:00:27:00:17:30 (Cadmus Computer Systems)\n",
      "\n",
      "Nmap done: 1 IP address (1 host up) scanned in 6.32 seconds\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Start Time:         2017-05-03 15:16:33 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:16:33 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "\n",
      "Starting Nmap 6.40 ( http://nmap.org ) at 2017-05-03 15:16 UTC\n",
      "Nmap scan report for snort0 (192.168.17.30)\n",
      "Host is up (0.00023s latency).\n",
      "PORT   STATE         SERVICE\n",
      "80/tcp open|filtered http\n",
      "MAC Address: 08:00:27:00:17:30 (Cadmus Computer Systems)\n",
      "\n",
      "Nmap done: 1 IP address (1 host up) scanned in 0.23 seconds\n",
      "\n",
      "                                 _   _           _ _\n",
      "                     _ __  _   _| |_| |__  _   _| | |\n",
      "                    | '_ \\| | | | __| '_ \\| | | | | |\n",
      "                    | |_) | |_| | |_| |_) | |_| | | |\n",
      "                    | .__/ \\__, |\\__|_.__/ \\__,_|_|_|\n",
      "                    |_|    |___/\n",
      "                       Sebastien Damaye, aldeid.com\n",
      "\n",
      "What would you like to do?\n",
      "1. Run a new campaign (will erase previous results)\n",
      "2. View results from previous campaign\n",
      "3. Exit\n",
      "Choose an option: \n",
      "(gateway mode)\n",
      "(debug mode)\n",
      "(offline)\n",
      "\n",
      "+------------------------------------------------------------------------+\n",
      "| pytbull will set off IDS/IPS alarms and/or other security devices      |\n",
      "| and security monitoring software. The user is aware that malicious     |\n",
      "| content will be downloaded and that the user should have been          |\n",
      "| authorized before running the tool.                                    |\n",
      "+------------------------------------------------------------------------+\n",
      "Do you accept (y/n)? \n",
      "BASIC CHECKS\n",
      "------------\n",
      "Checking root privileges......................................... [   OK   ]\n",
      "Checking remote port 21/tcp (FTP)................................ [   OK   ]\n",
      "Checking remote port 22/tcp (SSH)................................ [   OK   ]\n",
      "Checking remote port 80/tcp (HTTP)............................... [   OK   ]\n",
      "Checking path for sudo........................................... [   OK   ]\n",
      "Checking path for nmap........................................... [   OK   ]\n",
      "Checking path for nikto.......................................... [   OK   ]\n",
      "Checking path for niktoconf...................................... [   OK   ]\n",
      "Checking path for hping3......................................... [   OK   ]\n",
      "Checking path for tcpreplay...................................... [   OK   ]\n",
      "Checking path for ab............................................. [   OK   ]\n",
      "Checking path for ping........................................... [   OK   ]\n",
      "Checking path for ncrack......................................... [   OK   ]\n",
      "Removing temporary file.......................................... [   OK   ]\n",
      "Cleaning database................................................ [   OK   ]\n",
      "\n",
      "TESTS\n",
      "------------\n",
      "Client Side Attacks.............................................. [   no   ]\n",
      "Test Rules....................................................... [   yes  ]\n",
      "Bad Traffic...................................................... [   yes  ]\n",
      "Fragmented Packets............................................... [   yes  ]\n",
      "Brute Force...................................................... [   yes  ]\n",
      "Evasion Techniques............................................... [   yes  ]\n",
      "ShellCodes....................................................... [   no   ]\n",
      "Denial of Service................................................ [   yes  ]\n",
      "Pcap Replay...................................................... [   no   ]\n",
      "Normal Usage..................................................... [   yes  ]\n",
      "IP Reputation.................................................... [   no   ]\n",
      "\n",
      "\n",
      "TEST RULES\n",
      "------------\n",
      "TEST #1 - Simple LFI............................................. [  done  ]\n",
      "TEST #2 - LFI using NULL byte.................................... [  done  ]\n",
      "TEST #3 - Full SYN Scan.......................................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nmap -sS -p- snort0\n",
      "[  done  ]\n",
      "TEST #4 - Full Connect() Scan.................................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/nmap -sT -p- snort0\n",
      "[  done  ]\n",
      "TEST #5 - SQL Injection.......................................... [  done  ]\n",
      "TEST #6 - Netcat Reverse Shell................................... [  done  ]\n",
      "TEST #7 - Nikto Scan............................................. \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi\n",
      "[  done  ]\n",
      "\n",
      "BAD TRAFFIC\n",
      "------------\n",
      "TEST #8 - Nmap Xmas scan......................................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nmap -sX -p 80 snort0\n",
      "[  done  ]\n",
      "TEST #9 - Malformed Traffic............................................................................\n",
      "Starting Ncrack 0.4ALPHA ( http://ncrack.org ) at 2017-05-03 15:17 UTC\n",
      "\n",
      "\n",
      "Ncrack done: 1 service scanned in 9.00 seconds.\n",
      "\n",
      "Ncrack finished.\n",
      "\n",
      "Starting Nmap 6.40 ( http://nmap.org ) at 2017-05-03 15:17 UTC\n",
      "Nmap scan report for snort0 (192.168.17.30)\n",
      "Host is up (0.00044s latency).\n",
      "Not shown: 995 closed ports\n",
      "PORT     STATE SERVICE    VERSION\n",
      "21/tcp   open  ftp        vsftpd 3.0.2\n",
      "| ftp-anon: Anonymous FTP login allowed (FTP code 230)\n",
      "|_drwxr-xr-x    2 0        0               6 Nov 05 19:43 pub\n",
      "22/tcp   open  ssh        OpenSSH 6.6.1 (protocol 2.0)\n",
      "| ssh-hostkey: 2048 5b:b2:16:02:76:fd:8e:3c:dc:e0:f3:df:31:3d:4b:e1 (RSA)\n",
      "|_256 2a:dd:72:1a:f5:01:be:03:cc:d2:3c:33:11:3b:77:f0 (ECDSA)\n",
      "80/tcp   open  http       Apache httpd 2.4.6 ((CentOS))\n",
      "| http-methods: Potentially risky methods: TRACE\n",
      "|_See http://nmap.org/nsedoc/scripts/http-methods.html\n",
      "|_http-title: Apache HTTP Server Test Page powered by CentOS\n",
      "111/tcp  open  rpcbind    2-4 (RPC #100000)\n",
      "| rpcinfo: \n",
      "|   program version   port/proto  service\n",
      "|   100000  2,3,4        111/tcp  rpcbind\n",
      "|_  100000  2,3,4        111/udp  rpcbind\n",
      "9100/tcp open  jetdirect?\n",
      "MAC Address: 08:00:27:00:17:30 (Cadmus Computer Systems)\n",
      "No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ).\n",
      "TCP/IP fingerprint:\n",
      "OS:SCAN(V=6.40%E=4%D=5/3%OT=21%CT=1%CU=36254%PV=Y%DS=1%DC=D%G=Y%M=080027%TM\n",
      "OS:=5909F491%P=x86_64-redhat-linux-gnu)SEQ(SP=F2%GCD=1%ISR=103%TI=Z%II=I%TS\n",
      "OS:=A)SEQ(SP=F2%GCD=1%ISR=103%TI=Z%TS=A)OPS(O1=M5B4ST11NW6%O2=M5B4ST11NW6%O\n",
      "OS:3=M5B4NNT11NW6%O4=M5B4ST11NW6%O5=M5B4ST11NW6%O6=M5B4ST11)WIN(W1=7120%W2=\n",
      "OS:7120%W3=7120%W4=7120%W5=7120%W6=7120)ECN(R=Y%DF=Y%T=40%W=7210%O=M5B4NNSN\n",
      "OS:W6%CC=Y%Q=)T1(R=Y%DF=Y%T=40%S=O%A=S+%F=AS%RD=0%Q=)T2(R=N)T3(R=N)T4(R=Y%D\n",
      "OS:F=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T5(R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%O\n",
      "OS:=%RD=0%Q=)T6(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T7(R=Y%DF=Y%T=40%W\n",
      "OS:=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)U1(R=Y%DF=N%T=40%IPL=164%UN=0%RIPL=G%RID=G%R\n",
      "OS:IPCK=G%RUCK=G%RUD=G)IE(R=Y%DFI=N%T=40%CD=S)\n",
      "\n",
      "Network Distance: 1 hop\n",
      "Service Info: OS: Unix\n",
      "\n",
      "TRACEROUTE\n",
      "HOP RTT     ADDRESS\n",
      "1   0.44 ms snort0 (192.168.17.30)\n",
      "\n",
      "OS and Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .\n",
      "Nmap done: 1 IP address (1 host up) scanned in 17.93 seconds\n",
      "\n",
      "Starting Nmap 6.40 ( http://nmap.org ) at 2017-05-03 15:17 UTC\n",
      "Nmap scan report for snort0 (192.168.17.30)\n",
      "Host is up (0.00049s latency).\n",
      "Not shown: 995 closed ports\n",
      "PORT     STATE SERVICE    VERSION\n",
      "21/tcp   open  ftp        vsftpd 3.0.2\n",
      "| ftp-anon: Anonymous FTP login allowed (FTP code 230)\n",
      "|_drwxr-xr-x    2 0        0               6 Nov 05 19:43 pub\n",
      "22/tcp   open  ssh        OpenSSH 6.6.1 (protocol 2.0)\n",
      "| ssh-hostkey: 2048 5b:b2:16:02:76:fd:8e:3c:dc:e0:f3:df:31:3d:4b:e1 (RSA)\n",
      "|_256 2a:dd:72:1a:f5:01:be:03:cc:d2:3c:33:11:3b:77:f0 (ECDSA)\n",
      "80/tcp   open  http       Apache httpd 2.4.6 ((CentOS))\n",
      "| http-methods: Potentially risky methods: TRACE\n",
      "|_See http://nmap.org/nsedoc/scripts/http-methods.html\n",
      "|_http-title: Apache HTTP Server Test Page powered by CentOS\n",
      "111/tcp  open  rpcbind    2-4 (RPC #100000)\n",
      "| rpcinfo: \n",
      "|   program version   port/proto  service\n",
      "|   100000  2,3,4        111/tcp  rpcbind\n",
      "|_  100000  2,3,4        111/udp  rpcbind\n",
      "9100/tcp open  jetdirect?\n",
      "MAC Address: 08:00:27:00:17:30 (Cadmus Computer Systems)\n",
      "No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ).\n",
      "TCP/IP fingerprint:\n",
      "OS:SCAN(V=6.40%E=4%D=5/3%OT=21%CT=1%CU=30305%PV=Y%DS=1%DC=D%G=Y%M=080027%TM\n",
      "OS:=5909F4A7%P=x86_64-redhat-linux-gnu)SEQ(SP=104%GCD=1%ISR=105%TI=Z%TS=A)S\n",
      "OS:EQ(SP=104%GCD=1%ISR=105%TI=Z%CI=I%TS=A)SEQ(SP=104%GCD=1%ISR=105%TI=Z%II=\n",
      "OS:I%TS=A)OPS(O1=M5B4ST11NW6%O2=M5B4ST11NW6%O3=M5B4NNT11NW6%O4=M5B4ST11NW6%\n",
      "OS:O5=M5B4ST11NW6%O6=M5B4ST11)WIN(W1=7120%W2=7120%W3=7120%W4=7120%W5=7120%W\n",
      "OS:6=7120)ECN(R=Y%DF=Y%T=40%W=7210%O=M5B4NNSNW6%CC=Y%Q=)T1(R=Y%DF=Y%T=40%S=\n",
      "OS:O%A=S+%F=AS%RD=0%Q=)T2(R=N)T3(R=N)T4(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD\n",
      "OS:=0%Q=)T5(R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)T6(R=Y%DF=Y%T=40%W=0\n",
      "OS:%S=A%A=Z%F=R%O=%RD=0%Q=)T7(R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)U1\n",
      "OS:(R=Y%DF=N%T=40%IPL=164%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)IE(R=Y%DFI\n",
      "OS:=N%T=40%CD=S)\n",
      "\n",
      "Network Distance: 1 hop\n",
      "Service Info: OS: Unix\n",
      "\n",
      "TRACEROUTE\n",
      "HOP RTT     ADDRESS\n",
      "1   0.49 ms snort0 (192.168.17.30)\n",
      "\n",
      "OS and Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .\n",
      "Nmap done: 1 IP address (1 host up) scanned in 17.96 seconds\n",
      "\n",
      "Starting Nmap 6.40 ( http://nmap.org ) at 2017-05-03 15:18 UTC\n",
      "Nmap scan report for snort0 (192.168.17.30)\n",
      "Host is up (0.00052s latency).\n",
      "Not shown: 995 closed ports\n",
      "PORT     STATE SERVICE    VERSION\n",
      "21/tcp   open  ftp        vsftpd 3.0.2\n",
      "| ftp-anon: Anonymous FTP login allowed (FTP code 230)\n",
      "|_drwxr-xr-x    2 0        0               6 Nov 05 19:43 pub\n",
      "22/tcp   open  ssh        OpenSSH 6.6.1 (protocol 2.0)\n",
      "| ssh-hostkey: 2048 5b:b2:16:02:76:fd:8e:3c:dc:e0:f3:df:31:3d:4b:e1 (RSA)\n",
      "|_256 2a:dd:72:1a:f5:01:be:03:cc:d2:3c:33:11:3b:77:f0 (ECDSA)\n",
      "80/tcp   open  http       Apache httpd 2.4.6 ((CentOS))\n",
      "| http-methods: Potentially risky methods: TRACE\n",
      "|_See http://nmap.org/nsedoc/scripts/http-methods.html\n",
      "|_http-title: Apache HTTP Server Test Page powered by CentOS\n",
      "111/tcp  open  rpcbind    2-4 (RPC #100000)\n",
      "| rpcinfo: \n",
      "|   program version   port/proto  service\n",
      "|   100000  2,3,4        111/tcp  rpcbind\n",
      "|_  100000  2,3,4        111/udp  rpcbind\n",
      "9100/tcp open  jetdirect?\n",
      "MAC Address: 08:00:27:00:17:30 (Cadmus Computer Systems)\n",
      "No exact OS matches for host (If you know what OS is running on it, see http://nmap.org/submit/ ).\n",
      "TCP/IP fingerprint:\n",
      "OS:SCAN(V=6.40%E=4%D=5/3%OT=21%CT=1%CU=40963%PV=Y%DS=1%DC=D%G=Y%M=080027%TM\n",
      "OS:=5909F4C1%P=x86_64-redhat-linux-gnu)SEQ(SP=105%GCD=1%ISR=10D%TI=Z%TS=A)S\n",
      "OS:EQ(SP=105%GCD=1%ISR=10D%TI=Z%II=I%TS=A)OPS(O1=M5B4ST11NW6%O2=M5B4ST11NW6\n",
      "OS:%O3=M5B4NNT11NW6%O4=M5B4ST11NW6%O5=M5B4ST11NW6%O6=M5B4ST11)WIN(W1=7120%W\n",
      "OS:2=7120%W3=7120%W4=7120%W5=7120%W6=7120)ECN(R=Y%DF=Y%T=40%W=7210%O=M5B4NN\n",
      "OS:SNW6%CC=Y%Q=)T1(R=Y%DF=Y%T=40%S=O%A=S+%F=AS%RD=0%Q=)T2(R=N)T3(R=N)T4(R=Y\n",
      "OS:%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T5(R=Y%DF=Y%T=40%W=0%S=Z%A=S+%F=AR\n",
      "OS:%O=%RD=0%Q=)T6(R=Y%DF=Y%T=40%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)T7(R=Y%DF=Y%T=40\n",
      "OS:%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)U1(R=Y%DF=N%T=40%IPL=164%UN=0%RIPL=G%RID=G\n",
      "OS:%RIPCK=G%RUCK=G%RUD=G)IE(R=Y%DFI=N%T=40%CD=S)\n",
      "\n",
      "Network Distance: 1 hop\n",
      "Service Info: OS: Unix\n",
      "\n",
      "TRACEROUTE\n",
      "HOP RTT     ADDRESS\n",
      "1   0.52 ms snort0 (192.168.17.30)\n",
      "\n",
      "OS and Service detection performed. Please report any incorrect results at http://nmap.org/submit/ .\n",
      "Nmap done: 1 IP address (1 host up) scanned in 17.92 seconds\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Random URI encoding (non-UTF8)\n",
      "+ Start Time:         2017-05-03 15:18:30 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:30 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Directory self-reference (/./)\n",
      "+ Start Time:         2017-05-03 15:18:34 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:34 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Premature URL ending\n",
      "+ Start Time:         2017-05-03 15:18:38 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:39 (GMT0) (1 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Prepend long random string\n",
      "+ Start Time:         2017-05-03 15:18:43 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:43 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Fake parameter\n",
      "+ Start Time:         2017-05-03 15:18:47 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:48 (GMT0) (1 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     TAB as request spacer\n",
      "+ Start Time:         2017-05-03 15:18:52 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:52 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Change the case of the URL\n",
      "+ Start Time:         2017-05-03 15:18:56 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:18:57 (GMT0) (1 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Using Encoding:     Use Windows directory separator (\\)\n",
      "+ Start Time:         2017-05-03 15:19:01 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:19:01 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Start Time:         2017-05-03 15:19:05 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:19:05 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "- ***** RFIURL is not defined in nikto.conf--no RFI tests will run *****\n",
      "- ***** SSL support not available (see docs for SSL install) *****\n",
      "- Nikto v2.1.5\n",
      "---------------------------------------------------------------------------\n",
      "+ Target IP:          192.168.17.30\n",
      "+ Target Hostname:    snort0\n",
      "+ Target Port:        80\n",
      "+ Start Time:         2017-05-03 15:19:10 (GMT0)\n",
      "---------------------------------------------------------------------------\n",
      "+ Server: Apache/2.4.6 (CentOS)\n",
      "+ 4197 items checked: 0 error(s) and 0 item(s) reported on remote host\n",
      "+ End Time:           2017-05-03 15:19:10 (GMT0) (0 seconds)\n",
      "---------------------------------------------------------------------------\n",
      "+ 1 host(s) tested\n",
      "........ \n",
      "\n",
      "***Debug: sending scapy payload: send(IP(dst=\"snort0\", ihl=2, version=3)/ICMP(), verbose=0)\n",
      "\n",
      "Sent 1 packets.\n",
      "[  done  ]\n",
      "TEST #10 - Land Attack........................................... \n",
      "\n",
      "***Debug: sending scapy payload: send(IP(src=\"snort0\",dst=\"snort0\")/TCP(sport=135,dport=135), verbose=0)\n",
      "\n",
      "Sent 1 packets.\n",
      "[  done  ]\n",
      "\n",
      "FRAGMENTED PACKETS\n",
      "------------\n",
      "TEST #11 - Ping of death......................................... \n",
      "\n",
      "***Debug: sending scapy payload: send(fragment(IP(dst=\"snort0\")/ICMP()/(\"X\"*60000)), verbose=0)\n",
      "\n",
      "Sent 41 packets.\n",
      "[  done  ]\n",
      "TEST #12 - Nestea Attack 1/3..................................... \n",
      "\n",
      "***Debug: sending scapy payload: send(IP(dst=\"snort0\", id=42, flags=\"MF\")/UDP()/(\"X\"*10), verbose=0)\n",
      "\n",
      "Sent 1 packets.\n",
      "[  done  ]\n",
      "TEST #13 - Nestea Attack 2/3..................................... \n",
      "\n",
      "***Debug: sending scapy payload: send(IP(dst=\"snort0\", id=42, frag=48)/(\"X\"*116), verbose=0)\n",
      "\n",
      "Sent 1 packets.\n",
      "[  done  ]\n",
      "TEST #14 - Nestea Attack 3/3..................................... \n",
      "\n",
      "***Debug: sending scapy payload: send(IP(dst=\"snort0\", id=42, flags=\"MF\")/UDP()/(\"X\"*224), verbose=0)\n",
      "\n",
      "Sent 1 packets.\n",
      "[  done  ]\n",
      "\n",
      "BRUTE FORCE\n",
      "------------\n",
      "TEST #15 - Bruteforce against FTP with ncrack.................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/ncrack -f -U data/ncrack-users.txt -P data/ncrack-passwords.txt snort0:21\n",
      "[  done  ]\n",
      "\n",
      "EVASION TECHNIQUES\n",
      "------------\n",
      "TEST #16 - Nmap decoy test (6th position)........................ \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nmap -sS -A -D 192.168.100.1,192.168.100.2,192.168.100.3,192.168.100.4,192.168.100.5,ME snort0\n",
      "[  done  ]\n",
      "TEST #17 - Nmap decoy test (7th position)........................ \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nmap -sS -A -D 192.168.100.1,192.168.100.2,192.168.100.3,192.168.100.4,192.168.100.5,192.168.100.6,ME snort0\n",
      "[  done  ]\n",
      "TEST #18 - Hex encoding.......................................... [  done  ]\n",
      "TEST #19 - Nmap scan with fragmentation.......................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nmap -PN -sS -A -f snort0\n",
      "[  done  ]\n",
      "TEST #20 - Nikto Random URI encoding............................. \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 1\n",
      "[  done  ]\n",
      "TEST #21 - Nikto Directory self reference........................ \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 2\n",
      "[  done  ]\n",
      "TEST #22 - Nikto Premature URL ending............................ \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 3\n",
      "[  done  ]\n",
      "TEST #23 - Nikto Prepend long random string...................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 4\n",
      "[  done  ]\n",
      "TEST #24 - Nikto Fake parameter.................................. \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 5\n",
      "[  done  ]\n",
      "TEST #25 - Nikto TAB as request spacer........................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 6\n",
      "[  done  ]\n",
      "TEST #26 - Nikto Change the case of the URL...................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 7\n",
      "[  done  ]\n",
      "TEST #27 - Nikto Windows directory separator..................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion 8\n",
      "[  done  ]\n",
      "TEST #28 - Nikto Carriage return as request spacer............... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion A\n",
      "[  done  ]\n",
      "TEST #29 - Nikto Binary value as request spacer.................. \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/bin/nikto -config /etc/nikto/config -h snort0 -Plugins cgi -evasion B\n",
      "[  done  ]\n",
      "TEST #30 - Javascript Obfuscation.............................This is ApacheBench, Version 2.3 <$Revision: 1430300 $>\n",
      "Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/\n",
      "Licensed to The Apache Software Foundation, http://www.apache.org/\n",
      "\n",
      "Benchmarking %target% (be patient)\n",
      "HPING snort0 (enp0s8 192.168.17.30): S set, 40 headers + 0 data bytes\n",
      "This is ApacheBench, Version 2.3 <$Revision: 1430300 $>\n",
      "Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/\n",
      "Licensed to The Apache Software Foundation, http://www.apache.org/\n",
      "\n",
      "Benchmarking %target% (be patient)...PING snort0 (192.168.17.30) 56(84) bytes of data.\n",
      "64 bytes from snort0 (192.168.17.30): icmp_seq=1 ttl=64 time=0.343 ms\n",
      "\n",
      "--- snort0 ping statistics ---\n",
      "1 packets transmitted, 1 received, 0% packet loss, time 0ms\n",
      "rtt min/avg/max/mdev = 0.343/0.343/0.343/0.000 ms\n",
      "... [  done  ]\n",
      "\n",
      "DENIAL OF SERVICE\n",
      "------------\n",
      "TEST #31 - DoS against MSSQL..................................... \n",
      "\n",
      "***Debug: sending scapy payload: sr1(IP(dst=\"snort0\")/TCP(dport=1433)/\"0\"*1000, verbose=0)\n",
      "\n",
      "Received 1001 packets, got 1000 answers, remaining 0 packets\n",
      "[  done  ]\n",
      "TEST #32 - ApacheBench DoS....................................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/ab -k -c 25 -n 10000 http://%target%/\n",
      "[  done  ]\n",
      "TEST #33 - hping SYN flood....................................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/sudo /usr/sbin/hping3 snort0 -S --faster -p 80 -I enp0s8 -c 50000 -a 1.2.3.4\n",
      "[  done  ]\n",
      "\n",
      "NORMAL USAGE\n",
      "------------\n",
      "TEST #34 - ApacheBench 10 requests............................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/ab -k -c 10 -n 10 http://%target%/\n",
      "[  done  ]\n",
      "TEST #35 - Standard ping......................................... \n",
      "\n",
      "***Debug: sending command: /usr/bin/ping -c 1 snort0\n",
      "[  done  ]\n",
      "\n",
      "\n",
      "-----------------------\n",
      "DONE. Check the report.\n",
      "-----------------------\n",
      "\n",
      "\n",
      "Webserver started at http://127.0.0.1:8080\n",
      "(use ^C to stop)\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: No route found for IPv6 destination :: (no default route?)\n",
      "apr_sockaddr_info_get() for %target%: Name or service not known (670002)\n",
      "\n",
      "--- snort0 hping statistic ---\n",
      "50000 packets transmitted, 0 packets received, 100% packet loss\n",
      "round-trip min/avg/max = 0.0/0.0/0.0 ms\n",
      "apr_sockaddr_info_get() for %target%: Name or service not known (670002)\n",
      "[03/May/2017:15:19:45] ENGINE Listening for SIGHUP.\n",
      "[03/May/2017:15:19:45] ENGINE Listening for SIGTERM.\n",
      "[03/May/2017:15:19:45] ENGINE Listening for SIGUSR1.\n",
      "[03/May/2017:15:19:45] ENGINE Bus STARTING\n",
      "[03/May/2017:15:19:45] ENGINE Started monitor thread '_TimeoutMonitor'.\n",
      "[03/May/2017:15:19:45] ENGINE Serving on 127.0.0.1:8080\n",
      "[03/May/2017:15:19:45] ENGINE Bus STARTED\n",
      "[03/May/2017:15:27:58] ENGINE Caught signal SIGTERM.\n",
      "[03/May/2017:15:27:58] ENGINE Bus STOPPING\n",
      "[03/May/2017:15:27:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 8080)) shut down\n",
      "[03/May/2017:15:27:58] ENGINE Stopped thread '_TimeoutMonitor'.\n",
      "[03/May/2017:15:27:58] ENGINE Bus STOPPED\n",
      "[03/May/2017:15:27:58] ENGINE Bus EXITING\n",
      "[03/May/2017:15:27:58] ENGINE Bus EXITED\n",
      "[03/May/2017:15:27:58] ENGINE Waiting for child threads to terminate...\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh attacker0 -c \"sudo su - -c \\\"cd /root/pytbull; echo -e '1\\ny\\n' | timeout 720 ./pytbull --debug --offline --mode=gateway -t snort0\\\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "`pytbull` starts a web server and exposes a summary of the scan as html, but in this setup it finds no\n",
    "alerts are triggered, apart from PSNG_TCP_PORTSCAN. Something is not right here, it is due to several reasons:\n",
    " \n",
    "- the rules alerting on preprocessors are not enabled in `snort` by default. They are available from snortrules-snapshot-*.tar.gz.\n",
    "\n",
    "- no rules specific for the tests performed by `pytbull` are enabled\n",
    "\n",
    "- the scanners manage to evade IDS/IPS \"Black Hat USA 2013 - Evading deep inspection for fun and shell\" https://www.youtube.com/watch?v=BkmPZhgLmRo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Versions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "snort-3.0.0-0.232.a4.el7.centos.x86_64\n",
      "snort-extra-1.0.0-0.232.a4.el7.centos.x86_64\n",
      "daq-2.2.1-1.el7.centos.x86_64\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant ssh snort0 -c 'rpm -q snort snort-extra daq'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vagrant 1.9.2\n",
      "5.1.18r114002\n",
      "PRETTY_NAME=\"Ubuntu 16.04.2 LTS\"\n",
      "Linux ubuntu 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant --version\n",
    "VBoxManage --version\n",
    "cat /etc/os-release | grep PRETTY_NAME\n",
    "uname -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# When done, destroy the test machines"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==> snort0: Forcing shutdown of VM...\n",
      "==> snort0: Destroying VM and associated drives...\n",
      "==> monitor0: Forcing shutdown of VM...\n",
      "==> monitor0: Destroying VM and associated drives...\n",
      "==> attacker0: Forcing shutdown of VM...\n",
      "==> attacker0: Destroying VM and associated drives...\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "vagrant destroy -f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# License\n",
    "\n",
    "BSD 2-clause"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}