docs from homer-app , heplify, rtcagent and captagent repos

docs/captagent_architecture.md

@@ -0,0 +1,80 @@
+## Abstract
+CaptAgent has been around for a while and went through several redesign phases.
+
+The current version (6) has been completely redesigned from the ground up and ships with new internal architecture expanding its performance range and core capabilities to handle more network protocols, the ability to relay and aggregate traffic for remote agents, parse and handle RTCP-XR and RTP statistics, RPC centralized control and much more in a modular architecture.
+
+## Internal Architecture
+##### Sockets, Pipes and Plans
+
+Captagent 6 features a fully modular design enabling users to design and program their packet capture and processing logic, leveraging specialized functionality provided on-demand via loadable dynamic modules.
+
+Core module types include:
+
+| type  | description  |
+|:--|:--|
+| socket  | responsible for capturing ingress packets according to settings _(ie: PCAP, RAW)_ |
+| protocol  | responsible for processing/dissecting/parsing protocol data _(ie: SIP, RTCP)_ |
+| transport  | responsible for providing egress transport for generated data _(ie: HEP, JSON)_|
+| function  | responsible for providing additional functionality _(ie: database, etc)_ |
+
+Core modules are loaded via the main ```captagent.xml``` configuration file and can be easily concatenated to create multiple, independent capture chains:
+
+<img src="http://i.imgur.com/GalHujZ.png">
+
+In the above example:
+
+```SOCKET``` -> ```PROFILE``` -> ```CAPTURE PLAN``` <--> ```MODULES (functions)```
+
+-------------
+
+
+###### CAPTURE CHAINS
+For each chain, the logic and functionality is managed using a "capture-plan" which defines the behavior of the packet processing pipe. Capture plans are defined within the socket configuration alongside the general capture settings. An example for PCAP socket follows:
+```
+            <settings>
+		<param name="dev" value="any"/>
+		<param name="promisc" value="true"/>
+		<param name="reasm" value="false"/>
+		<param name="capture-plan" value="sip_capture_plan.cfg"/>
+		<param name="filter">
+		    <value>portrange 5060-5091</value>
+		</param>
+	    </settings>
+```
+
+In the above example, packets captured by the socket would be processed by *capture-plan* in  ```sip_capture_plan.cfg```:
+
+```
+# PCAP socket module
+capture[pcap] {
+        # PROTO SIP module
+	# Ie: check source/destination IP/port, message size, etc.
+	if(msg_check("size", "100")) {
+	    # Parse SIP Protocol
+	    if(parse_sip()) {
+		# use HEP TRANSPORT module (transport_hep.xml)	
+		if(!send_hep("hepsocket")) {
+		    clog("ERROR", "Error sending HEP!");
+		}
+            }
+       }
+}
+```
+
+The capture-plan can access all functions provided by the loaded modules globally.
+
+
+### Main Features
+* Multiple incoming sockets (PCAP, RAW, PF_RING, RX-RING, FILE)
+* Multiple outgoing types (HEP, JSON, CSV)
+* HTTP JSON API for statistics, config changes etc
+* RTCP-XR collector module
+* RTCP output module (output in raw or json format)
+* Capture scenario configuration (pseudo scripting via flex, bison)
+* Call transaction tracking
+* TCP/UDP reassembling and defragmentation.
+* applying and change capture filter on demand
+* LUA scripting (JITLua) (experimental)
+* V7 Javascripting sandbox (experimental)
+* SIPFIX Support (experimental)
+* Websocket encapsulation support

docs/captagent_capture_plans.md

@@ -0,0 +1,52 @@
+### Capture Plans
+Capture Plans are pipelines attached to capture sockets and utilized to define processing logic, 
+using global functions and methods exported by the loaded modules as defined in ```captagent.xml```.
+
+The socket is defined in each capture plan, supporting the following `capture[...]` types:
+
+* pcap
+* tzsp
+* collector
+
+##### Example Configuration Chain
+```socket_pcap``` -> ```{profile}``` -> ```capture_plan```
+
+##### Example Pointer
+/usr/local/etc/captagent/socket_pcap.xml
+```
+        <profile name="socketspcap_sip" description="HEP Socket" enable="true" serial="2014010402">
+	    <settings>
+		<param name="dev" value="eth0"/>
+		<param name="promisc" value="true"/>
+		<param name="reasm" value="false"/>
+		<param name="tcpdefrag" value="false"/>
+		<param name="capture-plan" value="sip_capture_plan.cfg"/>
+		<param name="filter">
+		    <value>portrange 5060-5091</value>
+		</param>
+	    </settings>
+	</profile>
+```
+
+##### Example Capture Plan
+/usr/local/etc/captagent/captureplans/sip_capture_plan.cfg
+```
+capture[pcap] {
+	# here we can check source/destination IP/port, message size
+	if(msg_check("size", "100")) {
+	    #Do parsing
+	    if(parse_sip()) {
+                # Drop unwanted methods
+                if(sip_check("rmethod","OPTIONS") || sip_check("rmethod","NOTIFY")) {
+                  drop;
+                } 
+
+		#Multiple profiles can be defined in transport_hep.xml	
+		if(!send_hep("hepsocket")) {
+		    clog("ERROR", "Error sending HEP!!!!");
+		}
+	    }
+	}
+	drop;
+}
+```

docs/captagent_configuration.md

@@ -0,0 +1,68 @@
+## CaptAgent: Configuration
+
+This section provides guidance to configure Captagent and its core modules on your system.
+
+#### Configuration Logic
+
+Understanding of CaptAgent configuration structure is key - read [this](https://github.com/sipcapture/captagent/wiki/About#sockets-pipes-and-plans) section carefully!
+
+
+-------------
+
+#### Basic Configuration
+
+The following are the default file locations _(unless otherwise specified during configuration)_:
+
+* Configuration: ```/usr/local/captagent/etc```
+* Capture Plans: ```/usr/local/captagent/etc/captureplans```
+* Modules: ```/usr/local/captagent/lib/modules```
+
+
+##### Configuration tree
+The default directory should contains the following using default profiles and plans:
+```
+captagent.xml
+captureplans/
+    sip_capture_plan.cfg
+    rtcp_capture_plan.cfg
+    rtcpxr_capture_plan.cfg
+    tzsp_capture_plan.cfg
+protocol_rtcp.xml
+protocol_sip.xml
+protocol_rtcpxr.xml
+protocol_diameter.xml
+socket_pcap.xml
+socket_tzsp.xml
+socket_collector.xml
+transport_hep.xml
+output_json.xml
+
+```
+
+##### Main Configuration
+To begin, edit and validate the configuration and the module paths in ```/usr/local/etc/captagent/captagent.xml``` to match your actual captagent config/lib path:
+
+```
+<configuration name="core.conf" description="CORE Settings" serial="2014024212">
+            <settings>
+                <param name="debug" value="3"/>
+                <param name="version" value="2"/>
+                <param name="serial" value="2014056501"/>
+                <param name="uuid" value="00781a4a-5b69-11e4-9522-bb79a8fcf0f3"/>
+                <param name="daemon" value="false"/>
+                <param name="syslog" value="false"/>
+                <param name="pid_file" value="/var/run/captagent.pid"/>
+                <param name="module_path" value="/usr/local/lib/captagent/modules"/>
+                <param name="config_path" value="/usr/local/etc/captagent"/>
+                <param name="capture_plans_path" value="/usr/local/etc/captagent/captureplans"/>
+                <param name="backup" value="/usr/local/etc/captagent/backup"/>
+                <param name="chroot" value="/var/lib/captagent"/>
+            </settings>
+        </configuration>
+```
+
+### Next:
+
+* Configure [Socket Modules](https://github.com/sipcapture/captagent/wiki/Socket-Modules)
+* Configure [Transport Modules](https://github.com/sipcapture/captagent/wiki/Transport-Modules)
+* Configure [Capture Plans](https://github.com/sipcapture/captagent/wiki/Capture-Plans)

docs/captagent_init_scripts.md

@@ -0,0 +1,52 @@
+# Captagent Init Scripts
+
+The init.d script can be used to start/stop captagent in a nicer way. 
+
+### Installation & Configuration
+
+A sample of init.d script for captagent is provided at:
+```
+/usr/src/captagent/init/deb/debian/captagent.init
+```
+
+Just copy the init file to ```/etc/init.d/captagent``` and change the permisions:
+
+```
+  cp /usr/src/captagent/init/deb/debian/captagent.init /etc/init.d/captagent
+  chmod 755 /etc/init.d/captagent 
+```
+
+then edit the file updating the $DAEMON and $CFGFILE values:
+
+```
+  DAEMON=/usr/local/captagent/bin/captagent
+  CFGFILE=/usr/local/captagent/etc/captagent/captagent.xml
+```
+
+You need also setup a configuration file in the /etc/default/ directory:
+```
+  cp /usr/src/captagent/init/deb/debian/captagent.default /etc/default/captagent
+```
+
+When using systemd _(we all wish we were not)_ the service file might be required:
+```
+  cp /usr/src/captagent/init/deb/debian/captagent.service /etc/systemd/system/captagent.service
+```
+
+Once installed and before using, edit the default file to reflect your captagent path:
+```
+   RUN_CAPTAGENT=yes
+   CFGFILE=/usr/local/captagent/etc/captagent/captagent.xml
+```
+
+
+### Automatic Startup
+To execute automatically at startup:
+```
+   update-rc.d captagent defaults
+```
+
+To exclude the script from startup:
+```
+   update-rc.d -f  captagent remove
+```

docs/captagent_installtion.md

@@ -0,0 +1,83 @@
+## CaptAgent: Installation
+
+This section provides guidance to download the latest code from our repository and compile it on your system.
+
+
+### Requirements
+
+* Captagent 6.2+ requires ```libuv``` 
+
+    If your system does not provide ```libuv``` and ```libuv-dev```, please install from our 
+[repository](https://github.com/sipcapture/captagent/tree/master/dependency) or compile it from [source]( https://github.com/libuv/libuv/releases)
+
+##### Operating Systems
+###### Debian 10 (buster):
+```
+apt-get install libexpat-dev libpcap-dev libjson-c-dev libtool automake flex bison libgcrypt-dev libuv1-dev libpcre3-dev libfl-dev
+
+```
+###### Debian 9 (stretch):
+```
+apt-get install libexpat1-dev libpcap-dev libjson-c-dev libtool automake flex bison libgcrypt11-dev libuv1-dev libpcre3-dev libfl-dev
+
+```
+###### Debian 8 (jessie):
+```
+apt-get install libexpat-dev libpcap-dev libjson0-dev libtool automake flex bison libuv-dev libgcrypt11-dev libfl-dev
+```
+###### Debian 7 (wheezy):
+```
+wget https://github.com/sipcapture/captagent/raw/master/dependency/debian/wheezy/libuv_1.8.0-2_amd64.deb
+dpkg -i libuv_1.8.0-2_amd64.deb
+
+apt-get install libexpat-dev libpcap-dev libjson0-dev libtool automake flex bison 
+
+```
+
+###### CentOS 7:
+```
+yum -y install json-c-devel expat-devel libpcap-devel flex-devel automake libtool bison libuv-devel flex
+```
+
+###### CentOS 6:
+```
+rpm -i https://github.com/sipcapture/captagent/raw/master/dependency/centos/6/libuv-1.8.0-1.el6.x86_64.rpm
+rpm -i https://github.com/sipcapture/captagent/raw/master/dependency/centos/6/libuv-devel-1.8.0-1.el6.x86_64.rpm
+yum -y install json-c-devel expat-devel libpcap-devel pcre-devel flex-devel automake libtool bison flex
+```
+
+
+
+### Clone & Compile
+```
+  cd /usr/src
+  git clone https://github.com/sipcapture/captagent.git captagent
+  cd captagent
+  ./build.sh
+  ./configure
+  make && make install
+```
+
+#### Build Options
+| Name        | Configure Flag       | Libraries           |
+|---          |---                   |---                  |
+| HEP Compression | --enable-compression |                     |
+| IPv6 Support  | --enable-ipv6        |                     |
+| PCRE Support  | --enable-pcre        | libpcre             |
+| SSL Support   | --enable-ssl         | openssl             |
+| TLS Support   | --enable-tls         | libgcrypt20 openssl |
+| MySQL Support | --enable-mysql       | libmysqlclient      |
+| Redis Support | --enable-redis       | libhiredis          |
+
+--------------
+
+#### TLS Support (experimental)
+
+To compile and enable TLS decryption features, please check the dedicated Wiki page.
+
+--------------
+
+
+Congratulations! You just installed your first basic instance of CaptAgent 6!
+
+#### Next: [Configure CaptAgent 6](https://github.com/sipcapture/captagent/wiki/Configuration)

docs/captagent_nat.md

@@ -0,0 +1,17 @@
+## NAT
+When the SDP details are reporting masqueraded or private IP ranges, the ```nat-mode``` switch can be enabled to force Captagent to use the received address for matching media sessions. Settings enabled by module ```database_hash``` should be applied in file ```database_hash.xml``` :
+
+```
+<?xml version="1.0"?>
+<document type="captagent_module/xml">
+    <module name="database_hash" description="HASH Database" serial="2014010402">
+	<profile name="database_hash" description="HASH RTCP" enable="true" serial="2014010402">
+	    <settings>
+		<param name="timer-expire" value="80"/>
+		 <!-- enable if you have nat devices and masqaraded ip -->
+		 <param name="nat-mode" value="true"/>		    
+	    </settings>
+	</profile>
+    </module>
+</document>
+```