Major documentation update

- API moved to its own folder
- Spec moved to its own folder
- README's added to several subfolders (e.g. bindings).
- Started a getting started guide.
- Lots of edits
- Website layout edits

README.md

@@ -1,24 +1,27 @@
-[__Open Ephys++__](https://jonnew.github.io/open-ephys-pcie/) is hardware, firmware, communication protocols,
-specifications, and APIs for serialized, very-high channel count, closed-loop
-electrophysiology. It is an evolution of the hardware and software introduced
-in [open ephys project](http://www.open-ephys.org/) and involves many of the
-same developers. The firmware and API are general purpose -- they can be used
-to acquire from and control custom headstages with arbitrary arrangements of
-sensors and actuators (e.g. cameras, bioamplifier chips, LED drivers, etc.) and
-are not limited to the hardware in this repository.
-
-This project is currently maintained by [jonnew](https://github.com/jonnew).
+[__Open Ephys++__](https://jonnew.github.io/open-ephys-pcie/) is hardware,
+firmware, communication protocols, specifications, and APIs for serialized,
+very-high channel count, closed-loop electrophysiology. It is an evolution of
+the hardware and software introduced in [Open Ephys
+project](http://www.open-ephys.org/) and involves many of the same developers.
+The firmware and API are general purpose -- they can be used to acquire from
+and control custom headstages with arbitrary arrangements of sensors and
+actuators (e.g. cameras, bioamplifier chips, LED drivers, etc.) and are not
+limited to the hardware in this repository.
+
+[![Join the chat at https://gitter.im/open-ephys-pcie/Lobby](https://badges.gitter.im/open-ephys-pcie/Lobby.svg)](https://gitter.im/open-ephys-pcie/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
 __Citing this work__: TODO
 
 ## Features
+- Formal specifications: serialization protocols, host communication protocols,
+  device drivers, and host API
 - Firmware and API permit acquisition and control of arbitrary arrangements of
   sensors and actuators:
 
-    - Miniscopes
     - Headstages
+    - Miniscopes
     - Photometry systems
-    - Etc. 
+    - Etc.
 
 - Submillisecond round-trip communication from brain, through host PC's main
   memory, and back again.
@@ -28,24 +31,52 @@ __Citing this work__: TODO
     - Optogenetic stimulation
     - Electrical stimulation
     - 3D-pose measurement
-    - Data, user control, and power via a tiny coaxial cable 
-    - Wireless communication 
+    - Data, user control, and power via a tiny coaxial cable
+    - Wireless communication
 
-- Formal specifications: serialization protocols, host communication protocols,
-  and host API
 - Low-level API implementation
 - High-level API language bindings and existing integration with [Open Ephys
   GUI](http://www.open-ephys.org/gui/) and [Bonsai](http://bonsai-rx.org/).
 - Quality documentation and easy routes to purchasing assembled devices.
 
 ## Repository Contents and Licensing
 Each top level directory of this repository corresponds to a distinct system
-module. These can be hardware components (e.g. `headstage-64`), or
-firmware/software/APIs (e.g. `oepcie`). __Each subdirectory may have distinct
-contributors and/or licenses__. Please refer to the README file within each
-directory for further information on usage, licensing, etc.
+module. These can be specifications (e.g. `spec`), hardware components (e.g.
+`headstage-64`), or programming interfaces (e.g. `api`). __Each subdirectory
+may have distinct contributors and/or licenses__. Please refer to the README
+file within each directory for further information on usage, licensing, etc.
+
+## Specification
+The [Open Ephys++ Specification](spec/README.md) formally specifies data
+serialization, host/PC communication, firmware blocks, device drivers and
+programming interfaces for this project. All firmware, software, and hardware
+artifacts in this repository implementations of this specification.
+Therefor, third party implementations that maintain compatibility with the spec
+will interoperate with the software and hardware within this project.
+Seriously, do a better job than us, we will be grateful! Also, if you have
+concerns with the spec, please get in touch. We want this to be
+used and applicable in a variety of circumstances.
+
+## Software
+
+### [Programming Interface](api/README.md)
+High-performance, host-side programming interfaces for integration with
+existing software and the creation of high level language bindings. 
+
+### [Bonsai Package]()
+- TODO: Myget distribution
+
+### [Open Ephys Plugin]()
+- TODO: Integration into open-ephys master
+
+## Firmware [WIP]
+
+Binary files for headstage and host FPGAs are available [here](TODO). Firmware
+source code is currently available under controlled release. Contact the
+maintainer for more information.
 
 ## Hardware
+
 ### [eib-64](eib-64/README.md)
 64 Channel electrode interface board. Designed for small rodent tetrode
 electrophysiology. Works with [headstage-64](./headstage-64/README.md).
@@ -94,19 +125,9 @@ Provides LEDs and simulated electrical loads for optical and electrical
 stimulation.
 
 ### [headstage-programmer](headstage-programmer/README.md)
-JTAG breakout for the [Intel USB Blaster 2](https://www.digikey.com/short/qqw7hm) 
+JTAG breakout for the [Intel USB Blaster 2](https://www.digikey.com/short/qqw7hm)
 used to program the headstages' MAX10 FPGA.
 
 ### pcie-analog-io [WIP]
 General purpose analog IO expansion board which communicates with the host
 computer via the  sits next to [pcie-host]() board.
-
-## Software
-### [liboepcie](oepcie/README.md)
-Host API specification and implementation.
-
-## Firmware [WIP]
-Binary files for headstage and host FPGAs are available [here](TODO) Firmware
-source code is currently available under controlled release. Contact the
-maintainer for more information.
-

api/README.md

@@ -0,0 +1,18 @@
+# Programming Interfaces
+The host-side APIs allow high-performance software interfaces and the creation
+of high level language bindings. API implementations based upon (1d):
+
+- [`liboepcie`](liboepcie) is an ANSI-C implementation of the [Open Ephys++ API
+  Specificaiton](../spec/README.md). It contains functions for configuring and
+  streaming data to and from hardware.
+- [`cppoepcie`](cppoepcie) C++14 bindings for `liboepcie`.
+- [`clroepcie`](clroepcie) CLR/.NET bindings for `liboepcie`.
+
+Minimal example host programs for each of these libraries can be found in the
+\*-test folder within each library directory. If you are interested in writing
+a binding or integrating and existing API with your software, feel free to get
+in touch so we can help you out.
+
+## Using Xillybus with `liboepcie`
+Instructions for using [Xillybus](http://xillybus.com/) as a communication
+backend can be found [here](xillybus-backend.md).

oepcie/clroepcie/README.md → api/clroepcie/README.md

@@ -5,10 +5,10 @@ CLR/.NET bindings for [`liboepcie`](../liboepcie/README.md).
 
 ### Visual Studio (Windows)
 1. Open the `clrpepcie.sln` solution in visual studio. 
-2. Running the solution will combine the library and test program, and then run
-   the test program
+2. "Running" the solution will compile the library and test program, and then
+   run the test program
 
-__Notes__
+__Notes (aka please help...)__
 
 - I removed the Any CPU build option because I don't understand how to use it
   properly. Instead, I build two times for each x64 and x86
@@ -45,3 +45,5 @@ $ cd clroepcie-test
 $ make
 ```
 
+## License
+[MIT](https://en.wikipedia.org/wiki/MIT_License)

oepcie/cppoepcie/README.md → api/cppoepcie/README.md

@@ -45,3 +45,6 @@ $ env CPUPROFILE=/tmp/host.prof ./host /tmp/xillybus_cmd_mem_32 /tmp/xillybus_as
 ```
 $ pprof ./host /tmp/host.prof
 ```
+
+## License
+[MIT](https://en.wikipedia.org/wiki/MIT_License)

oepcie/README.md → api/liboepcie/README.md

@@ -1,26 +1,56 @@
-# Open Ephys++ API
+# `liboepcie`
+ANSI C implementation of the [Open Ephys++ API Specification](../spec.pdf).
 
-1. The [Open-ephys++ API specification](spec.pdf) which defines the
-   requirements of a host API that communicates with hardware in this project
+## Build
+For build options, look at the [Makefile](Makefile). To build and install:
+```
+$ make <options>
+$ make install PREFIX=/path/to/install
+```
+to place headers in whatever path is specified by PREFIX. PREFIX defaults to
+`/usr/lib/include`. You can uninstall (delete headers and libraries) via
+```
+$ make uninstall PREFIX=/path/to/uninstall
+```
 
-2. API implementations based upon this specification:
+## Test Programs
+The [liboepcie-test](liboepcie-test) directory contains minimal working programs that use this library.
 
-    - [liboepcie](liboepcie) is an ANSI-C open-ephys++ API implementation.
-    It contains functions for configuring and stream data to and from hardware.
-    - [cppoepcie](cppoepcie) C++14 bindings for liboepcie.
-    - [clroepcie](clroepcie) CLR/.NET bindings for liboepcie.
+1. `firmware` : Emulate hardware. Stream fake data over UNIX pipes (Linux only)
+1. `host` : Basic data acquisition loop. Communicate with `firmware` or actual
+   hardware.
 
-    Example host programs for each of these libraries can be found in the
-    \*-test folder within each library directory.
+## Performance testing
+1. Install google perftools:
+```
+$ sudo apt-get install google-perftools
+```
+2. Link test programs against the CPU profiler:
+```
+$ cd liboepcie-test
+$ make profile
+```
+5. Run the `firmware` program to serve fake data. Provide a numerical argument
+   specifying the number of fake frames to produce. It will tell you how long
+   it takes `host` to sink all these frames. This is host processing time +
+   UNIX pipe read/write.
+```
+$ cd bin
+$ ./firmware 10e6
+```
+4. Run the `host` program while dumping profile info:
+```
+$ env CPUPROFILE=/tmp/host.prof ./host /tmp/xillybus_cmd_mem_32 /tmp/xillybus_async_read_8 /tmp/xillybus_data_read_32
+```
+5. Examine output
+```
+$ pprof ./host /tmp/host.prof
+```
 
 ## License
 [MIT](https://en.wikipedia.org/wiki/MIT_License)
 
-## Using Xillybus with `liboepcie`
-Instructions for using [Xillybus](http://xillybus.com/) as a communication
-backend can be found [here](xillybus-backend.md).
-
-## `liboepcie`: Detailed Description
+## Detailed Description
 
 `liboepcie` is a C library that implements the [Open Ephys++ API
 Specification](spec.pdf). It is written in ANSI-C to facilitate

spec/README.md

@@ -0,0 +1,65 @@
+# Open Ephys++ Specification [WIP]
+
+The goals of this project are bigger than simply producing headstage and
+acquisition hardware for our own needs. We have found that proper headstage
+serialization and host to PC communication are some of the trickiest parts of
+the development process. Therefore developers often resort to brittle solutions
+that meet their needs but are hard to inter-operate (e.g. NI DAQs).
+Alternatively, they may turn to ready-made options with poor performance (e.g.
+Opal Kelly development boards). Therefore, these interfaces become the limiting
+hardware for both terms of integration (e.g. using disparate hardware in the
+same rig)  as well as closed-loop performance (e.g. loop delay).
+
+To address this, we have created a set of __general purpose__ communication
+protocols, device drivers, and programming interfaces to support __arbitrary
+mixtures of hardware__. These designs are formally described by the [Open Ephys++
+Specificaiton](spec.pdf), which was developed with lots of community input.
+
+The hope is that this acquisition system will get the boring part out of the
+way: it serves as a high performance, open-source platform upon which cool
+neuroscience tools can by built. We also hope that the use of a common
+protocols will allow easy integration of disparate hardware to develop rich
+feedback control loops for neuroscience projects. If you are interested in
+developing against this specification (e.g. for your miniscope, headstage,
+position tracker, 2P microscope, etc), we would love to [hear from
+you](https://gitter.im/open-ephys-pcie/Lobby) and talk about the best way to
+proceed. If you prefer a private conversation, please contact us via [Open
+Ephys](http://www.open-ephys.org/contact/).
+
+## Contents
+The Open [Open Ephys++ Specificaiton](spec.pdf) formally defines the
+requirements of the following components:
+
+1. Headstage to host serialization protocol specification
+1. Host to PC communication protocol specification
+1. Host device driver specification [WIP]
+1. Host programming interface specification
+
+In addition to these specifications, we have created the following flagship
+implementations which form data acquisition system at the base of our [next
+generation headstages](../headstage-64). These implementations are modular --
+you can use them form your project!  No need to reinvent the wheel:
+
+
+- Firmware implementations based on (1,2).
+    - TODO
+
+- Device driver implementation based on (3)
+    - TODO
+- API implementations based upon (4):
+    - [liboepcie](../api/liboepcie) is an ANSI-C open-ephys++ API implementation.
+    It contains functions for configuring and stream data to and from hardware.
+    - [cppoepcie](../api/cppoepcie) C++14 bindings for liboepcie.
+    - [clroepcie](../api/clroepcie) CLR/.NET bindings for liboepcie.
+
+If you think these implementations suck, feel free to use the spec to develop
+for yourself. Your product will be automatically compatible with Open Ephys
+tools!
+
+## Feedback
+We know we did not do a perfect job with this specification. [Please tell us
+why!](https://gitter.im/open-ephys-pcie/Lobby) -- it will help this project be
+generally useful for the neuroscience research.
+
+## License
+[MIT](https://en.wikipedia.org/wiki/MIT_License)

oepcie/spec.md → spec/spec.md

@@ -1,7 +1,7 @@
 ---
 title: |
     Open Ephys++ Communication Protocol and API Specification \
-    Version 0.0
+    Version 0.1
 author: Jonathan P. Newman, Wilson Lab, MIT
 institution:
 date: \today{}
@@ -39,6 +39,10 @@ abstract: |
 - Low level: aimed at the creation of language bindings and
   application-specific libraries
 
+\newpage
+# Headstage Serialization Protocol {#ser-protocol}
+TODO
+
 \newpage
 # FPGA/Host PC communication {#comm-protocol}
 Communication between the acquisition board firmware and API shall occur
@@ -238,6 +242,11 @@ input channel.
 
 \newpage
 
+\newpage
+# Device Driver Specification  {#driver-protocol}
+TODO
+
+\newpage
 # Required API Types and Behavior {#api-spec}
 In the following sections we define required API datatypes and how they are
 used by the API to communicate with hardware. An implementation of this API,