ARUHA-906: Rewrite readme (#718)

* ARUHA-906: Rewrite readme

* ARUHA-906: Rewrite readme. small fixes

* ARUHA-906: Rewrite readme. test SVG image inline

* ARUHA-906: Rewrite readme. use png instead of svg

* ARUHA-906: Rewrite readme. use png instead of svg

* ARUHA-906: Rewrite readme. cosmetic changes

* ARUHA-906: Fix urls with { and }

* ARUHA-906: Remove outdated file

* ARUHA-906: reformat readme

* ARUHA-906: Fix headers

* ARUHA-906: Rename CONDUCT to CODE_OF_CONDUCT according to GitHub

* ARUHA-906: Link to the website

* ARUHA-906: Fix some links

* ARUHA-906: Post review fixes

docs/README.md

@@ -64,7 +64,6 @@ Then you can see the result in the browser on  `http://localhost:4000`
 Every changes in the source will automatically rebuild documentation 
 so you only need to refresh the browser page.
 
-
 ## Acknowledgments
 
 The template based on [swaggyll](https://github.com/hauptrolle/swaggyll) but heavily modified.

docs/_documentation/architecture_timelines.md

@@ -3,14 +3,14 @@ title: Timelines
 position: 101
 ---
 
-Timelines
----------
+## Timelines
+
 
 This document covers Timelines internals. It's meant to explain how
 timelines work, to help you understand the code and what each part of
 it contributes to the overall picture.
 
-# Fake timeline: your first timeline
+### Fake timeline: your first timeline
 
 Before timelines, Nakadi would connect to a single Kafka cluster,
 which used to be specified in the application yaml file. This was a
@@ -32,66 +32,66 @@ migration process is as follow:
    storage and different topic is going to be created by Nakadi for
    this event type.
 
-# Timeline creation
+### Timeline creation
 
 Timeline creation is coordinated through a series of locks and
 barriers using Zookeeper. Following we depict an example of what the
 ZK datastructure looks like at each step.
 
-## Initial state
+#### Initial state
 
 Every time a Nakadi application is launched, it tries to create the
 following ZK structure:
 
-```
-- timelines
-  + - lock                    lock for timeline versions synchronization
-  + - version: {version}      monotonically incremented long value (version of timelines configuration)
-  + - locked_et
-  + - nodes                   nakadi nodes
-    + - {node1}: {version}    Each nakadi node exposes the version used on this node
-    + - {node2}: {version}
+```yaml
+timelines:
+  lock: -                    lock for timeline versions synchronization
+  version: {version}      monotonically incremented long value (version of timelines configuration)
+  locked_et: -
+  nodes:                    nakadi nodes
+    node1: {version}    Each nakadi node exposes the version used on this node
+    node2: {version}
 ```
 
 In order to not override the initial structure, due to concurrency,
 each instance needs to take the lock `/nakadi/timelines/lock` before
 executing.
 
-## Start timeline creation for et_1
+#### Start timeline creation for et_1
 
 When a new timeline creation is initiated, the first step is to
 acquire a lock to update timelines for et_1 by creating an ephemeral
 node at `/timelines/locked_et/et_1`.
 
-```
-- timelines
-  + - lock
-  + - version: 0
-  + - locked_et:
-    + - et_1
-  + - nodes
-    + - node1: 0
-    + - node2: 0
+```yaml
+timelines:
+  lock: -
+  version: 0
+  locked_et:
+    et_1: -
+  nodes:
+    node1: 0
+    node2: 0
 ```
 
-## Notify all Nakadi nodes about change: the version barrier
+#### Notify all Nakadi nodes about change: the version barrier
 
 Next, the instance coordinating the timeline creation bumps the
 version node, which all Nakadi instances are listening to changes, so
 they are notified when something changes.
 
-```
-- timelines
-  + - lock
-  + - version: 1       # this is incremented by 1
-  + - locked_et:
-    + - et_1
-  + - nodes
-    + - node1: 0
-    + - node2: 0
+```yaml
+timelines:
+  lock: -
+  version: 1       # this is incremented by 1
+  locked_et:
+    et_1: -
+  nodes:
+    node1: 0
+    node2: 0
 ```
 
-## Wait for all nodes to react to the new version
+#### Wait for all nodes to react to the new version
 
 Each Nakadi instance watches the value of the
 `/nakadi/timelines/version/` node. When it changes, each instance
@@ -102,18 +102,19 @@ Once each instance has updated its local list of locked event types,
 it bumps its own version, to let the timeline creator initiator know
 that it can proceed.
 
-```
-- timelines
-  + - lock
-  + - version: 1
-  + - locked_et:
-    + - et_1
-  + - nodes
-    + - node1: 1       # each instance updates its own version
-    + - node2: 1
+
+```yaml
+timelines:
+  lock: -
+  version: 1 
+  locked_et:
+     et_1: -
+  nodes:
+    node1: 1       # each instance updates its own version
+    node2: 1
 ```
 
-## Proceed with timeline creation
+#### Proceed with timeline creation
 
 Once all instances reacted, the creation proceeds with the initiator
 inserting the necessary database entries in the timelines table, and
@@ -122,40 +123,43 @@ storage. It also creates a topic in the new storage. Be aware that if
 a timeline partition has never been used, the offset stored is -1. If
 it has a single event, the offset is zero and so on.
 
-## Remove lock and notify all instances again
+#### Remove lock and notify all instances again
 
 Following the same logic for initiating the creation of a timeline,
 locks are deleted and version is bumped. All Nakadi instances react by
 removing their local locks and switching timeline if necessary.
 
-```
-- timelines
-  + - lock
-  + - version: 2
-  + - locked_et:
-  + - nodes
-    + - node1: 1
-    + - node2: 1
+```yaml
+timelines:
+  lock: -
+  version: 2 
+  locked_et:     
+  nodes:
+    node1: 1
+    node2: 1
 ```
 
+
 After every instance reacted, it should look like:
 
-```
-- timelines
-  + - lock
-  + - version: 2
-  + - locked_et:
-  + - nodes
-    + - node1: 2
-    + - node2: 2
+```yaml
+timelines:
+  lock: -
+  version: 2 
+  locked_et:
+  nodes:
+    node1: 2       # each instance updates its own version
+    node2: 2
 ```
 
-## Done
+#### Done
 
 All done here. A new timeline has been created successfully. All
 operations are logged so in case you need to debug things, just take a
 look at INFO level logs.
 
-# Cursors
+<!---
+## Cursors
 
 TODO: Describe cursors.
+-->

docs/_documentation/developing.md

@@ -9,7 +9,6 @@ position: 13
 
 Nakadi is hosted on Github - [zalando/nakadi](https://github.com/zalando/nakadi/) and you can clone or fork it from there. 
 
-<a name="dev-building"></a>
 ## Building
 
 The project is built with [Gradle](https://gradle.org). 
@@ -23,7 +22,6 @@ The gradle setup is fairly standard, the main dev tasks are:
 
 Pull requests and master are built using Travis CI and you can see the build history [here](https://travis-ci.org/zalando/nakadi).
 
-<a name="dev-running-tests"></a>
 ## Running Tests
 
 There are a few build commands for testing -
@@ -32,7 +30,6 @@ There are a few build commands for testing -
 - `./gradlew acceptanceTest`: will run the acceptance tests
 - `./gradlew fullAcceptanceTest`: will run the ATs in the context of Docker
 
-<a name="dev-running-containers"></a>
 ## Running Containers
 
 There are a few build commands for running Docker -
@@ -41,7 +38,6 @@ There are a few build commands for running Docker -
 - `./gradlew stopAndRemoveDockerContainer`: shutdown the docker processes
 - `./gradlew startStoragesInDocker`: start the storage container that runs Kafka and PostgreSQL. This is handy for running Nakadi directly or in your IDE.
 
-<a name="dev-ide"></a>
 ## IDE Setup
 
 For working with an IDE, the `./gradlew eclipse` IDE task is available and you'll be able to import the `build.gradle` into Intellij IDEA directly.

docs/_documentation/faq.md

@@ -7,22 +7,22 @@ position: 14
 
 ## Table of Contents
 
-- [How long will events be persisted for?](#how-long-will-events-be-persisted-for)
-- [How do I define how long will events be persisted for?](#how-do-i-define-how-long-will-events-be-persisted-for)
-- [How many partitions will an event type be given?](#how-many-partitions-will-an-event-type-be-given)
-- [How do I configure the number of partitions?](#how-do-i-configure-the-number-of-partitions)
-- [Which partitioning strategy should I use?](#which-partitioning-strategy-should-i-use)
-- [How can I keep track of a position in a stream?](#how-can-i-keep-track-of-a-position-in-a-stream)
-- [What's an effective schema?](#whats-an-effective-schema)
-- [Nakadi isn't validating metadata and/or event identifiers, what's going on?](#nakadi-isnt-validating-metadata-andor-event-identifiers-whats-going-on)
-- [What clients are available?](#what-clients-are-available)
-- [How do I disable OAuth for local development?](#how-do-i-disable-oauth-for-local-development)
-- [I want to send arbitrary JSON, how do I avoid defining a JSON Schema?](#i-want-to-send-arbitrary-json-how-do-i-avoid-defining-a-json-schema)
-- [Can I post something other than JSON as an event?](#can-i-post-something-other-than-json-as-an-event)
-- [I get the message "Is the docker daemon running on this host?" - Help!](#i-get-the-message-is-the-docker-daemon-running-on-this-host---help)
-- [What's the reason for newest available offset being bigger than oldest offset?](#whats-the-reason-for-newest-available-offset-being-bigger-than-oldest-offset)
-- [Does Nakadi support compression?](#does-nakadi-support-compression)
-- [How do I contribute to the project?](#how-do-i-contribute-to-the-project)
+- [How long will events be persisted for?](#how-long-will-events-be-persisted-for-)
+- [How do I define how long will events be persisted for?](#how-do-i-define-how-long-will-events-be-persisted-for-)
+- [How many partitions will an event type be given?](#how-many-partitions-will-an-event-type-be-given-)
+- [How do I configure the number of partitions?](#how-do-i-configure-the-number-of-partitions-)
+- [Which partitioning strategy should I use?](#which-partitioning-strategy-should-i-use-)
+- [How can I keep track of a position in a stream?](#how-can-i-keep-track-of-a-position-in-a-stream-)
+- [What's an effective schema?](#whats-an-effective-schema-)
+- [Nakadi isn't validating metadata and/or event identifiers, what's going on?](#nakadi-isnt-validating-metadata-andor-event-identifiers-whats-going-on-)
+- [What clients are available?](#what-clients-are-available-)
+- [How do I disable OAuth for local development?](#how-do-i-disable-oauth-for-local-development-)
+- [I want to send arbitrary JSON, how do I avoid defining a JSON Schema?](#i-want-to-send-arbitrary-json-how-do-i-avoid-defining-a-json-schema-)
+- [Can I post something other than JSON as an event?](#can-i-post-something-other-than-json-as-an-event-)
+- [I get the message "Is the docker daemon running on this host?" - Help!](#i-get-the-message--is-the-docker-daemon-running-on-this-host-----help-)
+- [What's the reason for newest available offset being bigger than oldest offset?](#whats-the-reason-for-newest-available-offset-being-bigger-than-oldest-offset-)
+- [Does Nakadi support compression?](#does-nakadi-support-compression-)
+- [How do I contribute to the project?](#how-do-i-contribute-to-the-project-)
 
 ----
 
@@ -122,4 +122,4 @@ The server will accept gzip encoded events when posted. On the consumer side, if
 
 #### How do I contribute to the project?
 
-Nakadi accepts contributions from the open-source community. Please see the [project issue tracker](https://github.com/zalando/nakadi/issues) for things to work on. Before making a contribution, please let us know by posting a comment to the relevant issue. And if you would like to propose a new feature, do start a new issue explaining the feature you’d like to contribute.
+Nakadi accepts contributions from the open-source community. Please see [CONTRIBUTE.md](https://github.com/zalando/nakadi/blob/master/CONTRIBUTING.md).

docs/_documentation/getting-started.md

@@ -39,7 +39,7 @@ To stop the running Nakadi:
 ### Notes
 
 If you're having trouble getting started, you might find an answer in the
-[Frequently Asked Questions (FAQ)](#faq) section of the documentation.
+[Frequently Asked Questions (FAQ)](#f-a-q) section of the documentation.
 
 #### Ports
 
@@ -53,10 +53,11 @@ Some ports need to be available to run the service:
 They allow the services to communicate with each other and should not be used
 by other applications.
 
-#### Mac OS and Docker
+<a name="macos"></a>
+### Mac OS Docker Settings
 
 Since Docker for Mac OS runs inside Virtual Box, you will  want to expose
-some ports first to allow Nakadi to access its dependencies -
+some ports first to allow Nakadi to access its dependencies:
 
 ```sh
 docker-machine ssh default \
@@ -67,6 +68,17 @@ docker-machine ssh default \
 ```
 
 Alternatively you can set up port forwarding on the "default" machine through
-its network settings in the VirtualBox UI, which look like this -
+its network settings in the VirtualBox UI. 
 
 ![vbox](./img/vbox.png)
+
+If you get the message "Is the
+docker daemon running on this host?" but you know Docker and VirtualBox are
+running, you might want to run this command:
+
+```sh
+eval "$(docker-machine env default)"
+```
+
+**Note:** Docker for Mac OS (previously in beta) version 1.12 (1.12.0 or 1.12.1) currently is not supported due to the [bug](https://github.com/docker/docker/issues/22753#issuecomment-242711639) in networking host configuration.
+