Documentation update

README.md

@@ -1,3 +1,150 @@
-# DeepMind Lab
+# <img src="docs/logo.png" alt="DeepMind Lab">
 
-A 3D learning environment.
+*DeepMind Lab* is a 3D learning environment based on id Software's
+[Quake III Arena](https://github.com/id-Software/Quake-III-Arena) via
+[ioquake3](https://github.com/ioquake/ioq3) and
+[other open source software](#upstream-sources).
+
+<div align="center">
+  <a href="https://www.youtube.com/watch?v=M40rN7afngY"
+     target="_blank">
+    <img src="http://img.youtube.com/vi/M40rN7afngY/0.jpg"
+         alt="DeepMind Lab - Nav Maze Level 1"
+         width="240" height="180" border="10" />
+  </a>
+  <a href="https://www.youtube.com/watch?v=gC_e8AHzvOw"
+     target="_blank">
+    <img src="http://img.youtube.com/vi/gC_e8AHzvOw/0.jpg"
+         alt="DeepMind Lab - Stairway to Melon Level"
+         width="240" height="180" border="10" />
+  </a>
+  <a href="https://www.youtube.com/watch?v=7syZ42HWhHE"
+     target="_blank">
+    <img src="http://img.youtube.com/vi/7syZ42HWhHE/0.jpg"
+         alt="DeepMind Lab - Laser Tag Space Bounce Level (Hard)"
+         width="240" height="180" border="10" />
+  </a>
+  <br /><br />
+</div>
+
+*DeepMind Lab* provides a suite of challenging 3D navigation and puzzle-solving
+tasks for learning agents. Its primary purpose is to act as a testbed for
+research in artificial intelligence, especially deep reinforcement learning.
+
+## About
+
+Disclaimer: This is not an official Google product.
+
+If you use *DeepMind Lab* in your research and would like to cite
+the *DeepMind Lab* environment, we suggest you cite
+the [DeepMind Lab paper](https://deepmind.com/documents/deepmind_lab.pdf).
+
+You can reach us at [[email protected]](mailto:[email protected]).
+
+## Getting started on Linux
+
+* Get [Bazel from bazel.io](http://bazel.io/docs/install.html).
+
+* Clone DeepMind Lab, e.g. by running
+
+```shell
+$ git clone https://github.com/deepmind/lab
+$ cd lab
+```
+
+* For a live example of a random agent, run
+
+```shell
+lab$ bazel run :random_agent --define headless=false -- \
+               --length=10000 --width=640 --height=480
+```
+
+Here is some [more detailed build documentation](docs/build.md),
+including how to install dependencies if you don't have them.
+
+### Play as a human
+
+To test the game using human input controls, run
+
+```shell
+lab$ bazel run :game -- --level_script tests/demo_map
+```
+
+### Train an agent
+
+*DeepMind Lab* ships with an example random agent in
+[`python/random_agent.py`](python/random_agent.py)
+which can be used as a starting point for implementing a learning agent. To let
+this agent interact with DeepMind Lab for training, run
+
+```shell
+lab$ bazel run :random_agent
+```
+
+The Python API for the agent-environment interaction is described
+in [docs/python_api.md](docs/python_api.md).
+
+*DeepMind Lab* ships with different levels implementing different tasks. These
+tasks can be configured using Lua scripts,
+as described in [docs/lua_api.md](docs/lua_api.md).
+
+-----------------
+
+## Upstream sources
+
+*DeepMind Lab* is built from the *ioquake3* game engine, and it uses the tools
+*q3map2* and *bspc* for map creation. Bug fixes and cleanups that originate
+with those projects are best fixed upstream and then merged into *DeepMind Lab*.
+
+* *bspc* is taken from [github.com/TTimo/bspc](https://github.com/TTimo/bspc),
+  revision e6f90a2dc02916aa2298da6ace70a8333b3f2405. There are virtually no
+  local modifications, although we integrate this code with the main ioq3 code
+  and do not use their copy in the `deps` directory. We expect this code to be
+  stable.
+
+* *q3map2* is taken from
+  [github.com/TTimo/GtkRadiant](https://github.com/TTimo/GtkRadiant),
+  revision 8557f1820f8e0c7cef9d52a78b2847fa401a4a95. A few minor local
+  modifications add synchronization and use C99 constructs to replace
+  formerly non-portable or undefined behaviour. We also expect this code to be
+  stable.
+
+* *ioquake3* is taken from
+  [github.com/ioquake/ioq3](https://github.com/ioquake/ioq3),
+  revision 1c1e1f61f180596c925a4ac0eddba4806d1369cd. The code contains extensive
+  modifications and additions. We aim to merge upstream changes occasionally.
+
+We are very grateful to the maintainers of these repositories for all their hard
+work on maintaining high-quality code bases.
+
+## External dependencies, prerequisites and porting notes
+
+*DeepMind Lab* currently ships as source code only. It depends on a few external
+software libraries, which we ship in several different ways:
+
+ * The `zlib`, `glib`, `libxml2`, `jpeg` and `png` libraries are referenced as
+   external Bazel sources, and Bazel BUILD files are provided. The dependent
+   code itself should be fairly portable, but the BUILD rules we ship are
+   specific to Linux on x86. To build on a different platform you will most
+   likely have to edit those BUILD files.
+
+ * Message digest algorithms are included in this package (in
+   [`//third_party/md`](third_party/md)), taken from the reference
+   implementations of their respective RFCs. A "generic reinforcement learning
+   API" is included in [`//third_party/rl_api`](third_party/rl_api), which has
+   also been created by the *DeepMind Lab* authors. This code is portable.
+
+ * Several additional libraries are required but are not shipped in any form;
+   they must be present on your system:
+   * SDL 2
+   * Lua 5.1 (later versions might work, too)
+   * gettext (required by `glib`)
+   * OpenGL: a hardware driver and library are needed for hardware-accelerated
+     human play, and OSMesa is required for the software-rendering, headless
+     library that machine learning agents will want to use.
+   * Python 2.7 (other versions might work, too)
+
+The build rules are using a few compiler settings that are specific to GCC. If
+some flags are not recognized by your compiler (typically those would be
+specific warning suppressions), you may have to edit those flags. The warnings
+should be noisy but harmless.

docs/build.md

@@ -0,0 +1,149 @@
+(Switch to: [Lua](lua_api.md) · [Python](python_api.md) ·
+ [Level Generation](level_generation.md) ·
+ [Tensor](tensor.md) · [Text Levels](text_level.md) ·
+ Build ·
+ [Known Issues](issues.md))
+
+# How to build *DeepMind Lab*
+
+*DeepMind Lab* uses [Bazel](https://www.bazel.io/) as its build system. Its main
+`BUILD` file defines a number of *build targets* and their dependencies. The
+build rules should work out of the box on Debian (Jessie or newer) and Ubuntu
+(version 14.04 or newer), provided the required packages are installed.
+*DeepMind Lab* also builds on other Linux systems, but some changes to the build
+files might be required, see below.
+
+*DeepMind Lab* is written in C99 and C++11, and you will need a sufficiently
+modern compiler. GCC 4.8 should suffice.
+
+### Step-by-step instructions for Debian or Ubuntu
+
+Tested on Debian 8.6 (Jessie) and Ubuntu 14.04 (Trusty) and newer.
+
+1. Install Bazel by adding a custom APT repository, as described
+   [on the Bazel homepage](http://bazel.io/docs/install.html#ubuntu) or using
+   an [installer](https://github.com/bazelbuild/bazel/releases).
+   This should also install GCC and zip.
+
+2. Install *DeepMind Lab*'s dependencies:
+
+   ```shell
+   $ sudo apt-get install lua5.1 liblua5.1-0-dev libffi-dev gettext \
+       freeglut3-dev libsdl2-dev libosmesa6-dev python-dev python-numpy
+   ```
+
+3. [Clone or download *DeepMind Lab*](https://github.com/deepmind/lab).
+
+4. Build *DeepMind Lab* and run a random agent:
+
+   ```shell
+   $ cd lab
+   # Build the Python interface to DeepMind Lab with OpenGL
+   lab$ bazel build :deepmind_lab.so --define headless=glx
+   # Build and run the tests for it
+   lab$ bazel run :python_module_test --define headless=glx
+   # Rebuild the Python interface in non-headless mode and run a random agent
+   lab$ bazel run :random_agent --define headless=false
+   ```
+
+The Bazel target `:deepmind_lab.so` builds the Python module that interfaces
+*DeepMind Lab*. It can be build in headless hardware rendering mode (`--define
+headless=glx`), headless software rendering mode (`--define headless=osmesa`) or
+non-headless mode (`--define headless=false`).
+
+The random agent target `:random_agent` has a number of optional command line
+arguments. Run
+
+``` shell
+lab$ bazel run :random_agent -- --help
+```
+
+to see those.
+
+### Building on Red Hat Enterprise Linux Server
+
+Tested on release 7.2 (Maipo).
+
+1. Add the Extra Packages as described on
+   [fedoraproject.org](http://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F)
+2. Install Bazel's and DeepMind Lab's dependencies
+
+   ```shell
+   sudo yum -y install unzip java-1.8.0-openjdk lua lua-devel libffi-devel zip \
+     java-1.8.0-openjdk-devel gcc gcc-c++ freeglut-devel SDL2 SDL2-devel \
+     mesa-libOSMesa-devel python-devel numpy
+   ```
+3. Download and run
+   a [Bazel binary installer](https://github.com/bazelbuild/bazel/releases),
+   e.g.
+
+   ```shell
+   sudo yum -y install wget
+   wget https://github.com/bazelbuild/bazel/releases/download/0.3.2/bazel-0.3.2-installer-linux-x86_64.sh
+   sh bazel-0.3.2-installer-linux-x86_64.sh
+   ```
+4. [Clone or download *DeepMind Lab*](https://github.com/deepmind/lab).
+5. Edit `lua.BUILD` to reflect how Lua is installed on your system:
+
+   ```python
+   cc_library(
+       name = "lua",
+       linkopts = ["-llua"],
+       visibility = ["//visibility:public"],
+   )
+   ```
+   The output of `pkg-config lua --libs --cflags` might be helpful to find the
+   right include folders and linker options.
+6. Build *DeepMind Lab* using Bazel as above.
+
+### Building on SUSE Linux
+
+Tested on SUSE Linux Enterprise Server 12.
+
+1. Install Bazel's and DeepMind Lab's dependencies
+
+   ```shell
+   sudo zypper --non-interactive install java-1_8_0-openjdk \
+     java-1_8_0-openjdk-devel gcc gcc-c++ lua lua-devel python-devel \
+     python-numpy-devel libSDL-devel libOSMesa-devel freeglut-devel
+   ```
+2. Download and run
+   a [Bazel binary installer](https://github.com/bazelbuild/bazel/releases),
+   e.g.
+
+   ```shell
+   sudo yum -y install wget
+   wget https://github.com/bazelbuild/bazel/releases/download/0.3.2/bazel-0.3.2-installer-linux-x86_64.sh
+   sh bazel-0.3.2-installer-linux-x86_64.sh
+   ```
+3. [Clone or download *DeepMind Lab*](https://github.com/deepmind/lab).
+4. Edit `lua.BUILD` to reflect how Lua is installed on your system:
+
+   ```python
+   cc_library(
+       name = "lua",
+       linkopts = ["-llua"],
+       visibility = ["//visibility:public"],
+   )
+   ```
+   The output of `pkg-config lua --libs --cflags` might be helpful to find the
+   right include folders and linker options.
+5. Edit `python.BUILD` to reflect how Python is installed on your system:
+
+   ```python
+   cc_library(
+       name = "python",
+       hdrs = glob([
+           "include/python2.7/*.h",
+           "lib64/python2.7/site-packages/numpy/core/include/**/*.h",
+       ]),
+       includes = [
+           "include/python2.7",
+           "lib64/python2.7/site-packages/numpy/core/include",
+       ],
+       visibility = ["//visibility:public"],
+   )
+   ```
+   The outputs of `rpm -ql python` and `rpm -ql python-numpy-devel` might be
+   helpful to find the rihgt include folders.
+6. Build *DeepMind Lab* using Bazel as above.

docs/issues.md

@@ -0,0 +1,50 @@
+(Switch to: [Lua](lua_api.md) · [Python](python_api.md) ·
+ [Level Generation](level_generation.md) ·
+ [Tensor](tensor.md) · [Text Levels](text_level.md) ·
+ [Build](build.md) ·
+ Known Issues)
+
+# Known Issues
+
+Please take note of the following subtleties when modifying *DeepMind Lab* or
+adapting it to different platforms.
+
+## Scripting
+
+* Server and client run at different frame rates. This causes a mismatch when
+  rewards that are given by scripts are eventually received by players.
+* The script supports fractional rewards, but the engine does not. This means
+  that only integer changes in reward are consumed by the agent.
+* The script API is currently designed to work with single-player levels.
+
+## Compiling
+
+* The LCC bytecode compiler has its own pointer representation (32 bits), which
+  may differ from the host's pointer representation. Arguments of trap calls
+  need to be translated properly at the VM's syscall boundary.
+* The order in which `.asm` files are linked into `.qvm` bytecode libraries is
+  important. The current order is taken from ioq3's original `Makefile`.
+* OSMesa must be compiled without thread-local storage (TLS) if the environment
+  is to be loaded from a shared library repeatedly (e.g. via `dlopen`).
+* Loading *DeepMind Lab* repeatedly from a shared library may be the only way to
+  reliably instantiate multiple copies of the environment in the same process,
+  since the original ioq3 game engine was not designed as a library and is
+  effectively not reentrant (e.g. it has a lot of global state and global
+  initialization).
+* The SDL library cannot be linked statically, since its sound subsystem does
+  not shut down cleanly. Linking SDL from a shared object works (as we do for
+  the freestanding game binary), as does disabling sound altogether (as we do
+  for the library version).
+
+## Threading
+
+The environments are thread compatible but not thread safe. A call to one
+environment will not interfere with a call to another but calls to the same
+environment from different threads at the same time may interfere.
+
+Individual render modes have their own special exceptions:
+
+*   Software rendering - The environment is fully thread compatible.
+*   Hardware rendering - The environment can be only called from the *same*
+    thread in which it was constructed.
+*   SDL rendering - The environment can only be called from the main thread.

docs/level_generation.md

@@ -0,0 +1,37 @@
+(Switch to: [Lua](lua_api.md) · [Python](python_api.md) ·
+ Level Generation ·
+ [Tensor](tensor.md) · [Text Levels](text_level.md) ·
+ [Build](build.md) ·
+ [Known Issues](issues.md))
+
+# Level generation
+
+Levels for DeepMind Lab are _Quake III Arena_ levels. They are packaged into `.pk3`
+files (which are ZIP files) and consist of a number of components, including the
+following:
+
+   * One `.bsp` file per level ("binary space partitioning"). This contains the
+     level geometry (the "map").
+   * Optionally an accompanying `.aas` file per level ("area awareness system").
+     This contains navigation information for the in-game bots of _Quake III
+     Arena_.
+   * Textures. Textures are referenced by maps. DeepMind Lab ships with a number
+     of default textures.
+
+You can use your own maps, zipped up as a `.pk3` file, by placing the `.pk3`
+file(s) into your base directory (e.g. `$HOME/.deepmind_lab/baselab`).
+
+DeepMind Lab includes tools to generate maps: The `q3map2` tool converts a
+`.map` file into a `.bsp` file, and the `bspc` tool generates an `.aas` file
+from a `.bsp` file. The `.map` format is a human-readable text format that
+describes a map as a sequence of _entities_. Map files are cumbersome to edit by
+hand, but a variety of level editors are freely available
+(e.g.  [GtkRadiant](https://github.com/TTimo/GtkRadiant)).
+
+In addition to built-in and user-provided levels, DeepMind Lab offers additional
+sources of levels:
+
+   * Procedurally generated levels, which are created on the fly.
+   * [Text levels](text_level.md), which are simple, human-readable text files
+     that can be compiled into `.map` files. This can either be done offline,
+     or programmatically inside DeepMind Lab using Lua extensions.