Esper: a framework for end-to-end video analytics

Esper is a development environment that provides out-of-the-box integration for a variety of tools useful for analyzing videos, including:

• Scanner: high-performance parallel processing of videos
• Rekall: data representation and operations for spatiotemporal data in Python
• VGrid: visualizing metadata on videos using Python and Javascript
• Frameserver: dynamically extracting frames from a video
• Django: managing SQL models and serving web pages
• Spark: parallel analysis of metadata

Esper is kind of like Ruby on Rails but for video analytics. It provides a skeleton of a data science workbench that you can customize for your particular project, along with a set of useful utility functions, models, preinstalled dependencies, and so on.

⚠️ WARNING ⚠️: Esper is a tool for programmers. It uses a programmatic query interface, which means we use REMOTE CODE EXECUTION to run queries. DO NOT expose this interface publicly, or else risk having a hacker trash your computer, data, and livelihood.

Setup

First, install Docker CE, Python 3.5, jq, and pip. If you're on Ubuntu, you can install Python/pip/jq as follows:

sudo apt-get install python3 python3-pip jq

Ensure that you have Docker version >= 17.12, which you can check by running:

$ docker --version
Docker version 17.12.0-ce, build c97c6d6

Note: If you have a GPU and are running on Linux, then install nvidia-docker2. Set device = "gpu-9.1-cudnn7 in config/local.toml.

Next, you need to install Esper. You can do this through pip, or see the developer instructions below for installing from source.

pip3 install esper-video

Finally you need to set up your Esper project.

mkdir my_project
cd my_project
wget https://raw.githubusercontent.com/scanner-research/esper/master/config/local.toml
esper -c local.toml
docker-compose up -d

TODO: next steps

Esper developers

If you're developing the Esper core platform or otherwise want to stay up to date with our dependencies, then you should clone and link our submodules.

git clone --recursive https://github.com/scanner-research/esper

TODO: remaining instructions