MNTDP

/!\ Documentation in progress, do not hesitate to open an issue for questions.

Installation:

Dependencies

conda create -n MNTDP python=3.8
conda activate MNTDP
pip install -r requirements.txt
conda install graphviz
conda install pydot

Depending on the stream graphviz and pydot may need to be installed, e.g. for configs/streams/s_in.yaml, as indicated above.

Now let's get Mongo and start a server:

mkdir -p /checkpoint/${USER}/mongo/{db,logs}
cd /checkpoint/${USER}/mongo 
wget -O - https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-4.2.0.tgz | tar -xzvf -  
./mongodb-linux-x86_64-ubuntu1804-4.2.0/bin/mongod --dbpath /checkpoint/${USER}/mongo/db --logpath /checkpoint/${USER}/mongo/logs/mongodb.log --fork

Which should give the following output: child process started successfully, parent exiting Mongo is used with Sacred to keep track of the results.

The next step is to start Ray's head node:

ray start --head --redis-port 6381

by default, ray will use all gpus available.

and then the experiment:

python run.py with configs/streams/s_plus.yaml

the different files in the config/streams/ directory corresponds to the streams of the CTrL benchmark. Use the configs/streams/s_test.yaml debug stream to test whether the code works correctly.

To stop the mongo server:

/checkpoint/${USER}/mongo/mongodb-linux-x86_64-ubuntu1804-4.2.0/bin/mongod --dbpath /checkpoint/${USER}/mongo/db --shutdown

To stop Ray head node:

ray stop

Additional configurations

Specific configurations for Mongo and Visdom can be provided by editing the corresponding file in the resources folder.