From baf25d5a51190e42fbda69dfcd3053e30651cb85 Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 19:36:53 +0200 Subject: [PATCH 1/9] Initial readme update. Has basic info on the three main sections. Hopefully sufficient for now. --- README.md | 90 ++++++++++++++++++++++++++----------------------------- 1 file changed, 42 insertions(+), 48 deletions(-) diff --git a/README.md b/README.md index 3e45ac65..69dafd4a 100644 --- a/README.md +++ b/README.md @@ -1,80 +1,74 @@ # Retinal RL -## Setting up the development environment +A deep learning framework for vision research using deep reinforcement learning. -We provide a singularity / apptainer container which should always be up to date and allow to run the code immediately. You do not need to build it yourself (but can, of course), you can just pull it! +## Apptainer Environment -### Install Singularity / Apptainer +### Installation -First you need to install [apptainer](https://github.com/apptainer/apptainer/) (or singularity) in order to run code. - -### Get the container - -Once you have apptainer installed, you can simply pull the container +1. Install [Apptainer](https://github.com/apptainer/apptainer/) to run the containerized environment. +2. Pull the pre-built container: ```bash apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image ``` -or try to build it on your own (no advantages of doing that, except you want to change some dependency in the .def file): - +Alternatively, build from source: ```bash apptainer build retinal-rl.sif resources/retinal-rl.def ``` -### Prepare config directory for experiments - -The repository comes with some example configuration files, which you find under 'resources/config_templates'. For running experiments however, they need to be in 'config'. -You can either copy them there by hand or run the following script from the top-level directory: +### Running Experiments +The scan command prints info about the proposed neural network architecture: ```bash -bash tests/ci/copy_configs.sh +apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=scan ``` - -### Test basic functionality - -Now you are basically ready to run experiments! -To test that everything is working fine, you can run: - +The experiment must always be specified with the `+experiment` flag. To train a +model, use the `train` command: ```bash -bash tests/ci/scan_configs.sh +apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=train ``` -The script loops over all experiments defined in config/experiment and runs a "scan" on them. -If instead you want to run a single experiment file, run: - -```bash -apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=scan system.device=cpu -``` +## Hydra Configuration -## Running retinal RL simulations [DEPRECATED] +The project uses Hydra for configuration management, offering flexible experiment organization and parameter overrides. -There are three main scripts for working with `retinal-rl`: +### Directory Structure -- `train.py`: Train a model. -- `analyze.py`: Generate some analyses. -- `enjoy.py`: Watch a real time simulation of a trained agent. +``` +base/config.yaml # General and system configurations +user/ +├── brain/ # Neural network architectures +├── dataset/ # Dataset configurations +├── optimizer/ # Training optimizers +└── experiment/ # Experiment configurations +``` -Each script can be run by python in `python -m {script}`, where {script} is the name of the desired script (without the `.py` extension), followed by a number of arguments. Note that `train.py` must always be run first to create the necessary files and folders, and once run will permanently set most (all?) of the arguments of the simulation, and will ignore changes to these arguments if training is resumed. +Basic information about the configuration variables is available in the template configs. Consult the hydra documentation for more information on [configuring your project](https://hydra.cc/docs/intro/). -Certain arguments must always be provided, regardless of script, namely: +### Default Configuration -- `--env`: Specifies the desired map. This will always have the form `retinal_{scenario}`, where scenario is the shared name of one of the `.wad`/`.cfg` file pairs in the `scenarios` directory. -- `--algo`: The training algorithm; for now this should always be `APPO`. -- `--experiment`: The directory under the `train_dir` directory where simulation results are saved. +1. Configuration templates may be copied to the user directory by running: +```bash +bash tests/ci/copy_configs.sh +``` -The following argument should always be set when training for the first time: +2. Default and user configurations can be sanity-checked with: +```bash +bash tests/ci/scan_configs.sh +``` -- `--encoder_custom`: The options are `simple`, which is a small, hard-coded network that still tends to perform well, and `lindsey`, which has a number of tuneable hyperparameters. +## Weights & Biases Integration -For specifying the form of the `lindsey` network, the key arguments are: +### Basic Configuration -- `--global_channels`: The number of channels in each CNN layers, except for the bottleneck layer. -- `--retinal_bottleneck`: Number of channels in the retinal bottleneck. -- `--vvs_depth`: Number of CNN layers in the ventral stream network. -- `--kernel_size`: Size of the kernels. +By default plots and analyses are saved locally. To enable Weights & Biases logging, add the `logging.use_wandb: True` flag to the configuration file. -Finally, when training a model there are a number of additional parameters for controlling the reinforcement learning brain, and adjusting simulation parameters. The key ones to worry about are +### Parameter Sweeps -- `--hidden_size`: The size of the hidden/latent state used to represent the RL problem. -- `--num_workers`: This is the number of simulation threads to run. This shouldn't be more than the number of cores on the CPU, and can be less if the simulation is GPU bottlenecked. +[Wandb sweeps](https://docs.wandb.ai/guides/sweeps) can be added to +`user/sweeps/{sweep_name}.yaml` and launched from the command line: +```bash +apptainer exec retinal-rl.sif python main.py +experiment="$experiment" +sweep="$sweep" command=sweep +``` From 6576e09d81f550df2023ad57ee11fbe0e07e1f5c Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 19:50:15 +0200 Subject: [PATCH 2/9] More changes --- README.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 69dafd4a..a57e1c7f 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ A deep learning framework for vision research using deep reinforcement learning. ### Installation -1. Install [Apptainer](https://github.com/apptainer/apptainer/) to run the containerized environment. +1. Install [Apptainer](https://apptainer.org/docs/user/latest/) to run the containerized environment. 2. Pull the pre-built container: ```bash @@ -30,12 +30,16 @@ model, use the `train` command: apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=train ``` +`apptainer` commands can typically be replaced with `singularity` if the latter is rather used. + ## Hydra Configuration -The project uses Hydra for configuration management, offering flexible experiment organization and parameter overrides. +The project uses Hydra for configuration management. ### Directory Structure +The structure of the `./config/` directory is as follows: + ``` base/config.yaml # General and system configurations user/ @@ -45,7 +49,7 @@ user/ └── experiment/ # Experiment configurations ``` -Basic information about the configuration variables is available in the template configs. Consult the hydra documentation for more information on [configuring your project](https://hydra.cc/docs/intro/). +Template configs are available under `./resources/config_templates/user/...`, which also provide documentation of the configuration variables themselves. Consult the hydra documentation for more information on [configuring your project](https://hydra.cc/docs/intro/). ### Default Configuration @@ -54,16 +58,20 @@ Basic information about the configuration variables is available in the template bash tests/ci/copy_configs.sh ``` -2. Default and user configurations can be sanity-checked with: +2. Template and custom configurations can be sanity-checked with: ```bash bash tests/ci/scan_configs.sh ``` +which runs the `scan` command for all experiments. ## Weights & Biases Integration ### Basic Configuration -By default plots and analyses are saved locally. To enable Weights & Biases logging, add the `logging.use_wandb: True` flag to the configuration file. +By default plots and analyses are saved locally. To enable Weights & Biases logging, add the `logging.use_wandb: True` flag to the command line: +```bash +apptainer exec retinal-rl.sif python main.py +experiment="$experiment" logging.use_wandb=True command=train +``` ### Parameter Sweeps @@ -72,3 +80,5 @@ By default plots and analyses are saved locally. To enable Weights & Biases logg ```bash apptainer exec retinal-rl.sif python main.py +experiment="$experiment" +sweep="$sweep" command=sweep ``` + +Typically the only command line arguments that need a `+` prefix will be `experiment` and `sweep`. From 770311e43c45786658a80fdee26a513fb53efabe Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 19:52:16 +0200 Subject: [PATCH 3/9] Trying out some syntax --- README.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index a57e1c7f..d1729878 100644 --- a/README.md +++ b/README.md @@ -4,16 +4,18 @@ A deep learning framework for vision research using deep reinforcement learning. ## Apptainer Environment +Retinal-Rl is designed to run in a containerized environment using [Apptainer](https://apptainer.org/docs/user/latest/). + ### Installation -1. Install [Apptainer](https://apptainer.org/docs/user/latest/) to run the containerized environment. +1. [Install] Apptainer(https://apptainer.org/docs/admin/main/installation.html) to run the containerized environment. -2. Pull the pre-built container: +2a. Pull the pre-built container: ```bash apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image ``` -Alternatively, build from source: +2b. Alternatively, build from source: ```bash apptainer build retinal-rl.sif resources/retinal-rl.def ``` From 579d0a49ad06f67549a2e1567bf6bf6bae027843 Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 19:54:50 +0200 Subject: [PATCH 4/9] More syntax --- README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d1729878..dd97c482 100644 --- a/README.md +++ b/README.md @@ -10,12 +10,13 @@ Retinal-Rl is designed to run in a containerized environment using [Apptainer](h 1. [Install] Apptainer(https://apptainer.org/docs/admin/main/installation.html) to run the containerized environment. -2a. Pull the pre-built container: +2. Get the container: + + - Either pull the pre-built container: ```bash apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image ``` - -2b. Alternatively, build from source: + - or build from source: ```bash apptainer build retinal-rl.sif resources/retinal-rl.def ``` From 07b5bd60ac43eec93df2ca893407b38c419f60e8 Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 19:55:26 +0200 Subject: [PATCH 5/9] Trying something else --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index dd97c482..c15078c6 100644 --- a/README.md +++ b/README.md @@ -12,11 +12,11 @@ Retinal-Rl is designed to run in a containerized environment using [Apptainer](h 2. Get the container: - - Either pull the pre-built container: +- Either pull the pre-built container: ```bash apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image ``` - - or build from source: +- or build from source: ```bash apptainer build retinal-rl.sif resources/retinal-rl.def ``` From ebb0bef8ca16fd089e3681398d232ad42222718a Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 19:56:32 +0200 Subject: [PATCH 6/9] This time with feeling? --- README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index c15078c6..75184eff 100644 --- a/README.md +++ b/README.md @@ -12,14 +12,14 @@ Retinal-Rl is designed to run in a containerized environment using [Apptainer](h 2. Get the container: -- Either pull the pre-built container: -```bash -apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image -``` -- or build from source: -```bash -apptainer build retinal-rl.sif resources/retinal-rl.def -``` + - Either pull the pre-built container: + ```bash + apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image + ``` + - or build from source: + ```bash + apptainer build retinal-rl.sif resources/retinal-rl.def + ``` ### Running Experiments From a41fa3c0a3fdb6277f26e363e5b12bc712aaad7e Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 20:00:44 +0200 Subject: [PATCH 7/9] Final version? --- README.md | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 75184eff..b99f80fd 100644 --- a/README.md +++ b/README.md @@ -8,18 +8,18 @@ Retinal-Rl is designed to run in a containerized environment using [Apptainer](h ### Installation -1. [Install] Apptainer(https://apptainer.org/docs/admin/main/installation.html) to run the containerized environment. +1. [Install](https://apptainer.org/docs/admin/main/installation.html) Apptainer to run the containerized environment. 2. Get the container: - - Either pull the pre-built container: - ```bash - apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image - ``` - - or build from source: - ```bash - apptainer build retinal-rl.sif resources/retinal-rl.def - ``` +- Either pull the pre-built container: +```bash +apptainer pull retinal-rl.sif oras://ghcr.io/berenslab/retinal-rl:singularity-image +``` +- or build from source: +```bash +apptainer build retinal-rl.sif resources/retinal-rl.def +``` ### Running Experiments @@ -37,7 +37,7 @@ apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=t ## Hydra Configuration -The project uses Hydra for configuration management. +The project uses [Hydra](https://hydra.cc/) for configuration management. ### Directory Structure @@ -52,9 +52,11 @@ user/ └── experiment/ # Experiment configurations ``` +### Default Configuration + Template configs are available under `./resources/config_templates/user/...`, which also provide documentation of the configuration variables themselves. Consult the hydra documentation for more information on [configuring your project](https://hydra.cc/docs/intro/). -### Default Configuration +### Configuration Management 1. Configuration templates may be copied to the user directory by running: ```bash @@ -69,6 +71,8 @@ which runs the `scan` command for all experiments. ## Weights & Biases Integration +Retinal-RL supports logging to [Weights & Biases](https://wandb.ai/site) for experiment tracking. + ### Basic Configuration By default plots and analyses are saved locally. To enable Weights & Biases logging, add the `logging.use_wandb: True` flag to the command line: @@ -78,10 +82,9 @@ apptainer exec retinal-rl.sif python main.py +experiment="$experiment" logging.u ### Parameter Sweeps -[Wandb sweeps](https://docs.wandb.ai/guides/sweeps) can be added to -`user/sweeps/{sweep_name}.yaml` and launched from the command line: +[Wandb sweeps](https://docs.wandb.ai/guides/sweeps) can be added to `user/sweeps/{sweep_name}.yaml` and launched from the command line: ```bash apptainer exec retinal-rl.sif python main.py +experiment="$experiment" +sweep="$sweep" command=sweep ``` -Typically the only command line arguments that need a `+` prefix will be `experiment` and `sweep`. +Typically the only command line arguments that need a `+` prefix will be `+experiment` and `+sweep`. From c35c0c8c169f7bc3e638e75fcafc42186c40f917 Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 20:05:30 +0200 Subject: [PATCH 8/9] Finale, take two --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index b99f80fd..83916533 100644 --- a/README.md +++ b/README.md @@ -25,12 +25,12 @@ apptainer build retinal-rl.sif resources/retinal-rl.def The scan command prints info about the proposed neural network architecture: ```bash -apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=scan +apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" command=scan ``` The experiment must always be specified with the `+experiment` flag. To train a model, use the `train` command: ```bash -apptainer exec retinal-rl.sif python main.py +experiment="$experiment" command=train +apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" command=train ``` `apptainer` commands can typically be replaced with `singularity` if the latter is rather used. @@ -77,14 +77,14 @@ Retinal-RL supports logging to [Weights & Biases](https://wandb.ai/site) for exp By default plots and analyses are saved locally. To enable Weights & Biases logging, add the `logging.use_wandb: True` flag to the command line: ```bash -apptainer exec retinal-rl.sif python main.py +experiment="$experiment" logging.use_wandb=True command=train +apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" logging.use_wandb=True command=train ``` ### Parameter Sweeps -[Wandb sweeps](https://docs.wandb.ai/guides/sweeps) can be added to `user/sweeps/{sweep_name}.yaml` and launched from the command line: +[Wandb sweeps](https://docs.wandb.ai/guides/sweeps) can be added to `user/sweeps/{sweep}.yaml` and launched from the command line: ```bash -apptainer exec retinal-rl.sif python main.py +experiment="$experiment" +sweep="$sweep" command=sweep +apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" +sweep="{sweep}" command=sweep ``` -Typically the only command line arguments that need a `+` prefix will be `+experiment` and `+sweep`. +Typically the only command line arguments that need a `+` prefix will be `+experiment` and `+sweep`. Also note that the `.yaml` extension may typically be dropped. From dfaf35caf13a9187a9eea1a04c725931fa6a8283 Mon Sep 17 00:00:00 2001 From: alex404 Date: Fri, 25 Oct 2024 20:13:56 +0200 Subject: [PATCH 9/9] Okay hopefully this is really truly it --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 83916533..731786ca 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,7 @@ apptainer build retinal-rl.sif resources/retinal-rl.def ### Running Experiments -The scan command prints info about the proposed neural network architecture: +The `scan` command prints info about the proposed neural network architecture: ```bash apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" command=scan ``` @@ -82,9 +82,9 @@ apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" logging. ### Parameter Sweeps -[Wandb sweeps](https://docs.wandb.ai/guides/sweeps) can be added to `user/sweeps/{sweep}.yaml` and launched from the command line: +Wandb [sweeps](https://docs.wandb.ai/guides/sweeps) can be added to `user/sweeps/{sweep}.yaml` and launched from the command line: ```bash apptainer exec retinal-rl.sif python main.py +experiment="{experiment}" +sweep="{sweep}" command=sweep ``` -Typically the only command line arguments that need a `+` prefix will be `+experiment` and `+sweep`. Also note that the `.yaml` extension may typically be dropped. +Typically the only command line arguments that need a `+` prefix will be `+experiment` and `+sweep`. Also note that `.yaml` extensions are dropped at the command line.