MPC-RL

Instructions

Use anaconda to create a virtual environment

Step 1. Install miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Step 2. Clone repo and create conda environment

conda env create -f environment.yml
conda activate cartpole

Train MPC-RL agent

RL w/ differentiable MPC (NN actor)

python train_RL_diff_MPC.py agent=ddpg_mpc

RL w/ differentiable MPC (actor made of paramaters only)

python train_RL_diff_MPC.py agent=ddpg_mpc_alternative

RL w/ MPC as part of the environmnent

python train_RL_env_MPC.py agent=sac

python train_RL_env_MPC.py agent=ddpg

python train_PPO_env_MPC.py agent=ppo

Tensorboard is deactivated by default. It can be activated running

python train_RL_diff_MPC.py agent=ddpg_mpc use_tb=true

python train_RL_env_MPC.py agent=sac use_tb=true

python train_RL_env_MPC.py agent=ddpg use_tb=true

python train_PPO_env_MPC.py agent=ppo use_tb=true

To access tensorboard run in another shell

tensorboard --logdir experiments