NAC

Installation

System requirements:

• Python3.6
• python3-venv
• tkinter (python3-tk)

All required python packages can be found in requirements.txt. Install using

python3 -m venv env
source env/bin/activate
pip3 install -r requirements.txt

NOTE: This requires tkinter, which can be installed on ubuntu by apt-get install python3-tk.

Contents

This package contains the implementation of the Natural Actor-Critic (NAC) algorithm. You can find the actual implementation in nac.py, and available models in model.py.

Usage and Examples

This section contains usage descriptions and code examples. You can also find complete example usages of the interface in bayesian_opt.py, and run_experiment.py.

Training

For training use nac.nac.train(...). Hyperparameters are directly passed to this function, and are explained in the docstring of that function. We do not repeat this documentation here to avoid inconsistency.

Setting a seed

Random values are generated by pytorch and numpy. To set a seed for these libraries, use

torch.manual_seed(seed)
np.random.seed(seed)

Running a saved model

You can use run_model.py to load and run a saved model .pt file.

Sources

• Natural Actor-Critic https://homes.cs.washington.edu/~todorov/courses/amath579/reading/NaturalActorCritic.pdf
• Setting up experiments http://www-clmc.usc.edu/publications/p/peters-ICHR2003.pdf
• Pytorch Documentation https://pytorch.org/docs/stable/index.html
• Numpy Documentation https://docs.scipy.org/doc/numpy/reference/