Custom gym environment support

[lucalazzaroni]

I'm currently working on a project (branching from this) where I created a new gym environment adding policies for autonomous driving in a highway. I defined my own reward weights for each of the actions the agent could take, so I would like to evaluate them, in order to get an optimal value for the reward function and, then, perform a DQN. My question is if your tool supports or not custom gym environments and, if so, how could I exploit it. I tried your notebook in colab but there's a problem with the python version, so I installed it on linux but I don't get how to implement it in my code.
Thanks in advance for your help.

[AdamGleave]

So long as the environments implement the standard Gym interface it should work fine with this code. In the notebook, all you need to do is replace gym.make(env_name) with whatever is needed to create your environment. (If your environments are registered in the Gym registry, then you just need to change env_name.)

The trickiest part will probably be getting the reward models you're using hooked up to the code -- it needs to follow the API here. It sounds like you're just comparing hand-designed reward functions, as opposed to learned reward models? In that case you might find PointMassGroundTruth a useful example of defining your own reward function.

The Colab issue I assume is because it only supports Python 3.6? It should run fine locally in Jupyter notebook, let me know if you run into any issues with the notebook.

[lucalazzaroni]

Thanks for the infos, I'll give it a try in the next few days. Meanwhile I opened the notebook locally with Jupyter but when it executes the line importing tensorflow a "kernel died" message prompts and the notebook restarts. I tried all the tensorflow 1.15.x versions but he problem persists. Any idea about it? I'm using Ubuntu 20.04.2 LTS from Oracle VirtualBox.

[AdamGleave]

Thanks for the infos, I'll give it a try in the next few days. Meanwhile I opened the notebook locally with Jupyter but when it executes the line importing tensorflow a "kernel died" message prompts and the notebook restarts. I tried all the tensorflow 1.15.x versions but he problem persists. Any idea about it? I'm using Ubuntu 20.04.2 LTS from Oracle VirtualBox.

Interesting, not a failure mode I've seen before. I'd suggest running ./scripts/build_venv.sh venv from the project root directory. This should install the same versions of everything we use for our continuous integration. Then . ./venv/bin/activate, pip install jupyter and try running the Jupyter notebook from there. If you're still seeing issues, can you import tensorflow OK outside of Jupyter?

It's worked on Ubuntu 20.04 in the past so that should be supported, although I've never tried inside VirtualBox. I could imagine that causing problems if TensorFlow is trying to use GPU for example. But otherwise should run fine.

[lucalazzaroni]

I followed the steps you suggested but the error persists, So i tried to import tensorflow outside jupyter but it returns me Illegal instruction (core dumped). At this point I think the problem could be VirtualBox

[AdamGleave]

That seems likely. TensorFlow is normally compiled to use extensions like AVX2 that may be missing in VirtualBox. There's a long thread at at tensorflow/tensorflow#17411 You could try compiling TensorFlow from source, or run native. We've not tested this software on anything other than Linux, but it should work OK so long as you get all the dependencies installed.

…

On Thu, 4 Feb 2021 at 11:01, lucalazzaroni ***@***.***> wrote: I followed the steps you suggested but the error persists, So i tried to import tensorflow outside jupyter but it returns me Illegal instruction (core dumped). At this point I think the problem could be VirtualBox — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#53 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALZ3IYN24JAZU7XP7K5ZVDS5J5AXANCNFSM4W6H6H7Q> .

[lucalazzaroni]

Hi @AdamGleave, sorry for my late response, I tried to compile tensorflow from source but no luck. I think I'll try again with colab and see if there's any possibility to use it along with python 3.7 instead of 3.6.