[OSPP-Week1] Build Offline Reinforcement Learning framework and test offline DQN

[pilgrimygy]

@findmyway
Hi, I am a student of the OSPP project Enriching Offline Reinforcement Learning Algorithms in ReinforcementLearning.jl. This summer, my main mission is to implement some modern offline reinforcement learning algorithms in ReinforcementLearningZoo and add some relative experiments in ReinforcementLearningExperiment.

Week 1:
In this week, I will build an offline RL framework to test the offline RL algorithms. I will only write a simple version (@Mobius1D writes the full version), so I only use it to test our algorithms and don’t make a PR. When @Mobius1D completes this framework, we will unify the interface. Specifically, my work is as follows.

1. Reuse the existing online RL framework to build the offline framework.
2. Build lightweight datasets (including cart-pole, mountain car, …)
3. Test the performance of offline DQN algorithms on these datasets. These algorithms may have some changes in detail and need to be discussed with @findmyway.

I will create a separate branch later; you can take a look and give some comments.

[findmyway]

Cool.

I will create a separate branch later; you can take a look and give some comments.

Cool, is it on your own fork?

[pilgrimygy]

Yes, in the framework branch

[findmyway]

Get it. You can also make a draft PR when you think it's ready.

[pilgrimygy]

@findmyway Can I do a simulated PR for you to review the code?

[findmyway]

Yes, that would be better.