-
-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Purpose of CommonRLInterface #166
Comments
In short, CommonRLInterface is simple enough to be shared across different packages, like POMDPs AlphaZero and ReinforcementLearning.jl |
Ok... So if I understand correctly, do you mean that other packages (like POMDPs, AlphaZero) would often directly use |
I'd say partially instead of entirely. So that we can easily test an environment across different packages. |
Are you referring to those environments that have already implemented their own |
So the Common here means that we all obey the same set of interfaces and don't need to know others' implementation details ( |
Yes. I understand. But in order for the |
Many things in |
Ok. Now I get it. Thanks! |
@Sid-Bhatia-0 A goal of CommonRLInterface (at least in my mind) is to get to a place where it can both be very expressive and very stable. We added the The historical perspective is this: When CommonRLInterface was started, RLBase, also had an In any case, now that we have it, I think it is nice to have a separate environment package that we can discuss apart from any particular RL framework. Of course, there is always this danger: https://xkcd.com/927/ 🤣 |
Thank you for pointing out these things @zsunberg
Nice idea! I never thought about something like this before.
I see. So are we hoping/aiming that the RLBase and CommonRLInterface will both eventually become equally expressive? This would imply that either one can be converted to the other in a lossless manner. Also, at such a stage, if it comes, some crucial packages like
Also, could you throw some light on what do you mean by this environment package? Methods like |
Yes, I think this should be a goal for CommonRLInterface at least.
That's up to @findmyway or other people who are working on those packages. There is a lot to learn about interface design. I will get a lot of experience with other people using CommonRLInterface in my class this Spring, and we can see what other people writing RL packages prefer. If things like
I just mean it has an interface for an environment and not agents or policies or experience buffers, etc. A complete RL framework like ReinforcementLearning.jl would have all of these. |
Got it. Thanks! |
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
I was wondering what is the purpose and use case of the
CommonRLInterface
.Let us say a user has their own RL interface called
MyRLInterface
.This means that in order to use
RLCore
,RLZoo
etc., one would have to do the following conversion (if usingCommonRLInterface
):MyRLInterface -> CommonRLInterface -> RLBase interface
Since
RLBase interface
is more expressive thanCommonRLInterface
, the conversionCommonRLInterface -> RLBase interface
would be lossless.If
MyRLInterface
is more expressive thanCommonRLInterface
, then the conversionMyRLInterface -> CommonRLInterface
may be lossy.If
MyRLInterface
is less expressive thanCommonRLInterface
, then the conversionMyRLInterface -> RLBase interface
can be done directly without the additional step to convert toCommonRLInterface
in between.So I am wondering when would one need to use the
CommoneRLInterface
?The text was updated successfully, but these errors were encountered: