Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Questions w.r.t. GNNPolicy implementation #5

Open
semihcanturk opened this issue Apr 13, 2023 · 1 comment
Open

Questions w.r.t. GNNPolicy implementation #5

semihcanturk opened this issue Apr 13, 2023 · 1 comment

Comments

@semihcanturk
Copy link

Hi @HannesStark @TobiasSchmidtDE , thank you for this repository -- I've been playing around with it for a bit now, and find it much easier to work with than the original NerveNet. I am planning to implement some updates for my own research, but I've noticed some issues with the current codebase, and was wondering if you had any pointers for me to look into.

  1. I see that most configs use GnnPolicy (v0), but there exists a more up-to-date policy GnnPolicy_V2. Do they have separate config files that are not pushed to the repo, or require additional hyperparameter tuning? I did perform some hyperparameter tuning on Ant (based on GNN_AntBulletEnv-v0.yaml), but cannot get the Ant to move even at 5M iterations with the GNN policies. The MLPs work well, however, pointing to either bad hyperparameters or a potential bug in the GNNPolicy. Did your eventual hyperparameters Any pointers w.r.t. these would be a huge help.
  2. Agents other than Ant need their .xml files to be edited slightly to be parsed correctly. Specifically, the parts with ignore in their names need to be commented out for to work:
    <body name="torso">
      <!--joint armature="0" axis="1 0 0" damping="0" limited="false" name="ignore1" pos="0 0 0" stiffness="0" type="slide"/-->
      <!--joint armature="0" axis="0 0 1" damping="0" limited="false" name="ignore2" pos="0 0 0" ref="1.25" stiffness="0" type="slide"/-->
      <!--joint armature="0" axis="0 1 0" damping="0" limited="false" name="ignore3" pos="0 0 0" stiffness="0" type="hinge"/-->

I'm currently testing if GNN training will work for a simpler agent e.g. Hopper. Are there any other issues I should be aware of? Many thanks!
@2361098148
Copy link

Hi @semihcanturk , I have met the same problem, I cannot get the Ant to move after training with GNN, have you figured out?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@semihcanturk @2361098148