Add xarm6 robot with different grippers

[dwaitbhatt]

Add xarm6 with left and right allegro grippers, xarm gripper, and a test version without a gripper.
This resolves the request in #635

[StoneT2000]

Thank you for the contribution @dwaitbhatt, I'll try and get around reviewing this later today or next monday!

I think only thing I need to verify really is to check if the new files aren't too large (if they are, we just make them downloadable instead of being on git), and if the simulation speed + collision meshes are reasonable.

[dwaitbhatt]

Thank you for the contribution @dwaitbhatt, I'll try and get around reviewing this later today or next monday!

I think only thing I need to verify really is to check if the new files aren't too large (if they are, we just make them downloadable instead of being on git), and if the simulation speed + collision meshes are reasonable.

Sounds good @StoneT2000! Some notes from my own testing:

• xarm6 with the default xarm gripper (xarm6_xarmgripper) is somehow not stable in simulation - when running with random actions, the gripper joints tend to go out of limits and it eventually leads all robot joints going blowing up and going to nan - similar to [Bug] xarm7_ability robot instability #633, although xarm6 takes much longer than 5 steps to become unstable. I suspect the issue there is also with the ability hand, not the arm. In my case, I am using the gripper URDF from the manufacturer's own repo.

• xarm6 with both allegro hands is stable, but for some reason the simulation is faster with CPU (60fps) than GPU (~20fps), atleast when tested with demo_robot_script and random actions.

[StoneT2000]

Thanks for the details.

Another question, for the actual joint controller of the xarm6 gripper, does the end effector have 1 joint (like a mimic joint) or are there multiple controlling

I recall that to correctly model that gripper we need to model the mimic joints correctly and correctly model that in sim.

Finally where are the urdf / assets from? Usually need to add a readme somewhere just for record keeping about where it's from and what modifications are made if any to the original URDF.

[dwaitbhatt]

Another question, for the actual joint controller of the xarm6 gripper, does the end effector have 1 joint (like a mimic joint) or are there multiple controlling

I recall that to correctly model that gripper we need to model the mimic joints correctly and correctly model that in sim.

It does have a single mimic joint to control the gripper, however the implementation is from the manufacturer's repo and not my own. Need to look into this more carefully.

Finally where are the urdf / assets from? Usually need to add a readme somewhere just for record keeping about where it's from and what modifications are made if any to the original URDF.

Here are the sources for the assets and the modifications on top:

1. xarm6_xarmgripper:

   • Source: xArm manufacturer's repo
   • Modifications:
     • Removed gazebo/ros related tags
     • Added safety_controller for safe operation beyond joint limits (inspired by panda description)
     • Added a material (specular/ambient/diffuse color) based on provided MTL files

2. xarm6_nogripper:

   • Source: Same as xarm6_xarmgripper, without gripper links and joints

3. xarm6_allegro_left:

   • Source: urdf from a previous issue, which was confirmed to work
   • Modifications:
     • Updated collision geometries to match xarm6_xarmgripper
     • Updated orientation of gripper so that the gripper wrist is connected to final link of the arm

4. xarm6_allegro_right:

   • Source: Arm description from xarm6_nogripper with right hand allegro gripper description from Maniskill assets

[StoneT2000]

Thanks for the info. RE the gripper modelling, I have maybe the correct code for how to model that which was uploaded in response to #477, its the floating version of robotiq gripper. It is a bit non-trivial though: https://github.com/haosulab/ManiSkill/blob/main/mani_skill/agents/robots/floating_robotiq_2f_85_gripper/floating_robotiq_2f_85_gripper.py

[StoneT2000]

noticed a small issue here. Can the red dot visual be removed? We can keep the invisible eef link.

Another is that is the original URDF meant to be all silver/gray colored? I think the xarm gripper joints are partially black.

[akuramshin]

@StoneT2000 @dwaitbhatt Hello! I loaded the embodiment xarm6_xarmgripper into the StackCube-v1 environment using the demo_manual_control.py script (using pd_ee_delta_pose as the default controller).

Proceeding to press the f/g keys to open/close the gripper causes the robot's ee pose to change despite only changing the gripper position.
out2.webm

The reason I am here is because I have been experiencing something similar with my agent setup of the robotiq 2F-85 gripper on the panda arm:
out3.webm

In both videos I am just periodically pressing either the f or g key.

Please let me know if this belongs in its own post. I just thought it was relevant since this PR also has the unexpected behaviour.

[StoneT2000]

Hi @akuramshin , how did you implement the robotiq one? Did you set the drives correctly / are you using our URDFs in this repo?

[akuramshin]

I followed your thread here: #477 (comment)

Basically replacing the original panda hand with the urdf of the robotiq gripper you provided. For the controller I copy pasted the controller implemented for the FloatingRobotiq2F85Gripper agent (including the drive creation).

[StoneT2000]

Ok for your issue @akuramshin can you create a different issue on this repo? This PR will just be for the xarm gripper which is different. I will follow up there.

[dwaitbhatt]

Hey @StoneT2000! I have made the suggested changes, here are some details:

• xarm_gripper geometry is quite different from robotiq_2f_85 (xarm_gripper has no inner finger or finger pads), and we need to find new "magic arrays" with precomputed poses for drive creation. I wasn't sure exactly which poses are required so I have simply replaced the xarm_gripper with robotiq for now, since they are functionally similar - both are one DoF parallel grippers.
• I added xarm6 SRDF from the manufacturer's repo, however I could not find official SRDFs for allegro and robotiq grippers, so I wrote them myself since there were too many unnecessary collision checks happening there.

Unresolved issues:

• Running demo_robot_script with xarm6_robotiq and --none-actions on GPU backend it throws a critical error and stops responding when I try to set a joint position target. This does not happen with CPU backend. Seems like it is related to _after_loading_articulation() for gripper modeling.
  [2024-10-24 00:05:29.936] [SAPIEN] [critical] PxArticulationJointReducedCoordinate::setDriveTarget(): it is illegal to call this method if PxSceneFlag::eENABLE_DIRECT_GPU_API is enabled!
• The simulation speed issue persists even with the SRDFs, where the simulation is slower with GPU and fast with CPU. This can be prominently seen by running demo_robot_script with xarm6_allegro and --random-actions. Actually this can be seen even without xarm6, when running demo_robot_script with allegro_hand_right on CPU vs GPU backend.
• Let me know if you think I should make assets downloadable.

[StoneT2000]

Sorry I am a little bit behind on the PRs here, I will try to take a deeper look into this later this week.

[dwaitbhatt]

Sorry I am a little bit behind on the PRs here, I will try to take a deeper look into this later this week.

Hello @StoneT2000! Did you get a chance to review this PR?