Low Success Rate of LLM Data Generation Policy

[ZhouhaoZhang]

Hello,

I encountered an issue while testing the success rate of the LLM data generation policy within the drawer environment. I enabled USE_CACHE and provided different episode_ids to make the environment random, to test the success rate of a fixed task. I found the agent was really hard to complete the task. Is it because the action sampling approach used in the paper is different from the release? Or there are some default settings need to be changed to ensure robust execution?

Additionally, I encountered two issues during the experiment:

1. The sampling process seems to be different even when the same episode_id is provided.

2. Sometimes, the sampling process abruptly halts, accompanied by the following log:

WARNING Failed to converge after inverse_kinematics.py:217
299 steps:
err_norm=0.140897
WARNING Failed to converge after inverse_kinematics.py:217
299 steps:
err_norm=0.00555371
INFO RRT succeeded with 189 steps mujocoEnv.py:659
[04/15/24 15:05:11] INFO Subsampled from 189 to 19 waypoints core.py:5277
[04/15/24 15:05:18] INFO 15.500: core.py:4423
EndEffectorAction(Pose(pos=(0.17353,0.
54968,0.11796),rot=(0.05965,0.67524,-0
.53264)), grip=True, contact=True)
INFO Subsampled from 2 to 2 waypoints core.py:5277
[04/15/24 15:05:19] INFO 16.000: core.py:4423
GraspLinkAction(link='bottom_drawer_ha
ndle')
[04/15/24 15:05:20] ERROR [GraspLinkAction] Link core.py:4439
'drawer/|drawer/drawer|drawer/bottom_d
rawer|drawer/bottom_drawer_handle' is
not visible
WARNING [GraspLinkAction] Link core.py:4489
'drawer/|drawer/drawer|drawer/bottom_d
rawer|drawer/bottom_drawer_handle' is
not visible|

or

[04/15/24 15:13:44] WARNING  Failed to converge after  inverse_kinematics.py:217
                             299 steps:                                         
                             err_norm=0.131395                                  
                    WARNING  Failed to converge after  inverse_kinematics.py:217
                             299 steps:                                         
                             err_norm=1.72527                                   
[04/15/24 15:13:45] ERROR    [GraspLinkAction] all candidates       core.py:4439
                             failed GraspLinkAction(link='crayon                
                             box'):[GraspLinkPoseAction] 0th action             
                             failed: [EndEffectorAction]                        
                             EndEffectorAction(Pose(pos=(0.51082,-0             
                             .08091,0.13890),rot=(-2.93959,-0.01795             
                             ,0.46428)), grip=False, contact=False)             
                             collision: ur5e/wsg50/base (10) and                
                             table/table (32), ur5e/wsg50/base (10)             
                             and table/table (32), ur5e/wsg50/base              
                             (10) and table/table (32),                         
                             ur5e/wsg50/base (10) and table/table               
                             (32), ur5e/wsg50/right_finger (13) and             
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/left_finger                             
                             (14),[GraspLinkPoseAction] 0th action              
                             failed: [EndEffectorAction]                        
                             EndEffectorAction(Pose(pos=(0.48009,-0             
                             .09655,0.15775),rot=(-3.13465,-0.00130             
                             ,1.00760)), grip=False, contact=False)             
                             collision: ur5e/upper_arm_link (4) and             
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/forearm_link (5) and                          
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/forearm_link (5) and crayon                   
                             box/crayon box (28), ur5e/forearm_link             
                             (5) and crayon box/crayon box (28),                
                             ur5e/forearm_link (5) and crayon                   
                             box/crayon box (28), ur5e/wsg50/base               
                             (10) and table/table (32),                         
                             ur5e/wsg50/base (10) and table/table               
                             (32), ur5e/wsg50/base (10) and                     
                             table/table (32), ur5e/wsg50/base (10)             
                             and table/table (32),                              
                             ur5e/wsg50/d435i/d435i (12) and                    
                             table/table (32),                                  
                             ur5e/wsg50/d435i/d435i (12) and                    
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/left_finger                             
                             (14),[GraspLinkPoseAction] 0th action              
                             failed: [EndEffectorAction]                        
                             EndEffectorAction(Pose(pos=(0.48238,-0             
                             .13097,0.07996),rot=(-1.43057,1.47630,             
                             0.10784)), grip=False, contact=False)              
                             collision: ur5e/forearm_link (5) and               
                             horse toy/horse toy (30),                          
                             ur5e/forearm_link (5) and horse                    
                             toy/horse toy (30), ur5e/forearm_link              
                             (5) and horse toy/horse toy (30),                  
                             ur5e/forearm_link (5) and horse                    
                             toy/horse toy (30), ur5e/forearm_link              
                             (5) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_2_link (7) and table/table              
                             (32), ur5e/wrist_2_link (7) and                    
                             table/table (32), ur5e/wrist_2_link                
                             (7) and table/table (32),                          
                             ur5e/wrist_3_link (8) and table/table              
                             (32), ur5e/wsg50/base (10) and horse               
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and table/table               
                             (32), table/table (32) and                         
                             ur5e/wsg50/base (10),                              
                             ur5e/wsg50/d435i/d435i (12) and                    
                             table/table (32)                                   
                    WARNING  [GraspLinkAction] all candidates       core.py:4489
                             failed GraspLinkAction(link='crayon                
                             box'):[GraspLinkPoseAction] 0th action             
                             failed: [EndEffectorAction]                        
                             EndEffectorAction(Pose(pos=(0.51082,-0             
                             .08091,0.13890),rot=(-2.93959,-0.01795             
                             ,0.46428)), grip=False, contact=False)             
                             collision: ur5e/wsg50/base (10) and                
                             table/table (32), ur5e/wsg50/base (10)             
                             and table/table (32), ur5e/wsg50/base              
                             (10) and table/table (32),                         
                             ur5e/wsg50/base (10) and table/table               
                             (32), ur5e/wsg50/right_finger (13) and             
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/left_finger                             
                             (14),[GraspLinkPoseAction] 0th action              
                             failed: [EndEffectorAction]                        
                             EndEffectorAction(Pose(pos=(0.48009,-0             
                             .09655,0.15775),rot=(-3.13465,-0.00130             
                             ,1.00760)), grip=False, contact=False)             
                             collision: ur5e/upper_arm_link (4) and             
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/forearm_link (5) and                          
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/forearm_link (5) and crayon                   
                             box/crayon box (28), ur5e/forearm_link             
                             (5) and crayon box/crayon box (28),                
                             ur5e/forearm_link (5) and crayon                   
                             box/crayon box (28), ur5e/wsg50/base               
                             (10) and table/table (32),                         
                             ur5e/wsg50/base (10) and table/table               
                             (32), ur5e/wsg50/base (10) and                     
                             table/table (32), ur5e/wsg50/base (10)             
                             and table/table (32),                              
                             ur5e/wsg50/d435i/d435i (12) and                    
                             table/table (32),                                  
                             ur5e/wsg50/d435i/d435i (12) and                    
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32),                                  
                             ur5e/wsg50/right_finger (13) and                   
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/right_finger (13),                      
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32),                                  
                             ur5e/wsg50/left_finger (14) and                    
                             table/table (32), table/table (32) and             
                             ur5e/wsg50/left_finger                             
                             (14),[GraspLinkPoseAction] 0th action              
                             failed: [EndEffectorAction]                        
                             EndEffectorAction(Pose(pos=(0.48238,-0             
                             .13097,0.07996),rot=(-1.43057,1.47630,             
                             0.10784)), grip=False, contact=False)              
                             collision: ur5e/forearm_link (5) and               
                             horse toy/horse toy (30),                          
                             ur5e/forearm_link (5) and horse                    
                             toy/horse toy (30), ur5e/forearm_link              
                             (5) and horse toy/horse toy (30),                  
                             ur5e/forearm_link (5) and horse                    
                             toy/horse toy (30), ur5e/forearm_link              
                             (5) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_1_link (6) and horse                    
                             toy/horse toy (30), ur5e/wrist_1_link              
                             (6) and horse toy/horse toy (30),                  
                             ur5e/wrist_2_link (7) and table/table              
                             (32), ur5e/wrist_2_link (7) and                    
                             table/table (32), ur5e/wrist_2_link                
                             (7) and table/table (32),                          
                             ur5e/wrist_3_link (8) and table/table              
                             (32), ur5e/wsg50/base (10) and horse               
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and horse                     
                             toy/horse toy (30), ur5e/wsg50/base                
                             (10) and horse toy/horse toy (30),                 
                             ur5e/wsg50/base (10) and table/table               
                             (32), table/table (32) and                         
                             ur5e/wsg50/base (10),                              
                             ur5e/wsg50/d435i/d435i (12) and                    
                             table/table (32)|                                  
corrupted double-linked list
Process finished with exit code 139 (interrupted by signal 11:SIGSEGV)

I'm a newcomer to this field, I would greatly appreciate any insights or guidance you can provide to help resolve these issues.

Thank you for your assistance!

[OceansDepp]

hi! I wonder how did you successfully using CACHE when generating the data? I just rollback to the first commit and download the responses files. But I still have the issue: 'openai.error.AuthenticationError: No API key provided. I am so confused and have no idea what might be the reason. Looking forward to your reply, thank you!