Grasp Generation Techniques

[kavidey]

Fully Automatic

Grasp Generation Pipeline

1. Downsample point cloud to known resolution (this increases speed and ensures that the algorithm works even if the distance from the camera to the object changes)
2. Select a set of points right around where the user clicked, this is the starting point for the optimization
3. Sement the object
   • Randomly grow the selection (only points within a specified radius of the currently active points are considered)
   • Calculate the error on the new selection. The objective function is weighted sum of the convexity of the point cloud (the distance between the point cloud and its convex hull) and the number of points
   • Use simulated annealing to decide whether to keep the new selection or not
   • Repeat until converged (usually < 10 iterations)
4. Generate a bounding box
   • How the bounding box is generated has a large impact on the output grasp
   • An axis-aligned bounding box is simple and generates straight-on grasps, but may not work well with objects that are at an angle
   • An automatically generated bounding box (with one of the three axes aligned with the SE2 orientation specified by the user) is more complex, but can work better with objects that are rotated in more than one axis
5. Pick a face of the bounding box that is facing out of the cell, and generate a 6DoF grasp pose based on it

Remaining questions

• Still need to test this on a larger variety of objects, especially mesh / bagged (how is the depth data affected?)

Fully Manual

In a fully manual setup, the user now needs to specify the width in addition to XY position and orientation relative to the camera. There are a few methods of specifying width:

• Preset width buttons (similar to speed presets in stretch_web_interface)
• Take mouse release position into account for SE2 press/release to get distance
  • User specifies the location of the both fingers
  • User specifies the center of the grasp and the width of the fingers

In a fully manual setup I think it also makes the most sense for the approach direction to straight on (unless we add in additional camera views / DoF)
That means that the only unconstrained grasp variable is how far the gripper should go into the bin. I think it is okay to just calculate the location of the point cloud and use that. Its not fully manual, but it should be pretty simple and accurate.
The issue of inaccurate depth data is is an issue here too, though a smaller one