Scene graphs

Some thoughts regarding more formal scene graphs as part of RTB, rather than Swift.

A motivating scenario

We have a scene comprising:

• a Panda robot, carrying a tray on which there are 2 glasses and with a camera attached to its shoulder
• a table with two plates, each plate has 3 cakes

The plates can be moved around on the table, and the table can be moved around.

We can share references to shapes (no state) so we only need one plate, one glass and one cake instance. The state lives in the scene element/node object.

What state to we need to compute and send to Swift?

For every scene element scene we need:

• a shape
• a list of its children
• a transform relative to its parent
• an alpha value
• some id to tie the RTB object to the Swift object, could be the .id of the RTB object

What we need to send to Swift after everything is updated is a dict containing a set of:

• key = the scene element id
• value = (pose with respect to world frame, alpha)

An alpha thought

It could be useful to set an alpha for a branch of the scene graph, for example we could make the table, plates and cakes translucent. Perhaps the alpha (or a special alpha) for a scene element multiplies the alpha of all its children, so simply by setting the alpha override for the table object would achieve this.

Maybe the same for color? Make a branch of the tree go blue.

Shapes

• the usual suspects sphere, cylinder, cuboid, mesh, axes etc. based on CollisionShape with attributes like transform wrt parent, color, alpha
• a Collection shape which is a list of primitive shapes. It has a transform and its children are all defined relative to it
• a Robot object which represents a set of set of shapes parameterised by a vector. Each link object would contain a display Collection and a collision Collection. The Robot object has a method to update all its shapes given a configuration vector, similar to fkine_all.

To achieve our scenario:

• we add the camera to the Collection associated with the shoulder link
• the tray shape's parent is the EE link of the Panda