Does the dataset contain transparent object instances?

[Saafke]

Hi,

Does the dataset contain transparent object instances, e.g. empty see-through bottles or cups? If so, how is your model's capability in handling these?

[ahmadyan]

Good question!

1. There are transparent objects (both bottles and cups) in the dataset.
2. Our models do have some issues with bottles and cups, not because they are transparent (AFAIR, transparent objects were being detected fine), but because they are symmetric. IMO estimating 3D bounding box of symmetric objects have gauge ambiguity in yaw, so in our evaluation that we reported here we compute different rotations for yaw of symmetric objects. See the paper for more details.
3. If you are interested in transparent objects, another work is Keypose which also open-sourced their dataset.

[Uio96]

Good question!

1. There are transparent objects (both bottles and cups) in the dataset.
2. Our models do have some issues with bottles and cups, not because they are transparent (AFAIR, transparent objects were being detected fine), but because they are symmetric. IMO estimating 3D bounding box of symmetric objects have gauge ambiguity in yaw, so in our evaluation that we reported here we compute different rotations for yaw of symmetric objects. See the paper for more details.
3. If you are interested in transparent objects, another work is Keypose which also open-sourced their dataset.

I did see you proposed a solution for symmetric objects in the paper. But it seems that your provided evaluator script did not treat them specially. I was wondering if the reported numbers in the Objectron paper also used the same script to calculate the numbers.

[lzhang57]

Good question!

1. There are transparent objects (both bottles and cups) in the dataset.
2. Our models do have some issues with bottles and cups, not because they are transparent (AFAIR, transparent objects were being detected fine), but because they are symmetric. IMO estimating 3D bounding box of symmetric objects have gauge ambiguity in yaw, so in our evaluation that we reported here we compute different rotations for yaw of symmetric objects. See the paper for more details.
3. If you are interested in transparent objects, another work is Keypose which also open-sourced their dataset.

I did see you proposed a solution for symmetric objects in the paper. But it seems that your provided evaluator script did not treat them specially. I was wondering if the reported numbers in the Objectron paper also used the same script to calculate the numbers.

The eval script for the symmetric objects is a bit different, you can find the code snippets in this discussion #26 (comment).

[Uio96]

@lzhang57 Thanks for the update. I am wondering if the current numbers for the symmetric objects in your paper are evaluated by this new script, e.g., bottle category? Or do you have a plan to use it to re-evaluate your methods in your next release?

[lzhang57]

@lzhang57 Thanks for the update. I am wondering if the current numbers for the symmetric objects in your paper are evaluated by this new script, e.g., bottle category? Or do you have a plan to use it to re-evaluate your methods in your next release?

The current numbers in our paper are evaluated in the same way as in this new script.

[Uio96]

@lzhang57 Awesome. I have a follow-up question. Of all 9 categories, which one do you apply this symmetric script on?

Except for the bottle category which is obviously symmetric, it is not very clear to me if others are symmetric or not. E.g., in your dataset, the cup category has both mug (which is not symmetric) and cup. I do not see labels discriminating them in the current release.

[lzhang57]

@Uio96 Of all 9 categories, we apply this symmetric script on cup and bottle categories.

The 'symmetry' of an object can be dependent on the viewing point. E.g. a mug can become non-symmetric when observed from some certain viewing points (when its handle is hidden), to mitigate that uncertainty we simply treat every mug instance as symmetric object.