TSDF Voxel Center Offset

[Algomorph]

Question: internally, how are voxel centers treated in TSDF?

TLDR; for instance, when a mesh is extracted, it seems like currently there is no extra offset:
https://github.com/intel-isl/Open3D/blob/ad4455f09bf47db99c791257fda96e77859ffb80/cpp/open3d/t/geometry/kernel/TSDFVoxelGridImpl.h#L714-L716
https://github.com/intel-isl/Open3D/blob/ad4455f09bf47db99c791257fda96e77859ffb80/cpp/open3d/t/geometry/kernel/TSDFVoxelGridImpl.h#L761-L763

Am I correct in thinking then that voxel_x_center_meters = voxel_size * voxel_global_x_coordinate?

[Algomorph]

Upon further experimentation & extending Open3D code, I believe voxel centers are directly represented by the multiple of voxel_size and the global coordinates, with no offset. This ultimately means spatial hash block coordinates will have an offset of 1/2 voxel from the nearest voxel center (since these have to include an N^3 cube of whole voxels), which may be relevant to any algorithms involving the hash blocks.

[theNded]

It is generally a matter of understanding. My interpretation is that the position at which we store the TSDF value is in practice the voxel center (which is, conceptually, usually regarded as a corner of the voxel where we need to add and subtract 0.5 voxel size back and forth). The surface extraction is based on the interpolation of isosurface values at these positions, and the TSDF values at these positions are computed with projective data association -- that being said, I would prefer to say that there is no bias/offset.

In fact, when I imagine the representation in my head, I usually view it as a collection of grid points, instead of cubes. That perspective might be closer to the implementation.

[Algomorph]

@theNded , just wanted to let you know, I haven't forgotten about this issue and it's still on my TODO list.
Currently, I am working on an extension that will allow Open3D to be used in a DynamicFusion context.

The sources are in a separate extension right now, but I think can be easily transferred to Open3D in the future when complete.
In case you're interested in taking a look at some of the sources, here are some links:

https://github.com/Algomorph/NeuralTracking/blob/main/csrc/geometry/WarpableTSDFVoxelGrid.cpp
https://github.com/Algomorph/NeuralTracking/blob/main/csrc/geometry/kernel/WarpableTSDFVoxelGridImpl.h
https://github.com/Algomorph/NeuralTracking/blob/main/csrc/geometry/Graph.cpp
https://github.com/Algomorph/NeuralTracking/blob/main/csrc/geometry/kernel/GraphImpl.h
https://github.com/Algomorph/NeuralTracking/blob/main/csrc/geometry/kernel/GraphUtilitiesImpl.h

It's a work-in-progress, I can update you once there is some tangible result.

[theNded]

That's super exciting! Please prioritize this over the small fixes.

I also have a deformable grid implementing the SLAC paper. This is not as sophisticated as DynamicFusion, just FYI.

I will be more than happy to help if you plan to merge these functionalities to Open3D!

[theNded]

Hey I want to let you know that there are some major upgrades to the hashmap interface and ongoing changes to the TSDF voxel grids. Please feel free to reach out either here or via email if you see any compatibility issues.

[Algomorph]

@theNded thanks for letting me know. We've dealt with a few compatibility issues so far, perhaps not so major. I can tell your code is changing for the better, though, so we'll make sure to work those changes in when we're ready. I'm on vacation right now until Thurs next week, but hopeful we'll have something really neat for you by the end of the year, still working out the quirks in dynamic geometry tracking over longer sequences.

[theNded]

Thanks for the update! I'm rewriting the voxel grid part as well for differentiable volumetric rendering, will let you know when it's in alpha shape.

[germanros1987]

Should we close this discussion @Algomorph ?

[Algomorph]

@germanros1987, the issue isn't really resolved because maintainers failed to address or understand the original problem. I think this post demonstrates the issue best. My recommendation would be to first understand the problem -- it's not a "convention" problem, it's a real geometry problem, in my view -- then close this convo and open a new issue, referencing all of the necessary details.

The dynamic scene reconstruction discussion is a separate topic and we can open a separate discussion for it later, when I am closer to being able to merge my fork into Open3D.

[Algomorph]

To really drive the point home, I estimate that your current schema actually causes about 23% incorrect block allocations given 8x8x8 hash block size. This percentage scales down given bigger hash block size. It's about 11.5% missing/unallocated blocks and about 11.5% unneeded/incorrect blocks being allocated if the block is aligned to the camera axis, and these ratios change (but their sum remains approximately the same) for other block orientations.

The reason you don't immediately see the effects is that, during subsequent allocations, often the correct blocks are getting hit.

The fix is super-simple, i.e. the cost of 3 additions per check, to account for the voxel centers not lying at the corner of the voxel hash blocks.