forked from NVIDIAGameWorks/kaolin
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implemented batched SurfaceMesh class that greatly simplifies working…
… with surface meshes (NVIDIAGameWorks#740) * Implemented batched SurfaceMesh class that greatly simplifies working with normals and other attributes loaded from different mesh representations. Changes also include: - non-backward compatible changes to USD and OBJ readers - extensive tutorial for working with meshes - much simplified boiler plate code in manu tutorials - utility to compute vertex normals - Added utility function to center points and simplified code duplicated across tutorials Signed-off-by: Maria Masha Shugrina <[email protected]> * update python in readthedocs Signed-off-by: Clement Fuji Tsang <[email protected]> * force scipy version to 1.10.1 Signed-off-by: Clement Fuji Tsang <[email protected]> --------- Signed-off-by: Maria Masha Shugrina <[email protected]> Signed-off-by: Clement Fuji Tsang <[email protected]> Co-authored-by: Maria Masha Shugrina <[email protected]> Co-authored-by: Clement Fuji Tsang <[email protected]>
1 parent
e3716b2
commit ea70a80
Showing
45 changed files
with
4,292 additions
and
544 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
.. _kaolin.rep: | ||
|
||
kaolin.rep | ||
========== | ||
|
||
This module includes higher-level Kaolin classes ("representations"). | ||
|
||
API | ||
--- | ||
|
||
Classes | ||
^^^^^^^ | ||
|
||
* :ref:`SurfaceMesh <kaolin.rep.SurfaceMesh>` | ||
* :ref:`Spc <kaolin.rep.Spc>` | ||
|
||
Other | ||
^^^^^^^^^ | ||
|
||
.. automodule:: kaolin.rep | ||
:members: | ||
:exclude-members: | ||
SurfaceMesh, | ||
Spc | ||
:undoc-members: | ||
:show-inheritance: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
:orphan: | ||
|
||
.. _kaolin.rep.Spc: | ||
|
||
kaolin.rep.Spc | ||
=========================== | ||
|
||
API | ||
--- | ||
|
||
.. autoclass:: kaolin.rep.Spc | ||
:members: | ||
:undoc-members: | ||
:show-inheritance: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,146 @@ | ||
:orphan: | ||
|
||
.. _kaolin.rep.SurfaceMesh: | ||
|
||
SurfaceMesh | ||
=========================== | ||
|
||
Tutorial | ||
-------- | ||
|
||
For a walk-through of :class:`kaolin.rep.SurfaceMesh` features, | ||
see `working_with_meshes.ipynb <https://github.com/NVIDIAGameWorks/kaolin/blob/master/examples/tutorial/working_with_meshes.ipynb>`_. | ||
|
||
API | ||
--- | ||
|
||
* :ref:`Overview <rubric mesh overview>` | ||
* :ref:`Supported Attributes <rubric mesh attributes>` | ||
* :ref:`Batching <rubric mesh batching>` | ||
* :ref:`Attribute Access and Auto-Computability <rubric mesh attribute access>` | ||
* :ref:`Inspecting and Copying <rubric mesh inspecting>` | ||
* :ref:`Tensor Operations <rubric mesh tensor ops>` | ||
|
||
.. autoclass:: kaolin.rep.SurfaceMesh | ||
:members: | ||
:undoc-members: | ||
:member-order: bysource | ||
:exclude-members: Batching, attribute_info_string, set_batching, to_batched, getattr_batched, cat, | ||
vertices, face_vertices, normals, face_normals, vertex_normals, uvs, face_uvs, faces, face_normals_idx, face_uvs_idx, | ||
material_assignments, materials, cuda, cpu, to, float_tensors_to, detach, get_attributes, has_attribute, has_or_can_compute_attribute, | ||
probably_can_compute_attribute, get_attribute, get_or_compute_attribute, check_sanity, to_string, as_dict, describe_attribute, | ||
unset_attributes_return_none, allow_auto_compute, batching, convert_attribute_batching | ||
|
||
|
||
.. _rubric mesh batching: | ||
|
||
.. rubric:: Supported Batching Strategies | ||
|
||
``SurfaceMesh`` can be instantiated with any of the following batching | ||
strategies, and supports conversions between batching strategies. Current | ||
batching strategy of a ``mesh`` object can be read from ``mesh.batching`` or | ||
by running ``print(mesh)``. | ||
|
||
For example:: | ||
|
||
mesh = kaolin.io.obj.load_mesh(path) | ||
print(mesh) | ||
mesh.to_batched() | ||
print(mesh) | ||
|
||
.. autoclass:: kaolin.rep.SurfaceMesh.Batching | ||
:members: | ||
|
||
.. automethod:: attribute_info_string | ||
.. automethod:: check_sanity | ||
.. automethod:: set_batching | ||
.. automethod:: to_batched | ||
.. automethod:: getattr_batched | ||
.. automethod:: cat | ||
.. automethod:: convert_attribute_batching | ||
|
||
.. _rubric mesh attribute access: | ||
|
||
.. rubric:: Attribute Access | ||
|
||
By default, ``SurfaceMesh`` will attempt to auto-compute missing attributes | ||
on access. These attributes will be cached, unless their ancestors have | ||
``requires_grad == True``. This behavior of the ``mesh`` object can be changed | ||
at construction time (``allow_auto_compute=False``) or by setting | ||
``mesh.allow_auto_compute`` later. In addition to this convenience API, | ||
explicit methods for attribute access are also supported. | ||
|
||
For example, using **convenience API**:: | ||
|
||
# Caching is enabled by default | ||
mesh = kaolin.io.obj.load_mesh(path, with_normals=False) | ||
print(mesh) | ||
print(mesh.has_attribute('face_normals')) # False | ||
fnorm = mesh.face_normals # Auto-computed | ||
print(mesh.has_attribute('face_normals')) # True (cached) | ||
|
||
# Caching is disabled when gradients need to flow | ||
mesh = kaolin.io.obj.load_mesh(path, with_normals=False) | ||
mesh.vertices.requires_grad = True # causes caching to be off | ||
print(mesh.has_attribute('face_normals')) # False | ||
fnorm = mesh.face_normals # Auto-computed | ||
print(mesh.has_attribute('face_normals')) # False (caching disabled) | ||
|
||
|
||
For example, using **explicit API**:: | ||
|
||
mesh = kaolin.io.obj.load_mesh(path, with_normals=False) | ||
print(mesh.has_attribute('face_normals')) # False | ||
fnorm = mesh.get_or_compute_attribute('face_normals', should_cache=False) | ||
print(mesh.has_attribute('face_normals')) # False | ||
|
||
|
||
.. automethod:: get_attributes | ||
.. automethod:: has_attribute | ||
.. automethod:: has_or_can_compute_attribute | ||
.. automethod:: probably_can_compute_attribute | ||
.. automethod:: get_attribute | ||
.. automethod:: get_or_compute_attribute | ||
|
||
.. _rubric mesh inspecting: | ||
|
||
.. rubric:: Inspecting and Copying Meshes | ||
|
||
To make it easier to work with, ``SurfaceMesh`` supports detailed print | ||
statements, as well as ``len()``, ``copy()``, ``deepcopy()`` and can be converted | ||
to a dictionary. | ||
|
||
Supported operations:: | ||
|
||
import copy | ||
mesh_copy = copy.copy(mesh) | ||
mesh_copy = copy.deepcopy(mesh) | ||
batch_size = len(mesh) | ||
|
||
# Print default attributes | ||
print(mesh) | ||
|
||
# Print more detailed attributes | ||
print(mesh.to_string(detailed=True, print_stats=True)) | ||
|
||
# Print specific attribute | ||
print(mesh.describe_attribute('vertices')) | ||
|
||
.. automethod:: to_string | ||
.. automethod:: describe_attribute | ||
.. automethod:: as_dict | ||
|
||
.. _rubric mesh tensor ops: | ||
|
||
.. rubric:: Tensor Operations | ||
|
||
Convenience operations for device and type conversions of some or all member | ||
tensors. | ||
|
||
.. automethod:: cuda | ||
.. automethod:: cpu | ||
.. automethod:: to | ||
.. automethod:: float_tensors_to | ||
.. automethod:: detach | ||
|
||
.. rubric:: Other |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
numpy<1.27.0,>=1.19.5 | ||
scipy==1.10.1 | ||
-f https://download.pytorch.org/whl/lts/1.8/torch_lts.html | ||
torch==1.8.2+cpu |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. | ||
# All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
from __future__ import annotations | ||
import torch | ||
|
||
|
||
def center_points(points: torch.FloatTensor, normalize: bool = False, eps=1e-6): | ||
r"""Returns points centered at the origin for every pointcloud. If `normalize` is | ||
set, will also normalize each point cloud spearately to the range of [-0.5, 0.5]. | ||
Note that each point cloud is centered individually. | ||
Args: | ||
points (torch.FloatTensor): point clouds of shape :math:`(\text{batch_size}, \text{num_points}, 3)`, | ||
(other channel numbers supported). | ||
normalize (bool): if true, will also normalize each point cloud to be in the range [-0.5, 0.5] | ||
eps (float): eps to use to avoid division by zero when normalizing | ||
Return: | ||
(torch.FloatTensor) modified points with same shape, device and dtype as input | ||
""" | ||
assert len(points.shape) == 3, f'Points have unexpected shape {points.shape}' | ||
|
||
vmin = points.min(dim=1, keepdim=True)[0] | ||
vmax = points.max(dim=1, keepdim=True)[0] | ||
vmid = (vmin + vmax) / 2 | ||
res = points - vmid | ||
if normalize: | ||
den = (vmax - vmin).max(dim=-1, keepdim=True)[0].clip(min=eps) | ||
res = res / den | ||
return res |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
from .spc import Spc | ||
from .surface_mesh import SurfaceMesh | ||
|
||
__all__ = [k for k in locals().keys() if not k.startswith('__')] |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
[pytest] | ||
log_cli = true | ||
log_cli_level = 10 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. | ||
# All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
import os | ||
import pytest | ||
import torch | ||
|
||
from kaolin.utils.testing import FLOAT_TYPES, with_seed | ||
import kaolin.ops.pointcloud | ||
|
||
|
||
@pytest.mark.parametrize('device, dtype', FLOAT_TYPES) | ||
def test_center_points(device, dtype): | ||
with_seed(9, 9, 9) | ||
if dtype == torch.half: | ||
rtol, atol = 1e-3, 1e-3 | ||
else: | ||
rtol, atol = 1e-5, 1e-8 # default torch values | ||
|
||
B = 4 | ||
N = 20 | ||
points = torch.rand((B, N, 3), device=device, dtype=dtype) # 0..1 | ||
points[:, 0, :] = 1.0 # make sure 1 is included | ||
points[:, 1, :] = 0.0 # make sure 0 is included | ||
points = points - 0.5 # -0.5...0.5 | ||
|
||
factors = 0.2 + 2 * torch.rand((B, 1, 1), device=device, dtype=dtype) | ||
translations = torch.rand((B, 1, 3), device=device, dtype=dtype) - 0.5 | ||
|
||
# Points are already centered | ||
assert torch.allclose(points, kaolin.ops.pointcloud.center_points(points), atol=atol, rtol=rtol) | ||
assert torch.allclose(points * factors, kaolin.ops.pointcloud.center_points(points * factors), atol=atol, rtol=rtol) | ||
|
||
# Points translated | ||
assert torch.allclose(points, kaolin.ops.pointcloud.center_points(points + 0.5), atol=atol, rtol=rtol) | ||
|
||
points_centered = kaolin.ops.pointcloud.center_points(points + translations) | ||
assert torch.allclose(points, points_centered, atol=atol, rtol=rtol) | ||
|
||
points_centered = kaolin.ops.pointcloud.center_points(points * factors + translations) | ||
assert torch.allclose(points * factors, points_centered, atol=atol, rtol=rtol) | ||
|
||
# Now let's also try to normalize | ||
points_centered = kaolin.ops.pointcloud.center_points(points * factors + translations, normalize=True) | ||
assert torch.allclose(points, points_centered, atol=atol, rtol=rtol) | ||
|
||
# Now let's test normalizing when there is zero range in one of the dimensions | ||
points[:, :, 1] = 1.0 | ||
points_centered = kaolin.ops.pointcloud.center_points(points * factors + translations, normalize=True) | ||
points[:, :, 1] = 0.0 | ||
assert torch.allclose(points, points_centered, atol=atol, rtol=rtol) | ||
|
||
# Now let's try normalizing when one element of the batch is degenerate | ||
points[0, :, :] = torch.tensor([0, 2., 4.], dtype=dtype, device=device).reshape((1, 3)) | ||
points_centered = kaolin.ops.pointcloud.center_points(points * factors + translations, normalize=True) | ||
points[0, :, :] = 0 | ||
assert torch.allclose(points, points_centered, atol=atol, rtol=rtol) |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/amsterdam.usda |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/ico_flat.mtl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/ico_flat.obj |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/ico_flat.usda |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/ico_smooth.mtl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/ico_smooth.obj |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/ico_smooth.usda |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../../../sample_data/meshes/pizza.usda |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters