Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datasets: SampleRays prunes custom fields from Batch #146

Open
orperel opened this issue May 1, 2023 · 0 comments
Open

Datasets: SampleRays prunes custom fields from Batch #146

orperel opened this issue May 1, 2023 · 0 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@orperel
Copy link
Collaborator

orperel commented May 1, 2023

Subclassing WispDataset to, i.e, return a MultiviewBatch with additional supervision fields, and applying SampleRays transform on top may cause some of the fields to be omitted.

For example, in the following, view_idx is omitted:

    def __getitem__(self, idx) -> MultiviewBatch:
        """Retrieve a batch of rays and their corresponding values.
        Rays are precomputed from the dataset's cameras, and are cached within the dataset.
        By default, rays are assumed to have corresponding rgb values, sampled from the dataset's images.

        Returns:
            (MultiviewBatch): A batch of rays and their rgb values. The fields can be accessed as a dictionary:
                "rays" - a wisp.core.Rays pack of ray origins and directions, pre-generated from the dataset camera.
                "rgb" - a torch.Tensor of rgb color which corresponds the gt image's pixel each ray intersects.
                "masks" - a torch.BoolTensor specifying if the ray hits a dense area or not.
                "view_idx" - the index of current camera / view.
                 This is estimated from the alpha channel of the gt image, where mask=True if alpha > 0.5.
        """
        out = MultiviewBatch(
            rays=self.data["rays"][idx],
            rgb=self.data["rgb"][idx],
            masks=self.data["masks"][idx]
            view_idx=idx
        )

        if self.transform is not None:
            out = self.transform(out)

        return out
@orperel orperel added the bug Something isn't working label May 1, 2023
@orperel orperel self-assigned this May 1, 2023
@orperel orperel changed the title SampleRays prunes custom fields from Batch Datasets: SampleRays prunes custom fields from Batch May 1, 2023
@orperel orperel added this to the Wisp 0.1.3 milestone May 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant