-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Match image masks with the images they overlay #286
Comments
This issue covers several topics: Q1. How masks and bboxes are associated with images in general Answers: A1. Images are (typically) file records. Masks and boxes can be associated with file records in one of the two ways:
A2, A3. A rigid way to locate a visualization feature (e.g. Voxel51) is to require a fixed, well-known column name for masks or bboxes, and a fixed format of coordinates. This is likely not ideal for datachain. A preferred way is to "cast" a visualization layer on a column, e.g. right-click and select "Visualize as bbox" on a column. Since there are several ways to provide bbox coordinates, we might offer sub-options as well – or require one fixed order assuming the users will do the transformation. It also rather straightforward to see if the selected column hosts a list (array of arrays), or a single bbox/mask array entry, so we need to visualize one or more objects per image. Note that several "casts" might be required to complete a view. For example, when bboxes are associated with labels, "Visualize as bbox label" cast might be added to pick the latter. To make this compound "view" permanent, we might also need to persist some special GUI config on a dataset. This concludes the required visualization features for masks and bboxes. Note that Voxel is still the leading provider of visualizations, so we may defer to their design for other featurettes – such as actions when mousing over the bbox or a mask shade in the image. Optional. For example, consider the following dataset before grouping by column "file":
And after grouping by column "file":
|
Description
If I run a query that produces a dataset with multiple images and multiple image masks, I want to associate each image mask with the image it overlays.
If DataChain doesn't provide a heuristic or data model out of the box which defines this association, users will have to develop non-standard ways to do this, which isn't ideal.
The text was updated successfully, but these errors were encountered: