Wrap nested_dask reduce
#414
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dougbrn
approved these changes
Sep 13, 2024
|
|
||
| """ | ||
| ndf = nd.NestedFrame.from_dask_dataframe(self._ddf.reduce(func, *args, meta=meta, **kwargs)) | ||
| return self.__class__(ndf, self._ddf_pixel_map, self.hc_structure) |
There was a problem hiding this comment.
Maybe a silly question; do you want the result of this to be a healpix dataset object, rather than a nestedframe? The output of this can have a large range of potential outputs, and will often not have any spatial information.
I could see the arguments for it being a catalog being that it's still valid hipscat if you have the index, pixel map, and structure, and if that makes joining this back into another catalog then that sounds good. I guess I'm just curious if you thought about that here at all?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds the
reducefunction to theCatalogclass, wrapping thereducefunction innd.NestedFrame