Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unyt_dask_array pickleability #270

Closed
chrishavlin opened this issue Sep 8, 2022 · 1 comment · Fixed by #185
Closed

unyt_dask_array pickleability #270

chrishavlin opened this issue Sep 8, 2022 · 1 comment · Fixed by #185

Comments

@chrishavlin
Copy link
Contributor

Opening this in anticipation of #185 being merged.

Currently, pickling a unyt_dask_array will strip units and return a plain dask array. Unfortunately it doesn't seem to be a trivial fix. Documenting here so that it's not forgotten...

from unyt.dask_array import unyt_from_dask
import dask
import pickle
x = unyt_from_dask(dask.array.random.random((10000,10000), chunks=100), 'm')
x2 = pickle.loads(pickle.dumps(x))
print(x)
# unyt_dask_array<random_sample, shape=(10000, 10000), dtype=float64, chunksize=(100, 100), chunktype=numpy.ndarray, units=m>
print(x2)
# dask.array<random_sample, shape=(10000, 10000), dtype=float64, chunksize=(100, 100), chunktype=numpy.ndarray>

x2 should equal x.

@neutrinoceros
Copy link
Member

#206 may give some pointers

@chrishavlin chrishavlin linked a pull request Sep 9, 2022 that will close this issue
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants