Pickling error in Python 3.11: TypeError: code() argument 13 must be str, not int

[alxmrs]

I've noticed this line causes an error on my machine:

cubed/cubed/tests/test_core.py

Line 592 in 1ac5061

c = dill.loads(dill.dumps(c))

Looking around, I think this is a problem in Python 3.11, see python/cpython#100316.

My developer environment specs

(cubed) (base) alxrsngrtn@alexanders-air cubed % python --version
Python 3.11.9
(cubed) (base) alxrsngrtn@alexanders-air cubed % arch
arm64
(cubed) (base) alxrsngrtn@alexanders-air cubed % uname -r
23.6.0
(cubed) (base) alxrsngrtn@alexanders-air cubed % sw_vers
ProductName:            macOS
ProductVersion:         14.6.1
BuildVersion:           23G93

[tomwhite]

What's the error that you get? I'm running with Python 3.11 on an M1 Mac and haven't seen this.

[alxmrs]

I just checked out main, synced a clean environment (via uv, with python 3.11 pinned), and ran all the tests.

Here is the full stack trace

cubed/tests/test_core.py:576 (test_array_pickle[single-threaded])
spec = cubed.Spec(work_dir=/private/var/folders/h7/c4_scx2s3nbg1x36qln787mm0000gn/T/pytest-of-alxrsngrtn/pytest-44/test_array_pickle_single_threa0, allowed_mem=100000, reserved_mem=0, executor=None, storage_options=None, zarr_compressor=default)
executor = <cubed.runtime.executors.local.SingleThreadedExecutor object at 0x112150210>

    def test_array_pickle(spec, executor):
        a = xp.asarray(
            [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]],
            chunks=(2, 2),
            spec=spec,
        )
        b = xp.asarray(
            [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]],
            chunks=(2, 2),
            spec=spec,
        )
        c = xp.matmul(a, b)
    
        # we haven't computed c yet, so pickle and unpickle, and check it still works
        # note we have to use dill which can serialize local functions, unlike pickle
>       c = dill.loads(dill.dumps(c))

test_core.py:592: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../.venv/lib/python3.11/site-packages/dill/_dill.py:275: in loads
    return load(file, ignore, **kwds)
../../.venv/lib/python3.11/site-packages/dill/_dill.py:270: in load
    return Unpickler(file, ignore=ignore, **kwds).load()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dill._dill.Unpickler object at 0x116be79d0>

    def load(self): #NOTE: if settings change, need to update attributes
>       obj = StockUnpickler.load(self)
E       TypeError: code() argument 13 must be str, not int

../../.venv/lib/python3.11/site-packages/dill/_dill.py:472: TypeError