Could not save checkpoint/cannot pickle 'weakref' object

[ndrubins]

Hi,

I'm getting this error running remove-background:

cellbender:remove-background: Saving a checkpoint...
cellbender:remove-background: Could not save checkpoint
cellbender:remove-background: Traceback (most recent call last):
  File "/home/miniconda/envs/py3_9/lib/python3.9/site-packages/cellbender/remove_background/checkpoint.py", line 115, in save_checkpoint
    torch.save(model_obj, filebase + '_model.torch')
  File "/home/miniconda/envs/py3_9/lib/python3.9/site-packages/torch/serialization.py", line 441, in save
    _save(obj, opened_zipfile, pickle_module, pickle_protocol)
  File "/home/miniconda/envs/py3_9/lib/python3.9/site-packages/torch/serialization.py", line 653, in _save
    pickler.dump(obj)
TypeError: cannot pickle 'weakref' object

It's reported in this issue, but the Ilia Kats's pyro repo is gone.

Any idea how to solve this?

Thanks

[sjfleming]

Hi @ndrubins , yes, this error happens when you use a version of pytorch >= 2.0

What is currently recommended is to install cellbender in an environment with python 3.7. Python 3.7 was used to develop and test cellbender, and python 3.7 also limits the pytorch version to < 2.0.

I am working to make cellbender compatible with pytorch >= 2.0, but I have not quite figured it out.

The progress on that can be tracked here: #203

[plijnzaad]

Could you please put the strict requirement on python 3.7 and torch < 0.2.0 somewhere prominent in the installation instructions? Many people including myself are wasting a lot of time on trying to understand/solve this problem ...