Issue encountered while running monai bundle inference on a CPU

[mpsampat]

Hello!
I am trying to run the inference.json for the WholeBody_ct_segmentation bundle. the inference.json file is here:
https://github.com/Project-MONAI/model-zoo/blob/dev/models/wholeBody_ct_segmentation/configs/inference.json
When I run on a CPU with 128 gb memory I get this error:

"RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU."

I tried to change line 15 in the inference.json file:
https://github.com/Project-MONAI/model-zoo/blob/dev/models/wholeBody_ct_segmentation/configs/inference.json#L15
I changed it from
"device": "$torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')",
to
"device": "cpu",

But i still get the same error as above.

1. Is it possible to run monai bundle inference on a CPU ?
2. If yes, could you tell me what I am doing incorrectly ?

thanks
Mehul

[ericspod]

I think this specific issue is that the weights for the model were saved as GPU tensors which is what torch.load will load them as without map_location set. Your solution for now is to set the map_location value for the CheckpointLoader (which uses torch.load) to be your device, so insert "map_location": "@device", after line 135 in inference.json. In the long term we need to make sure our saved weights for bundles are saved to CPU so that we can avoid this issue in the future.

[mpsampat]

thank you @ericspod . I will try out this solution and report back here.