You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FYI:
I have encountered several fatal SIGBUS errors using TotalSegmentator which uses nnUnetV2 under the hood.
The errors occurred when running TotalSegmentator on large inputs.
After some digging, I was able to boil the issue down to the predict_from_data_iterator function of nnUNet, which passes the (in my case quite large) prediction result tensors to worker processes. For this step, the tensors need to be serialized, which apparently can cause memory alignment issues and trigger SIGBUS errors.
I noticed the error would disappear if I removed the multiprocessing logic, but numpy arrays seem much more stable regarding serialization and fixed the SIGBUS errors even with multiprocessing, hence this nnUNet PR.
The text was updated successfully, but these errors were encountered:
FYI:
I have encountered several fatal SIGBUS errors using TotalSegmentator which uses nnUnetV2 under the hood.
The errors occurred when running TotalSegmentator on large inputs.
After some digging, I was able to boil the issue down to the predict_from_data_iterator function of nnUNet, which passes the (in my case quite large) prediction result tensors to worker processes. For this step, the tensors need to be serialized, which apparently can cause memory alignment issues and trigger SIGBUS errors.
I noticed the error would disappear if I removed the multiprocessing logic, but numpy arrays seem much more stable regarding serialization and fixed the SIGBUS errors even with multiprocessing, hence this nnUNet PR.
The text was updated successfully, but these errors were encountered: