Issue with train_3d.py - KeyError: 2 in function.py #17
Comments
|
I think the key is associated with multi-organ issue, and there might be sth wrong in your dataset. Did you follow the steps in our provided dataset and provide command? I've double checked by git clone this repository and run with the datasets, and it seems no issue from my side. |
|
Dear Jiayuan Zhu,
Thank you for your prompt response.
I followed the provided steps to download and extract the BTCV dataset, and
then ran the code as instructed. However, I encountered the multi-organ
issue you mentioned. I am currently studying and trying to understand your
code, and I believe this will help me address the problem. Thank you for
pointing that out.
Additionally, I would like to ask for more details about the specific GPU
model you used for running the training and inference processes. I
encountered the following issue across multiple GPUs, including 2080, 3090,
and A6000, despite fully installing all dependencies listed in the provided
.yml file and confirming that PyTorch can access the GPU:
…---------------------------------------------------------------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/mnt/Medical-SAM2-main/train_3d.py", line 115, in <module>
main()
File "/mnt/Medical-SAM2-main/train_3d.py", line 105, in main
tol, (eiou, edice) = function.validation_sam(args, nice_test_loader,
epoch, net, writer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/Medical-SAM2-main/func_3d/function.py", line 272, in
validation_sam
_, _, _ = net.train_add_new_bbox(
^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/Medical-SAM2-main/sam2_train/sam2_video_predictor.py", line
439, in train_add_new_bbox
out_frame_idx, out_obj_ids, out_mask_logits = self.train_add_new_points(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/Medical-SAM2-main/sam2_train/sam2_video_predictor.py", line
523, in train_add_new_points
current_out, _ = self._run_single_frame_inference(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/Medical-SAM2-main/sam2_train/sam2_video_predictor.py", line
1351, in _run_single_frame_inference
pred_masks_gpu = fill_holes_in_mask_scores(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/Medical-SAM2-main/sam2_train/utils/misc.py", line 255, in
fill_holes_in_mask_scores
is_hole = (labels > 0) & (areas <= max_area)
^^^^^^^^^^
*RuntimeError: CUDA error: no kernel image is available for execution on
the device*
*Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.*
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
I would appreciate any guidance you can provide on resolving this issue.
Thank you once again for your support.
Best regards,
Xukun Zhang
Jiayuan Zhu ***@***.***> 于2024年8月29日周四 21:04写道：
I think the key is associated with multi-organ issue, and there might be
sth wrong in your dataset. Did you follow the steps in our provided dataset
and provide command? I've double checked by git clone this repository and
run with the datasets, and it seems no issue from my side.
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AVUQWFWYICGR6EJKOWYMFTTZT4L6JAVCNFSM6AAAAABNIVBTXSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJXGYYDEOJUHE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
the cuda issue has been addressed in sam2: facebookresearch/sam2#144. follow the solution there to see if that helps. |
|
Dear Jiayuan Zhu,
Thank you for your prompt response and for pointing me to the solution in
SAM2.
I have followed the suggested solution from the referenced GitHub issue.
However, the problem persists on my end. I set up the environment using the
same Python version and executed conda env create -f environment.yml across
multiple GPUs. Unfortunately, I still encounter the same error message.
I would appreciate any further guidance or alternative solutions you could
provide to help resolve this issue.
Thank you for your continued support.
Best regards,
Xukun
Jiayuan Zhu ***@***.***> 于2024年8月29日周四 21:25写道：
… the cuda issue has been addressed in sam2:
facebookresearch/sam2#144
<facebookresearch/sam2#144>.
follow the solution there to see if that helps.
—
Reply to this email directly, view it on GitHub
<#17 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AVUQWFWL52SR6PSZ4NM3MQDZT4OMRAVCNFSM6AAAAABNIVBTXSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJXGY2TGMRSHE>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Could you please tell me how you fixed it? I have the same error as you. |
I'm running the MedSAM2 code for the "3D case - Abdominal Multiple Organs Segmentation" following the provided instructions. After downloading the BTCV dataset and setting up the environment, I encountered an error when running the command:
python train_3d.py -net sam2 -exp_name BTCV_MedSAM2 -sam_ckpt ./checkpoints/sam2_hiera_small.pt -sam_config sam2_hiera_s -image_size 1024 -val_freq 1 -prompt bbox -prompt_freq 2 -dataset btcv -data_path ./data/btcvThe error occurs in function.py at the line:
bbox = bbox_dict[id][ann_obj_id]with the following traceback:
KeyError: 2It seems like the prompt for this slice wasn't passed correctly, leading to the KeyError. Could you please provide guidance on how to resolve this issue?
Thank you!
The text was updated successfully, but these errors were encountered: