Infinite loop while loading the lidar folders!

[amanysh99]

During the execution of the code, after switching to CPU processing and selecting a subset of the dataset (specifically, the 'rr_dataset_23_11' folder), I encountered an issue where the program appears to enter an infinite loop while attempting to load 4545 lidar files from three folders. I am currently unable to determine the cause of this issue.

[Kait0]

Can you debug and report in which line of code the training enters an infinite loop?
Have not seen this before.

[amanysh99]

Umm while debugging the code it enters an infinite loop here:

[amanysh99]

Here is the train.py and model.py after switching to CPU: https://github.com/amanysh99/Carla

[Kait0]

hm? the main function is not very explicit.

maybe its the multiprocessing strategy.
You could try changing the fork method to
mp.set_start_method('spawn')
or
mp.set_start_method('forkserver')

fork is the default but can sometimes deadlock. You can try the other two and see if it helps.

[amanysh99]

Unfortunately I had tried the two options sorrowfully all in veins :(

[Kait0]

transfuser/team_code_transfuser/train.py

Line 159 in 3852e65

dataloader_train = DataLoader(train_set, sampler=sampler_train, batch_size=args.batch_size, worker_init_fn=seed_worker, generator=g_cuda, num_workers=8, pin_memory=True)

I would change the num workers in the dataloaders to 0.
That will turn off parallelism and helps you debug easier.
You can try debugging then again, you should get a better error message of where the code enters an infinite loop.

[amanysh99]

It enters an infinite loop again without displaying any obvious error message!

[Kait0]

and you can't debug?
If debugging doesn't work you can go the traditional way with placing print( statements everywhere to see where the code hangs.

[Kait0]

aren't you mixing windows style paths with Linux style paths?

Also I would avoid putting spaces in paths, should generally work but sometimes not.

r'D:\Lab 6\transfuser\team_code_transfuser\data\rr_dataset_23_11'
should work python can handle windows paths.
Does the folder actually exist if you post this path in the explorer?

[amanysh99]

Yes that exactly my fault that I mixed Windows and Linux style paths, but I solved and after checking the folder exist, but still enters an infinite loop:(

[amanysh99]

Now I am just trying to debug the code on only one folder of data which is "lr_dataset_23_11" and just one scenario inside that folder which is "Routes_clipped_Town03_lr_Seed0" and root0000, so the path will be like this: "D:\Lab6\transfuser\team_code_transfuser\data\lr_dataset_23_11\Routes_clipped_Town03_lr_Seed0\Route_0000", so it can now loop over the "rgb, lidar, semantic, depth,....", and I am ensuring that the code read the path correctly, but it still enters infinite loop I am really can't figure why that happen?
Deepseek suggest that there is a conflict in the data structure!