Segfault when reading many tiles in parallel

[SimonDanisch]

Hi I'm getting the following segfault:

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffc196c5ddf -- get_adjusted_ptr at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_personality.cc:233
in expression starting at none:0
get_adjusted_ptr at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_personality.cc:233
__gxx_personality_imp at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_personality.cc:604
_GCC_specific_handler at /workspace/srcdir/gcc-13.2.0/libgcc\unwind-seh.c:289
__gxx_personality_seh0 at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_personality.cc:810
_chkstk at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RtlFindCharInUnicodeString at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RtlRaiseException at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
RaiseException at C:\WINDOWS\System32\KERNELBASE.dll (unknown line)
_Unwind_RaiseException at /workspace/srcdir/gcc-13.2.0/libgcc\unwind-seh.c:334
__cxa_throw at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_throw.cc:93
getBytes at /workspace/srcdir/LASzip/src\bytestreamin_file.hpp:126
init at /workspace/srcdir/LASzip/src\lasreaditemcompressed_v3.cpp:1647
read at /workspace/srcdir/LASzip/src\lasreadpoint.cpp:478
laszip_read_point at /workspace/srcdir/LASzip/src\laszip_dll.cpp:4579
getindex at ...\dev\PointClouds\src\laszip.jl:161
iterate at ...\dev\PointClouds\src\laszip.jl:144 [inlined]
iterate at ...\dev\PointClouds\src\laszip.jl:144 [inlined]
iterate at .\generator.jl:45 [inlined]
_collect at .\array.jl:800
collect_similar at .\array.jl:709 [inlined]
map at .\abstractarray.jl:3374 [inlined]
get_points at ...\dev\Tyler\src\provider\pointclouds\geotiles-pointcloud-provider.jl:68
unknown function (ip: 00000239f8f49228)
load_tile_data at ...\dev\Tyler\src\provider\pointclouds\geotiles-pointcloud-provider.jl:78
fetch_tile at ...\dev\Tyler\src\tiles.jl:97 [inlined]
#2 at ...\dev\Tyler\src\tiles.jl:48
get! at ....julia\packages\LRUCache\ctUcD\src\LRUCache.jl:169
run_loop at ...\dev\Tyler\src\tiles.jl:46
#7 at ...\dev\Tyler\src\tiles.jl:87
unknown function (ip: 00000239f8efd34b)
jl_apply at C:/workdir/src\julia.h:2156 [inlined]
start_task at C:/workdir/src\task.c:1202
Allocations: 968254642 (Pool: 968252436; Big: 2206); GC: 247

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: pure virtual method called
EXCEPTION_ACCESS_VIOLATION at 0x3a73ccb -- terminate called without an active exception

[25224] signal 22: SIGABRT
zip/src\lasreadpoint.cpp:412
in expression starting at none:0
in expression starting at noin expression starting at none:0
read at /workspace/srcdir/LASzip/src\lasreadpoint.cpp:412
laszip_read_point at /workspace/srcdir/LASzip/src\laszip_dll.cpp:4579
getindex at ...\dev\PointClouds\src\laszip.jl:161
iterate at ...\dev\PointClouds\src\laszip.jl:144 [inlined]
iterate at .\generator.jl:45 [inlined]
collect_to! at .\array.jl:838
collect_to_with_first! at .\array.jl:816 [inlined]
collect_to_with_first! at .\array.jl:816 [inlined]
_collect at .\array.jl:810
raise at C:\WINDOWS\System32\msvcrt.dll (unknown line)
abort at C:\WINDOWS\System32\msvcrt.dll (unkmap at .\abstractarray.jl:3374 [inlined]
get_points at ...\dev\Tyler\src\provider\pointclouds\geotiles-pointcloud-provider.jl:68
unknown function (ip: 00000239f8f49228)
load_tile_data at ...\load_tile_data at ...\dev\Tyler\src\provider\pointclouds\geotiles-pointcloud-provider.jl:78
fetch_tile at ...\dev\Tyler\src\tiles.jl:97 [inlined]
#2 at ...\dev\Tyler\src\tiles.jl:48
get! at ....julia\packages\LRUCache\ctUcD\src\LRUCache.jl:169
run_loop at ...\dev\Tyler\src\tiles.jl:46
#7 at ...\dev\Tyler\src\tiles.jl:87
v3/libsupc++\vterminate.cc:95
#7 at ...\dev\Tyler\src\tiles.jl:87
unknown function (ip: 00000239f8efd34b)
__terminate at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_terminate.cc:48
jl_apply at C:/workdir/src\julia.h:2156 [inlined]
start_task at C:/workdir/src\task.c:1202
Allocations: 968254642 (Pool: 968252436; Big: 2206); GC: 247
terminate at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\eh_terminate.cc:58
__cxa_pure_virtual at /workspace/srcdir/gcc-13.2.0/libstdc++-v3/libsupc++\pure.cc:50

It only happens if I really spam the reading of tiles, is laszip + pointclouds not threadsafe, or is this something unrelated?

[mfsch]

I don’t have a ton of visibility into the LASzip library, but from here it sounds like multi-threading is indeed problematic. I would have thought that loading separate files (rather than accessing parts of the same file) would still work, but maybe there’s still some shared state between readers?

It would be great if we could spread the LAZ decompression over all cores eventually and perhaps even be bandwidth-limited when you have enough CPU cores. In principle that should be doable since the LAZ points are stored in chunks which can be decompressed independently. However, that would likely require porting the decompression to Julia, which is a) quite a bit of work, and b) probably quite error-prone.