[BUG] JavaScript heap out of memory

[kreisligaspieler]

Steps to reproduce

Use following pipeline:

pipeline TestPipeline {
Extractor -> TextFileInterpreter->RangeSelector->CSVInterpreter->ColumnDeleter->TableInterpreter->Loader;
block Extractor oftype HttpExtractor {
            url: "https://derwebmaster.eu/wp-content/uploads/2024/02/star0000-1.csv";}
block TextFileInterpreter oftype TextFileInterpreter {encoding: "utf8";}
block RangeSelector oftype TextRangeSelector {lineFrom: 1;}
block CSVInterpreter oftype CSVInterpreter {enclosing: '';delimiter: ",";}
block ColumnDeleter oftype ColumnDeleter {delete: [];}
block TableInterpreter oftype TableInterpreter {
            header: false;
            columns: ["col1" oftype integer,"col2" oftype integer,"col3" oftype integer,"col4" oftype integer,"col5" oftype integer,"col6" oftype decimal,"col7" oftype integer,"col8" oftype integer,"col9" oftype decimal,"col10" oftype integer,"col11" oftype decimal,"col12" oftype integer];}
block Loader oftype SQLiteLoader {table: "tDbNEGvkAEdn"; file: "test.sqlite";}
}

Original source of the csv: https://sdm.lbl.gov/fastbit/data/samples.html

Description

• Expected: Pipeline should download the file and store the content in a database
• Actual: FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

Debug:
[TestPipeline] Overview:
Blocks (7 blocks with 1 pipes):
-> Extractor (HttpExtractor)
-> TextFileInterpreter (TextFileInterpreter)
-> RangeSelector (TextRangeSelector)
-> CSVInterpreter (CSVInterpreter)
-> ColumnDeleter (ColumnDeleter)
-> TableInterpreter (TableInterpreter)
-> Loader (SQLiteLoader)

[Extractor] Fetching raw data from https://derwebmaster.eu/wp-content/uploads/2024/02/star0000-1.csv

Console output:
<--- Last few GCs --->

[99110:0x65186f0] 716409 ms: Mark-Compact (reduce) 2047.2 (2078.4) -> 2047.2 (2079.1) MB, 29.36 / 0.00 ms (+ 602.9 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 999 ms) (average mu = 0.992, current mu = 0.97[99110:0x65186f0] 718079 ms: Mark-Compact (reduce) 2048.2 (2079.1) -> 2048.2 (2080.1) MB, 1666.92 / 0.00 ms (average mu = 0.689, current mu = 0.002) allocation failure; scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

1: 0xcc08f6 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
2: 0x1054130 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
3: 0x1054417 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
4: 0x1273655 [node]
5: 0x128a178 [node]
6: 0x126167e v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
7: 0x1262964 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
8: 0x123f586 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node]
9: 0x12310e4 v8::internal::FactoryBasev8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Taggedv8::internal::Map, v8::internal::AllocationAlignment) [node]
10: 0x12329ae v8::internal::FactoryBasev8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [node]
11: 0x1232a26 v8::internal::FactoryBasev8::internal::Factory::NewStringFromOneByte(v8::base::Vector, v8::internal::AllocationType) [node]
12: 0x15762fa v8::internal::ValueDeserializer::ReadObjectInternal() [node]
13: 0x15766ef v8::internal::ValueDeserializer::ReadObject() [node]
14: 0x1575d1c v8::internal::ValueDeserializer::ReadDenseJSArray() [node]
15: 0x157618e v8::internal::ValueDeserializer::ReadObjectInternal() [node]
16: 0x15766ef v8::internal::ValueDeserializer::ReadObject() [node]
17: 0x1575d1c v8::internal::ValueDeserializer::ReadDenseJSArray() [node]
18: 0x157618e v8::internal::ValueDeserializer::ReadObjectInternal() [node]
19: 0x15766ef v8::internal::ValueDeserializer::ReadObject() [node]
20: 0x157682b v8::internal::ValueDeserializer::ReadObjectWrapper() [node]
21: 0x10772f9 v8::ValueDeserializer::ReadValue(v8::Localv8::Context) [node]
22: 0xe4761b node::worker::Message::Deserialize(node::Environment*, v8::Localv8::Context, v8::Localv8::Value*) [node]
23: 0xe4b327 [node]
24: 0x7ff875dceadd
Abgebrochen (Speicherabzug geschrieben)

[rhazn]

(For reference, the file is 185mb large)

Thanks for reporting this :).