Memory overflow when running score_pipeline

[EdenWuyifan]

alpha-automl/alpha_automl/automl_manager.py

Line 85 in 0a65e4a

score, start_time, end_time = score_pipeline(pipeline.get_pipeline(), self.X, self.y, self.scoring,

Found out a memory overflow issue for running score_pipeline with the whole dataset in automl_manager.py. This will likely crush the whole process when working with a large dataset. I encountered this issue for automl cup phase 1 (98,000 rows of data). Here’s my solution so far: 23c9ba4

This problem also occurs when testing huggingface text classification example locally with docker image.