You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The code will read the export message from a queue, find all of the files from a single leaf partition, compact them into 1 file, sort the data and save a new parquet file in S3.
The text was updated successfully, but these errors were encountered:
Background
Create the code which will run in an ECS job to export the files for a single leaf partition.
Parent issue
Description
The code will read the export message from a queue, find all of the files from a single leaf partition, compact them into 1 file, sort the data and save a new parquet file in S3.
The text was updated successfully, but these errors were encountered: