diff --git a/config/data-loader-config.example.yml b/config/data-loader-config.example.yml index addffe59..f0bca9c8 100644 --- a/config/data-loader-config.example.yml +++ b/config/data-loader-config.example.yml @@ -30,8 +30,10 @@ Config: no_confirmation: false # Max violations to display, default is 10, can be overridden by -M/--max-violations argument max_violations: 10 - #Disable saving parent IDs in children + # Disable saving parent IDs in children no_parents: false + # Split the loading transaction into separate transactions for each file + split_transactions: false # S3 bucket name, if you are loading from an S3 bucket, can be overridden by -b/--bucket argument s3_bucket: diff --git a/docs/data-loader.md b/docs/data-loader.md index 95261812..95775f6c 100644 --- a/docs/data-loader.md +++ b/docs/data-loader.md @@ -63,6 +63,7 @@ An example configuration file can be found in ````config/data-loader-config.exam * ````no_confirmation````: Automatically confirms any confirmation prompts that are displayed during the data loading * ````max_violations````: The maximum number of violations (per data file) to be displayed in the console output during data loading * ````no_parents````: Does not save parent node IDs in children nodes +* ````split_transactions````: Splits the database load operations into separate transactions for each file * ````s3_bucket````: The name of the S3 bucket containing the data to be loaded * ````s3_folder````: The name of the S3 folder containing the data to be loaded * ````loading_mode````: The loading mode to be used @@ -157,6 +158,11 @@ All of command line arguments can be specified in the configuration file. If an * Command : ````--no-parents```` * Not Required * Default Value : ````false```` +* **Enable Split Transactions Mode** + * Creates a separate database transactions for each file while loading + * Command : ````--split-transactions```` + * Not Required + * Default Value : ````false```` * **Dataset Directory** * The directory containing the data to be loaded, a temporary directory if loading from an S3 bucket * Command : ````--dataset