Skip to content

Commit

Permalink
Removal of experimental notification and inclusion in functionality.
Browse files Browse the repository at this point in the history
  • Loading branch information
rtjd6554 committed Nov 20, 2024
1 parent 58a0115 commit d7f45b6
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 1 deletion.
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ stack.
- Persistent EMR bulk import stack: Similar to the above stack, but the EMR cluster is persistent, i.e. it never
shuts down. This is appropriate if there is a steady stream of import jobs. The cluster can either be of fixed
size or it can use EMR managed scaling.
- EMR Serverless Bulk Import stack: Similar to the above 2 stacks in behaviour. This stack is created at Sleeper
instance deployment. It is the default way in which the bulk imports are run and provides benefit by the fact that
when no bulk import jobs are
- Dashboard stack: This displays properties of the system in a Cloudwatch dashboard.

The following functionality is experimental:
Expand Down
1 change: 0 additions & 1 deletion docs/05-ingest.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,6 @@ There are several stacks that allow data to be imported using the bulk import pr
- `EmrServerlessBulkImportStack` - this causes an EMR Serverless application to be created when the Sleeper instance is deployed.
This is the default EMR Bulk Import Stack. The advantage of using EMR Serverless is that when there are no bulk import jobs
the applications stops with no wasted compute. The startup of the application is greatly reduced compared to standard EMR.
This stack is experimental.
- `EmrBulkImportStack` - this causes an EMR cluster to be deployed each time a job is submitted to the EMR bulk import
queue. Each job is processed on a separate EMR cluster. The advantage of the cluster being used for one job and then
destroyed is that there is no wasted compute if jobs are submitted infrequently. The downside is that there is a
Expand Down

0 comments on commit d7f45b6

Please sign in to comment.