Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: DynamoDB Recommendation of offset batches #1263

Merged
merged 1 commit into from
Nov 22, 2024
Merged

docs: DynamoDB Recommendation of offset batches #1263

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ...tion-dynamodb-integration/src/test/java/projection/docs/javadsl/ProjectionDocExample.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ public void atLeastOnceExample(

Optional<DynamoDBProjectionSettings> settings = Optional.empty();

int saveOffsetAfterEnvelopes = 100;
int saveOffsetAfterEnvelopes = 25;
Duration saveOffsetAfterDuration = Duration.ofMillis(500);

Projection<EventEnvelope<ShoppingCart.Event>> projection =
Expand All @@ -407,7 +407,7 @@ public void exactlyOnceGroupedWithinExample(

Optional<DynamoDBProjectionSettings> settings = Optional.empty();

int groupAfterEnvelopes = 20;
int groupAfterEnvelopes = 25;
Duration groupAfterDuration = Duration.ofMillis(500);

Projection<EventEnvelope<ShoppingCart.Event>> projection =
Expand All @@ -433,7 +433,7 @@ public void atLeastOnceGroupedWithinExample(

Optional<DynamoDBProjectionSettings> settings = Optional.empty();

int groupAfterEnvelopes = 20;
int groupAfterEnvelopes = 25;
Duration groupAfterDuration = Duration.ofMillis(500);

Projection<EventEnvelope<ShoppingCart.Event>> projection =
Expand Down
15 changes: 15 additions & 0 deletions docs/src/main/paradox/dynamodb.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,16 @@ Scala

By default, Akka Projection DynamoDB shares the @extref:[DynamoDB client configuration](akka-persistence-dynamodb:config.html#dynamodb-client-configuration) with Akka Persistence DynamoDB.

### Batch writes

Offsets are written in batches for @ref:[at-least-once](#at-least-once) and @ref:[at-least-once (grouped)](#at-least-once-grouped-).
To reduce the risk of write throttling it is recommended to save at most 25 offsets at a time. This is configured by:

```hcon
akka.projection.at-least-once.save-offset-after-envelopes = 25
akka.projection.grouped.group-after-envelopes = 25
```

### Reference configuration

The following can be overridden in your `application.conf` for projection specific settings:
Expand Down Expand Up @@ -164,6 +174,8 @@ defined in the configuration section `akka.projection.at-least-once`. There is a
offset too often, but the drawback is that there can be more duplicates that will be processed again when the
projection is restarted.

Offsets are written in batches. To reduce the risk of write throttling it is recommended to save at most 25 offsets at a time.

The @ref:[`ShoppingCartHandler` is shown below](#generic-handler).

## exactly-once (grouped)
Expand Down Expand Up @@ -204,6 +216,9 @@ Java
Scala
: @@snip [at least once grouped within](/akka-projection-dynamodb-integration/src/test/scala/projection/docs/scaladsl/ProjectionDocExample.scala) { #at-least-once-grouped-within }

Offsets are written immediately after the group of envelopes has been processed. To reduce the risk of write throttling
it is recommended to save at most 25 offsets at a time, and therefore not exceed this for the group size.

The @ref:[`GroupedShoppingCartHandler` is shown below](#grouped-handler).

## Handler
Expand Down
Loading