Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make lock size required #183

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -422,9 +422,10 @@ There are three properties for customizing this event locking on transmission:
considered for transmission. The default is currently 60 seconds but might change in future releases.

* **lock-size** (events): Defines the maximum amount of events which are loaded into memory and published in one run
(in one submission per event type). By default, *all* events are loaded into memory. In some future release, this
property will become mandatory. This should be set to a value which is not too high so out-of-memory situations
are avoided.
(in one submission per event type). **This property is mandatory** (even if you disable sending events completely).
This should be set to a value which is low enough so out-of-memory situations
are avoided (depending on your typical event sizes), and high enough to allow the needed throughput.
Using the value 0 will disable the limit completely, but this is not recommended.

Example:
```yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ public EventLogWriter eventLogWriter(EventLogRepository eventLogRepository, Obje

@Bean
public EventLogRepository eventLogRepository(NamedParameterJdbcTemplate namedParameterJdbcTemplate,
@Value("${nakadi-producer.lock-size:0}") int lockSize) {
@Value("${nakadi-producer.lock-size}") int lockSize) {
return new EventLogRepositoryImpl(namedParameterJdbcTemplate, lockSize);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,11 @@
@ActiveProfiles("test")
@SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.MOCK,
properties = { "zalando.team.id:alpha-local-testing", "nakadi-producer.scheduled-transmission-enabled:false" },
properties = {
"zalando.team.id:alpha-local-testing",
"nakadi-producer.scheduled-transmission-enabled:false",
"nakadi-producer.lock-size:100"
},
classes = { TestApplication.class, EmbeddedDataSourceConfig.class }
)
public abstract class BaseMockedExternalCommunicationIT {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
@SpringBootTest(properties = {
"nakadi-producer.scheduled-transmission-enabled:false",
"nakadi-producer.lock-duration:300",
"nakadi-producer.lock-duration-buffer:30"})
"nakadi-producer.lock-duration-buffer:30",
"nakadi-producer.lock-size:100"})
public class LockTimeoutIT extends BaseMockedExternalCommunicationIT {
private static final String MY_EVENT_TYPE = "myEventType";

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
webEnvironment = SpringBootTest.WebEnvironment.MOCK,
properties = {
"nakadi-producer.scheduled-transmission-enabled:false",
"nakadi-producer.lock-size:100",
// as we are not defining a mock nakadi client, we need to provide these properties:
"nakadi-producer.encoding:ZSTD",
"nakadi-producer.nakadi-base-uri:http://nakadi.example.com/",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@
@ActiveProfiles("test")
@SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.MOCK,
properties = {"zalando.team.id:alpha-local-testing", "nakadi-producer.scheduled-transmission-enabled:false", "spring.flyway.enabled:false"},
properties = {
"zalando.team.id:alpha-local-testing",
"nakadi-producer.scheduled-transmission-enabled:false",
"spring.flyway.enabled:false",
"nakadi-producer.lock-size:100"
},
classes = {TestApplication.class, EmbeddedDataSourceConfig.class}
)
public class NonNakadiProducerFlywayCallbackIT {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
// no "test" profile, as this would include the mock client.
@SpringBootTest(
webEnvironment = SpringBootTest.WebEnvironment.MOCK,
properties = {"nakadi-producer.submission-enabled:false"},
properties = {"nakadi-producer.submission-enabled:false", "nakadi-producer.lock-size:100"},
classes = { TestApplication.class, EmbeddedDataSourceConfig.class }
)
public class SubmissionDisabledIT {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"management.security.enabled=false",
"zalando.team.id:alpha-local-testing",
"nakadi-producer.scheduled-transmission-enabled:false",
"nakadi-producer.lock-size:100",
"management.endpoints.web.exposure.include:snapshot-event-creation"
},
classes = {TestApplication.class, EmbeddedDataSourceConfig.class, SnapshotEventGenerationWebEndpointIT.Config.class}
Expand Down