Issue 3070 - Wait for state store commits to finish based on logs #3063
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
patchwork01
added
one-review-required
pr-stacked-top
…ed-state-store-committer-throughput
patchwork01
removed
the
pr-stacked-top
patchwork01
changed the title
|
The system test is failing because the instance admin role doesn't have permissions to query the logs. That should be easy to fix. |
rtjd6554
reviewed
Aug 14, 2024
| case COMPACTION_JOB_ID_ASSIGNMENT: | ||
| return StateStoreCommitRequest.forCompactionJobIdAssignment( | ||
| context.deserialize(requestObj, CompactionJobIdAssignmentCommitRequest.class)); | ||
| case SPLIT_PARTITION: | ||
| return StateStoreCommitRequest.forSplitPartition( | ||
| context.deserialize(requestObj, SplitPartitionCommitRequest.class)); | ||
| case STORED_IN_S3: |
There was a problem hiding this comment.
Any reason for the re-ordering of the case statements?
There was a problem hiding this comment.
I felt like the stored in S3 type should be separate from the others because it's handled differently. I'm not sure there's a good way to make this clear though.
rtjd6554
reviewed
Aug 14, 2024
...drivers/src/main/java/sleeper/systemtest/drivers/statestore/StateStoreCommitterLogEntry.java
Outdated
Show resolved
Hide resolved
rtjd6554
reviewed
Aug 14, 2024
| @@ -129,7 +130,10 @@ private String getMessageGroupId(Message message) { | |||
| } | |||
|
|
|||
| private StateStoreCommitRequest readCommitRequest(Message message) { | |||
| return new StateStoreCommitRequestDeserialiser(instance.getTablePropertiesProvider()) | |||
| LoadS3ObjectFromDataBucket noLoadFromS3 = key -> { | |||
| throw new UnsupportedOperationException("Did not expect message held in S3"); | |||
There was a problem hiding this comment.
Language for message here sounds wrong, not able to think of better suggestion though
There was a problem hiding this comment.
I've just made it actually load from S3, thanks.
rtjd6554
reviewed
Aug 14, 2024
rtjd6554
approved these changes
Aug 14, 2024
patchwork01
deleted the
3060-measure-deployed-state-store-committer-throughput
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make sure you have checked all steps below.
Issue
PR"
Tests
Documentation
separate issue for that below.
NOTICES file to reflect this.