Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adds an ORDERED_LIST_STATE capability to the Java SDK. #32067

Merged
merged 3 commits into from
Aug 20, 2024
Merged

Adds an ORDERED_LIST_STATE capability to the Java SDK. #32067

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0b53afd
Adds an ORDERED_LIST_STATE capability to the Java SDK.
acrites Aug 2, 2024
564bdce
Updates CHANGES.md with OrderedListState.
acrites Aug 14, 2024
7947b51
Merge branch 'master' into capabilities
acrites Aug 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
* Go SDK Minimum Go Version updated to 1.21 ([#32092](https://github.com/apache/beam/pull/32092)).
* [BigQueryIO] Added support for withFormatRecordOnFailureFunction() for STORAGE_WRITE_API and STORAGE_API_AT_LEAST_ONCE methods (Java) ([#31354](https://github.com/apache/beam/issues/31354)).
* Updated Go protobuf package to new version (Go) ([#21515](https://github.com/apache/beam/issues/21515)).
* Adds OrderedListState support for Java SDK via FnApi.

## Breaking Changes

Expand Down
4 changes: 4 additions & 0 deletions model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/beam_runner_api.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1681,6 +1681,10 @@ message StandardProtocols {
// the ProcessBundleProgressResponse messages.
SDK_CONSUMING_RECEIVED_DATA = 9
[(beam_urn) = "beam:protocol:sdk_consuming_received_data:v1"];

// Indicates whether the SDK supports ordered list state.
ORDERED_LIST_STATE = 10
[(beam_urn) = "beam:protocol:ordered_list_state:v1"];
}
}

Expand Down
1 change: 1 addition & 0 deletions sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/Environments.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,7 @@ public static Set<String> getJavaCapabilities() {
capabilities.add(BeamUrns.getUrn(Primitives.TO_STRING));
capabilities.add(BeamUrns.getUrn(StandardProtocols.Enum.DATA_SAMPLING));
capabilities.add(BeamUrns.getUrn(StandardProtocols.Enum.SDK_CONSUMING_RECEIVED_DATA));
capabilities.add(BeamUrns.getUrn(StandardProtocols.Enum.ORDERED_LIST_STATE));
acrites marked this conversation as resolved.
Show resolved Hide resolved
return capabilities.build();
}

Expand Down
3 changes: 3 additions & 0 deletions sdks/java/core/src/test/java/org/apache/beam/sdk/util/construction/EnvironmentsTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,9 @@ public void testCapabilities() {
assertThat(
Environments.getJavaCapabilities(),
hasItem(BeamUrns.getUrn(RunnerApi.StandardProtocols.Enum.DATA_SAMPLING)));
assertThat(
Environments.getJavaCapabilities(),
hasItem(BeamUrns.getUrn(RunnerApi.StandardProtocols.Enum.ORDERED_LIST_STATE)));
// Check that SDF truncation is supported
assertThat(
Environments.getJavaCapabilities(),
Expand Down
Loading