Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Processing time timers are fired after the window closed in Dataflow #32727

Open
1 of 17 tasks
baeminbo opened this issue Oct 10, 2024 · 0 comments
Open
1 of 17 tasks

Comments

@baeminbo
Copy link
Contributor

baeminbo commented Oct 10, 2024

What happened?

The pipeline has a DoFn to set a processing time timer in 30 seconds for each input element and add the input value to a bag state. The source generates an element every second, and the window size is 60s with 20s lateness allowed.

In a test with Dataflow Java Legacy Runner, I expected the processing timer wouldn't be fired since the window was closed after the late element arrived. But, actually, the processing timer was fired and the state was empty.

AdyeWWofQq3z3b7
B63TUQhPxNgGi23

I believe the timers should be cleaned up, the same as states are cleaned up.

FYI. This issue also happens in Runner v2. b/366283709

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant