Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(batch_imports): report liveness during job init #28375

Merged
merged 2 commits into from
Feb 6, 2025

Conversation

oliverb123
Copy link
Contributor

No description provided.

@posthog-bot
Copy link
Contributor

Hey @oliverb123! 👋
This pull request seems to contain no description. Please add useful context, rationale, and/or any other information that will help make sense of this change now and in the distant Mars-based future.

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR adds a liveness check mechanism during batch import job initialization to prevent false unhealthy states when initializing sinks or sources takes extended time.

  • Added start_init_liveness_loop function in rust/batch-import-worker/src/main.rs that spawns a background task reporting health every 5 seconds
  • Implemented AtomicBool-controlled background task termination using Relaxed memory ordering
  • Added liveness reporting before job initialization and cleanup after completion
  • Integrated with existing health registry system using HealthHandle for status reporting

1 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile

rust/batch-import-worker/src/main.rs Show resolved Hide resolved
@oliverb123 oliverb123 merged commit 44aa2af into master Feb 6, 2025
84 checks passed
@oliverb123 oliverb123 deleted the olly_batch_imports_job_init_healthcheck branch February 6, 2025 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants