Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some more changes to reduce adjacent server flakes #7917

Merged
merged 1 commit into from
Oct 23, 2024

Conversation

msullivan
Copy link
Member

  1. Make worker threads (like net) not attempt new connections
    once the server has started shutting down.
  2. Don't respond to messages that prompt updating database
    metadata for database's we don't have introspected anyway.
  3. Mark a lot of worker tasks as being interruptable. When they
    weren't interruptable, server exit had to wait for them to
    exit, which they didn't always do. This caused connections
    to get held open.

Most of the locally observable flakes I've been seeing were fixed by
these changes, though I am still getting
test_server_adjacent_database_propagation failures where it fails to
connect to the newly create database. Still working on that.

@msullivan
Copy link
Member Author

I have never been happier in my life to have had a lint error. When I saw one test suite failed I assumed it was the flake again.

1. Make worker threads (like net) not attempt new connections
   once the server has started shutting down.
2. Don't respond to messages that prompt updating database
   metadata for database's we don't have introspected anyway.
3. Mark a lot of worker tasks as being interruptable. When they
   weren't interruptable, server exit had to wait for them to
   exit, which they didn't always do. This caused connections
   to get held open.

*Most* of the locally observable flakes I've been seeing were fixed by
these changes, though I am still getting
test_server_adjacent_database_propagation failures where it fails to
connect to the newly create database. Still working on that.
@msullivan msullivan merged commit 7c0c2a8 into master Oct 23, 2024
23 checks passed
@msullivan msullivan deleted the flake-fighting-1 branch October 23, 2024 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants