Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TaskMatcher rewrite [very wip] #7196

Draft
wants to merge 1 commit into
base: priority
Choose a base branch
from
Draft

TaskMatcher rewrite [very wip] #7196

wants to merge 1 commit into from

Conversation

dnr
Copy link
Member

@dnr dnr commented Jan 31, 2025

What changed?

  • Rewrite TaskMatcher to use explicit queues (to be extended into priority queues).
  • taskReader pushes all tasks into TaskMatcher, and loads more when the number outstanding is too low.

This is not quite ready to merge, and there's at least one significant issue left to fix, but it passes most functional tests. I'm just making a PR to refer to it.

Why?

  • Ability to prioritize uniformly across all pending tasks including queries+nexus.
  • Simpler architecture, no new component between taskReader and TaskMatcher.
  • More separated concerns, e.g. "forwarding" is (mostly) in one place instead of spread around, "rate limiting" is in one place, etc.
  • Easier to understand code, no more nested selects (this is very subjective of course).
  • Maybe better performance (after optimizations).

How did you test it?

existing functional tests, need to update unit tests

Potential risks

lots of new code, may be new bugs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant