Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve pickNextTask #410

Merged
merged 10 commits into from
Aug 9, 2024
Merged

Improve pickNextTask #410

merged 10 commits into from
Aug 9, 2024

Conversation

samholmes
Copy link
Contributor

@samholmes samholmes commented Jul 31, 2024

The goal of this PR is to improve the readablity and maintainability of the pickNextTask routine in UtxoEngineProcessor. This is done by removing unnecessary code decoupling and bringing routine code closer together while fixing any issues discovered during this audit. And last but not least, replacing Deferred promises with the more appropriate use of generators.

Note to reviewer

I'm not certainly happy with how the types are written for the generator implementation, so please scrutinize that aspect. Additionally, I haven't tested the BlockbookElectrum refactor, so that is an area to look closely at during review.

CHANGELOG

Does this branch warrant an entry to the CHANGELOG?

  • Yes
  • No

Dependencies

none

Description

none

This allows us to simplify UtxoEngineProcessor and includes server
scoring in places where it was missed. Now all WebSocket tasks are
scored using a single implementation so long as the WsTask is created
by one of the ServerStates methods.
There is no point to calculating this on every pickNextTask.
This change simplifies the processor arguments and removes unnecessary
CPU usage on every pickNextTask. More importantly, it removes the extra
memory usage by allocating a boolean to be GC'd after pickNextTask pops
from the stack.
All in all, this change seems like a win.
This is to be explicit about the pickNextTask routine.
This is dead code, and it makes sense to let the onQueueSpace callback
function be defined only in the config during instantiation.
This cleans up the pickNextTask function and also removes redundant
error handling for each "process" function.
@samholmes samholmes marked this pull request as ready for review August 6, 2024 21:38
Copy link
Contributor

@swansontec swansontec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to approve this and let QA pick up the pieces. These dynamic state machines are hard to reason about, so the best approach is to just run it.

Regarding the types, my general rule is to replace any with unknown if you don't know / don't care what something is. If you do this across the board, one of two things will happen. In the good case, there are no type errors, because nobody actually reads these unknown values. In the less-good case, you will have type errors in a handful of places that actually read the unknown values. For these places use an as any to cast the unknown value, and add a comment explaining why the conversion is safe. Or use something like a cleaner or type assertion to make it explicit (like (x: unknown): x is Foo => ...). The idea is to concentrate the type badness into one place, so the rest of the codebase is clean.

So basically, just replace WsTask<any> with WsTask<unknown> and see what happens.

src/common/utxobased/engine/ServerStates.ts Outdated Show resolved Hide resolved
src/common/utxobased/engine/ServerStates.ts Outdated Show resolved Hide resolved
@samholmes
Copy link
Contributor Author

I am going to approve this and let QA pick up the pieces. These dynamic state machines are hard to reason about, so the best approach is to just run it.

Regarding the types, my general rule is to replace any with unknown if you don't know / don't care what something is. If you do this across the board, one of two things will happen. In the good case, there are no type errors, because nobody actually reads these unknown values. In the less-good case, you will have type errors in a handful of places that actually read the unknown values. For these places use an as any to cast the unknown value, and add a comment explaining why the conversion is safe. Or use something like a cleaner or type assertion to make it explicit (like (x: unknown): x is Foo => ...). The idea is to concentrate the type badness into one place, so the rest of the codebase is clean.

So basically, just replace WsTask<any> with WsTask<unknown> and see what happens.

I'll go ahead an poke around with the type checking by replacing any with unknown to see if anything rears its head.

As far as testing, I've tested sync over a normal circumstances. Though, the BlockbookElectrum changes haven't been tested and I'm not certain if I did those changes correctly. I'll have to look closer at that piece.

I'm also wondering if there is a type alias that I can define that would harden the type checking throughout the layers.

Copy link
Contributor

@swansontec swansontec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That came out pretty nice.

@samholmes samholmes enabled auto-merge August 9, 2024 20:21
@samholmes samholmes merged commit 4a6708c into master Aug 9, 2024
2 checks passed
@swansontec swansontec deleted the sam/pickNextTask branch August 12, 2024 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants