Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Evaluator Closeable #3600

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

alexarchambault
Copy link
Contributor

Cleaning-up the worker cache upon close

@alexarchambault
Copy link
Contributor Author

I decided to open a PR for that for #3579 (#3579 (comment) in particular), but I'm now realizing this might not be needed for it after all. Don't know what you think of that, I think it's a good idea to clean up things when the Evaluator isn't needed anymore.

Cleaning-up the worker cache upon close
@alexarchambault
Copy link
Contributor Author

(This makes Evaluator extends AutoCloseable, so this breaks bin compat…)

@alexarchambault alexarchambault marked this pull request as draft September 24, 2024 22:18
@lihaoyi
Copy link
Member

lihaoyi commented Sep 24, 2024

I'm not sure this is the right thing to do. Right now in production, the workerCache is this long lived thing that is shared across main0 invocations, and I think can out-live the individual evaluator that created it to be used in the next Mill command's evaluator. If that's the case, then having the workerCache be closed together with the evaluator at end of life would be wrong.

That code around sharing stuff across main0 commands is pretty gnarly, but that's what I last remember it being like

@alexarchambault
Copy link
Contributor Author

alexarchambault commented Sep 25, 2024

It seems this works at the end (CI is green), as in the watch loop, the previous RunnerState isn't closed, so that underlying Evaluators aren't either, and workerCaches are kept intact and can keep being used. But I don't think I did that on purpose, that's more an oversight 😅

@lihaoyi
Copy link
Member

lihaoyi commented Sep 26, 2024

Yeah that's why it's green, and then the next question is more what should the logic be here? I don't actually know, but although the current logic of injecting the workerCache into the Evaluator is pretty weird and roundabout, I'm inclined to leave it for now. I suspect that refactoring things enough to do things "properly" will require breaking bincompat, so we should wait until after 0.12.0 has landed and we're working towards 0.13.x and have the flexibility to make more sweeping changes without being restricted by bincompat requirements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants