Skip to content

Fix verification of tracked struct from high-durability query #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 2, 2024

Conversation

carljm
Copy link
Collaborator

@carljm carljm commented Aug 1, 2024

The test models a situation where we have two files (0, 1), where file 0 has LOW durability and file 1 has HIGH durability. We can query an index for each file, and a definitions from that index (just a sub-part of the index), and we can infer each file. The index and definitions queries depend only on the File they operate on, but the infer query has some other dependencies: infer(0) depends on infer(1), and infer(1) also depends directly on File(0) (this is to model that resolving a module can depend on existence of various files at various durabilities.) The dependency graph is shown below: light color represents low durability, black represents high durability, dark gray are structs and struct fields; I didn't get around to adding their durability to the graph yet. (The diff used to automatically generate the graph is included in the PR too; if we hit many more bugs like this I'll probably clean this code up and try to make it generally reusable.)

Screenshot 2024-08-01 at 10 19 02 PM

The panic occurs because definitions(1) is high durability, and depends on index(1) which is also high durability. index(1) creates the tracked struct Definition(1), and infer(1) (which is low durability) depends on Definition.file(1).

After a change to file 0 (low durability), we only shallowly verify definitions(1) -- it doesn't need deep-verify because it is high durability. So that means we never verify index(1) at all (deeply or shallowly), which means we never mark Definition(1) validated. So when we deep-verify infer(1), we try to access its dependency Definition.file(1), and hit the panic because we are accessing a tracked struct that has never been re-validated or re-recreated in R2.

@carljm
Copy link
Collaborator Author

carljm commented Aug 2, 2024

I think one possible fix is that when we short circuit shallow verify due to durability, we also need to mark-validated not just our own outputs, but also all outputs of all our input queries, recursively (which must also be at least as durable as we are, and which we are never going to verify at all (even shallowly) due to the short circuit.)

@carljm
Copy link
Collaborator Author

carljm commented Aug 2, 2024

I think it's either that, or we modify maybe_changed_after on a tracked struct so that it is valid to call it even if it hasn't been created/validated in the current revision yet, and we first check its durability and maybe validate it. This might be the simpler approach, and I think it's also correct.

@carljm carljm force-pushed the struct-panic branch 2 times, most recently from a82faac to c954e8c Compare August 2, 2024 05:22
@carljm
Copy link
Collaborator Author

carljm commented Aug 2, 2024

I've adjusted the test slightly so that it doesn't require input-builders and can run against main-branch Salsa (it still fails there), and I've got a working fix (using the second approach) in salsa-rs#550

@MichaReiser
Copy link
Owner

Let's create a PR against this repo as well so that we can go ahead and merge it without having to wait for upstream salsa

@carljm
Copy link
Collaborator Author

carljm commented Aug 2, 2024

Updated to match salsa-rs#550

@carljm carljm changed the base branch from input-builder to master August 2, 2024 16:50
@carljm carljm changed the title test showing the "access tracked struct from previous revision" bug Fix verification of tracked struct from high-durability query Aug 2, 2024
@carljm carljm merged commit 1ea01bd into MichaReiser:master Aug 2, 2024
@carljm carljm deleted the struct-panic branch August 2, 2024 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants