Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul arch-split in machine, ASpec and AInvs #805

Merged
merged 12 commits into from
Aug 8, 2024

Conversation

Xaphiosis
Copy link
Member

@Xaphiosis Xaphiosis commented Jul 26, 2024

In #791 I went over AInvs for AARCH64 only. I've since expanded the approach to ASpec as well, and deployed it across all the architectures. The speed benefits still apply, and now ASpec is faster as well. I'll see if I can do a better benchmark, or we can pull it out of the regression tests.

🦆🦆🦆 I have put a lot of effort into splitting up the commits into being topical, because this is a rather large PR. They should not overlap much, so if you review a line in the commit view it's unlikely to be changed in another commit.

@mbrcknl @michaelmcinerney FYI in case you are interested in the new arch-splitting tech/approach.

  • add FIXME arch_split to PR linter to stop people reintroducing it
  • remove apply_trace I accidentally left in (proof/invariant-abstract/X64/ArchCSpace_AI.thy in ainvs: disambiguate acap_rights_update_id)

It was a mix of ARM_A and ARM_HYP_A previously. AInvs+Refine+CRefine
needed updates.

Signed-off-by: Rafal Kolanski <[email protected]>
@Xaphiosis Xaphiosis added enhancement cleanup proof engineering nicer, shorter, more maintainable etc proofs arch-split splitting proofs into generic and architecture dependent labels Jul 26, 2024
@Xaphiosis Xaphiosis requested review from lsf37 and corlewis July 26, 2024 03:47
Copy link
Member

@lsf37 lsf37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice improvement in consistency and reduction of interpretations. I only spotted a few old FIXMEs that can be removed, otherwise ready to merge from my side (when the apply_trace is removed, the linter can be in a separate PR if you prefer).

(already in Arch locale)

Signed-off-by: Rafal Kolanski <[email protected]>
Implicitly use L4V_ARCH for global_naming prefix.

Signed-off-by: Rafal Kolanski <[email protected]>
* try get rid of `interpretation Arch .` (slow) in lieu of `(in Arch)`
  (faster) or proper requalifying (nearly instant)
* get rid of requalifications that were already requalified, or were
  global (thanks to new warnings)
* put arch_global_naming on same line as `context Arch begin`

Signed-off-by: Rafal Kolanski <[email protected]>
* get rid of `global_naming Arch`, this is no longer needed since we can
  requalify directly from `arch_global_naming` with `arch_requalify`
  commands
* add missing `arch_global_naming` for `context Arch`, for consistency
* update X64 Refine `Arch.` references (other arches did not need this)

Signed-off-by: Rafal Kolanski <[email protected]>
@Xaphiosis
Copy link
Member Author

Very nice improvement in consistency and reduction of interpretations. I only spotted a few old FIXMEs that can be removed, otherwise ready to merge from my side (when the apply_trace is removed, the linter can be in a separate PR if you prefer).

Thanks for spotting those FIXMEs, I cleared them out (and some others) in a separate commit, if you want to check that.
Fixed the apply_trace as well.
The linter, yes, probably in a separate PR, I don't know how to do that yet.

Copy link
Member

@corlewis corlewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, but the first commit message could mention spec being updated as well.
Ignore this, I confused myself.

Copy link
Member

@corlewis corlewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice to see this improved consistency. Thank you for the efforts in splitting up the commits, that made it a lot nicer to review.

by (simp add: X64.switchFpuOwner_def Arch.no_fail_machine_op_lift)
by (simp add: X64.switchFpuOwner_def X64.no_fail_machine_op_lift)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why is this qualified at all? It seems like it would be more consistent with other architectures (and other uses in X64) if it wasn't.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess looking forward we can consider whether this will be requalified when we arch_split further, that would influence how we want to make this consistent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well spotted! Probably someone getting overzealous with adding qualification, there's indeed no need for it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like removing this qualification breaks the X64 proof. I will investigate why.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, this isn't an issue with ASpec/AInvs, this is someone doing an Arch-dependent proof outside of the Arch context, because why not. Will revert this "fix" and leave it for proper arch-splitting of Refine.

Comment on lines 607 to 608
(* is this the right way? we need this fact globally but it's proven with
ARM defns. *)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor, but it looks like you've removed this comment on most architectures but left it for this one. Was that intentional?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, thanks

Comment on lines +22 to +27
arch_finalise_cap_bcorres
prepare_thread_delete_bcorres

lemmas aobj_ref_arch_cap_simps[simp] = aobj_ref_arch_cap

lemmas [wp] = arch_finalise_cap_bcorres prepare_thread_delete_bcorres
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why were these added to LevityCatch_AI? Is it that there's nowhere better for them? I would have thought that we would like to minimise its content, with it mainly containing lemmas that could be moved elsewhere.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely a derp. Would this be acceptable instead?

           apply (simp add: select_ext_fa[simplified free_asid_select_def]
                            free_asid_select_def o_def returnOk_liftE[symmetric]
                       split del: if_split)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure, I guess this was for the conversation just below?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, right, yes. Sorry, I missed this one as well!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to have these in a generic theory file, but couldn't find one that was in the import hierarchy in time for these to be useful. I'll take another look while I'm fixing X64, but I don't know if I'll find anyplace better. Advice welcome.

free_asid_select_def o_def returnOk_liftE[symmetric]
split del: if_split)
free_asid_select_def o_def returnOk_liftE[symmetric] split del: if_split)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this, the previous indentation looks better.

@corlewis
Copy link
Member

corlewis commented Aug 6, 2024

Some good looking benchmarks for this

master:
  N = 10
  Average ARM ASpec: (00:01:16 real, 00:02:44 cpu, 4.10GB)
  Standard Deviation: (00:00:01 real, 00:00:03 cpu, 0.31GB)
arch-split_ainvs:
  N = 10
  Average ARM ASpec: (00:01:06 real, 00:02:34 cpu, 3.96GB)
  Standard Deviation: (00:00:01 real, 00:00:03 cpu, 0.22GB)
Percentage change: -13.2%

master:
  N = 10
  Average ARM AInvs: (00:07:44 real, 00:40:56 cpu, 12.97GB)
  Standard Deviation: (00:00:18 real, 00:01:41 cpu, 2.82GB)
arch-split_ainvs:
  N = 10
  Average ARM AInvs: (00:07:01 real, 00:37:48 cpu, 14.23GB)
  Standard Deviation: (00:00:16 real, 00:01:28 cpu, 3.30GB)
Percentage change: -9.3%
master:
  N = 10
  Average ARM_HYP ASpec: (00:01:54 real, 00:03:47 cpu, 5.75GB)
  Standard Deviation: (00:00:02 real, 00:00:05 cpu, 0.78GB)
arch-split_ainvs:
  N = 10
  Average ARM_HYP ASpec: (00:01:34 real, 00:03:22 cpu, 5.91GB)
  Standard Deviation: (00:00:02 real, 00:00:05 cpu, 0.49GB)
Percentage change: -17.6%

master:
  N = 10
  Average ARM_HYP AInvs: (00:11:08 real, 00:51:48 cpu, 14.40GB)
  Standard Deviation: (00:00:12 real, 00:01:04 cpu, 2.60GB)
arch-split_ainvs:
  N = 10
  Average ARM_HYP AInvs: (00:09:57 real, 00:46:44 cpu, 13.04GB)
  Standard Deviation: (00:00:10 real, 00:00:51 cpu, 0.87GB)
Percentage change: -10.5%
master:
  N = 10
  Average X64 ASpec: (00:01:44 real, 00:03:31 cpu, 4.21GB)
  Standard Deviation: (00:00:01 real, 00:00:02 cpu, 0.56GB)
arch-split_ainvs:
  N = 10
  Average X64 ASpec: (00:01:26 real, 00:03:09 cpu, 4.30GB)
  Standard Deviation: (00:00:01 real, 00:00:01 cpu, 0.62GB)
Percentage change: -17.7%

master:
  N = 10
  Average X64 AInvs: (00:10:12 real, 00:53:56 cpu, 14.59GB)
  Standard Deviation: (00:00:19 real, 00:01:44 cpu, 0.88GB)
arch-split_ainvs:
  N = 10
  Average X64 AInvs: (00:08:52 real, 00:49:04 cpu, 12.56GB)
  Standard Deviation: (00:00:15 real, 00:01:17 cpu, 0.49GB)
Percentage change: -13.1%
master:
  N = 10
  Average RISCV64 ASpec: (00:01:22 real, 00:03:01 cpu, 5.00GB)
  Standard Deviation: (00:00:00 real, 00:00:02 cpu, 0.57GB)
arch-split_ainvs:
  N = 10
  Average RISCV64 ASpec: (00:01:09 real, 00:02:44 cpu, 5.27GB)
  Standard Deviation: (00:00:01 real, 00:00:03 cpu, 0.77GB)
Percentage change: -16.4%

master:
  N = 10
  Average RISCV64 AInvs: (00:06:35 real, 00:37:22 cpu, 13.06GB)
  Standard Deviation: (00:00:14 real, 00:01:17 cpu, 2.82GB)
arch-split_ainvs:
  N = 10
  Average RISCV64 AInvs: (00:05:56 real, 00:34:32 cpu, 12.09GB)
  Standard Deviation: (00:00:11 real, 00:01:08 cpu, 2.93GB)
Percentage change: -10.1%
master:
  N = 10
  Average AARCH64 ASpec: (00:01:54 real, 00:03:53 cpu, 5.71GB)
  Standard Deviation: (00:00:01 real, 00:00:04 cpu, 1.28GB)
arch-split_ainvs:
  N = 10
  Average AARCH64 ASpec: (00:01:33 real, 00:03:27 cpu, 5.03GB)
  Standard Deviation: (00:00:02 real, 00:00:04 cpu, 0.92GB)
Percentage change: -18.3%

master:
  N = 10
  Average AARCH64 AInvs: (00:09:54 real, 00:49:26 cpu, 16.72GB)
  Standard Deviation: (00:00:32 real, 00:03:05 cpu, 1.53GB)
arch-split_ainvs:
  N = 10
  Average AARCH64 AInvs: (00:08:30 real, 00:47:07 cpu, 14.01GB)
  Standard Deviation: (00:00:17 real, 00:01:29 cpu, 1.20GB)
Percentage change: -14.1%

@Xaphiosis
Copy link
Member Author

@corlewis I addressed your comments, hopefully, with the latest commits (they're named after the commit I'm supposed to squash them into)
Thank you so much for the proper benchmarking! My own sad effort showed ARM_HYP being slower in AInvs, which was a disappointment. Speaking of disappointments, do you have any guesses why ARM got comparatively poorer improvements? I didn't notice anything relevant during this pass... context begin interpretation Arch . is now removed completely from spec/abstract and proof/invariant-abstract, and that was the bulk of the speedup.

@corlewis
Copy link
Member

corlewis commented Aug 8, 2024

Speaking of disappointments, do you have any guesses why ARM got comparatively poorer improvements? I didn't notice anything relevant during this pass... context begin interpretation Arch . is now removed completely from spec/abstract and proof/invariant-abstract, and that was the bulk of the speedup.

My only guess is based on the fact that ARM seems to have the smallest spec and proof. Maybe that means its arch-specific files are smaller, and that interpreting its Arch locale wasn't as slow as compared to the other arches?

@Xaphiosis
Copy link
Member Author

Speaking of disappointments, do you have any guesses why ARM got comparatively poorer improvements? I didn't notice anything relevant during this pass... context begin interpretation Arch . is now removed completely from spec/abstract and proof/invariant-abstract, and that was the bulk of the speedup.

My only guess is based on the fact that ARM seems to have the smallest spec and proof. Maybe that means its arch-specific files are smaller, and that interpreting its Arch locale wasn't as slow as compared to the other arches?

My impression was that RISCV64 is the smallest?

* prevent duplication between arches by moving requalifications from
  Arch theories to generic ones
  * this strategy is not available for new constants that are introduced
    in the Arch locale that need to be referenced in generic definitions
    or locale instantiations
* fix up the inconsistent qualification of set_cap_valid_arch_caps_simple

Signed-off-by: Rafal Kolanski <[email protected]>
Rename the wellformed_acap version to wf_acap_rights_update_id,
and the valid_arch_cap version to valid_acap_rights_update_id.

Signed-off-by: Rafal Kolanski <[email protected]>
Avoid unnecessary Arch context interpretation (slow).
Remove attempts at requalifying entities that are already generic
(misleading).

Signed-off-by: Rafal Kolanski <[email protected]>
Makes global_naming implicit (based on L4V_ARCH).

Signed-off-by: Rafal Kolanski <[email protected]>
This was generic on these two architectures, not consistent with others.

Signed-off-by: Rafal Kolanski <[email protected]>
MiscMachine_A already contains asid_high_bits, asid_low_bits and
asid_bits. Other architectures don't duplicate them in Arch_Structs_A,
so ARM and ARM_HYP shouldn't either.

For ARM, this also means fixing up DRefine+DPolicy+CamkesCdlRefine to
refer to the MiscMachine_A definitions when needed (CapDL duplicates
them again in Types_D, but as they don't import machine theories, those
can't be removed).

Signed-off-by: Rafal Kolanski <[email protected]>
These proofs have been arch-split for some time now so the FIXMEs appear
resolved. They are also non-specific, so it is impossible to tell at
this point whether they referred to anything.

Signed-off-by: Rafal Kolanski <[email protected]>
@Xaphiosis
Copy link
Member Author

Squashed

@corlewis
Copy link
Member

corlewis commented Aug 8, 2024

Speaking of disappointments, do you have any guesses why ARM got comparatively poorer improvements? I didn't notice anything relevant during this pass... context begin interpretation Arch . is now removed completely from spec/abstract and proof/invariant-abstract, and that was the bulk of the speedup.

My only guess is based on the fact that ARM seems to have the smallest spec and proof. Maybe that means its arch-specific files are smaller, and that interpreting its Arch locale wasn't as slow as compared to the other arches?

My impression was that RISCV64 is the smallest?

Sorry, by smallest I meant quickest out of the ones I had tested. I'll also test RISCV64 to see how it compares, I didn't get around to it before.

@corlewis
Copy link
Member

corlewis commented Aug 8, 2024

I'll also test RISCV64 to see how it compares, I didn't get around to it before.

I've added RISCV64 results to my earlier comment. Like you thought, it's the shortest/quickest AInvs. I don't know how much to read into it, but it does also show one of the smallest improvements.

@Xaphiosis Xaphiosis merged commit 2545cfe into seL4:master Aug 8, 2024
13 of 14 checks passed
@Xaphiosis Xaphiosis deleted the arch-split_ainvs branch August 8, 2024 08:05
@Xaphiosis
Copy link
Member Author

I'll also test RISCV64 to see how it compares, I didn't get around to it before.

I've added RISCV64 results to my earlier comment. Like you thought, it's the shortest/quickest AInvs. I don't know how much to read into it, but it does also show one of the smallest improvements.

Cool, thank you! The improvement on it still beat ARM somehow, especially in ASpec. I remain puzzled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-split splitting proofs into generic and architecture dependent cleanup enhancement proof engineering nicer, shorter, more maintainable etc proofs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants