[Upgrades] Support preparing upgrade while still up

[msullivan]

This requires splitting up "preparing" the upgrade (which populates
the new namespaces) and "finalizing" the upgrade (which flips all the
trampolines and performs schema fixups).

This also does some more work towards making the upgrade recoverable
(by running the finalizations in transactions, and not committing any until
all are ready), but there are still important TODOs on that front.

Progress on #6697.

[elprans]

We should probably also add a way to abort/clean-up non-finalized upgrades?

[msullivan]

We should probably also add a way to abort/clean-up non-finalized upgrades?

Yeah, that will be in one of the next follow-ups.

The tests are failing with "out of shared memory" while trying to finalize on github actions. Probably holding open all these huge transactions won't actually fly.
I'm just removing that mechanism for now, and will address the issue in a follow-up.

I think the new plan will be to try each finalization migration in a transaction that we rollback, and then once they have been validated we will apply them all. (We could maybe support both modes, too?) I'll also make it so that we compile the script for it in the preparation phase and stash it in the database, so we don't have any compiler overhead in the finalization critical section.