Help with major protocol version increments in testnets

[nfrisby]

Consensus has recently identified a feature of the Hard Fork Combinator as a mistake: the configuration of a (test) node can schedule era transitions to happen in certain epochs. Unfortunately, some developers already rely on that misfeature in their test suites.

I see three options (listed in this relatively short comment on ouroboros-consensus Issue 416 IntersectMBO/ouroboros-consensus#416 (comment)), but they all require extra work from someone.

• In my current opinion, the best long-term solution is to entirely remove this misfeature. However, that will require these test suite authors to do the extra work of introducing the necessary transactions at the necessary time and also preparing the ledger state accordingly. I imagine that Ledger could provide some functions that might make it as easy as possible to prepare the initial ledger state and the relevant transactions, though the test suites would still have to arrange for those transactions to arrive at the right time.

• A trade-off solution would be to upstream the existing feature from the HFC into the Ledger: the test node configuration data would instead schedule increments of the major protocol version (which the HFC would interpret the same as always). But the Ledger team might still consider that a misfeature for their own reasons?

This Issue is to invent and provide the necessary interface for one or both of the above options. (If Ledger rejects the trade-off solution, then we're blocked on the various test suite owners also being in the conversation: accepting the longer-term solution and helping us determine what interface would actually benefit them.)

[lehins]

This is a very interesting problem. I do agree with you on all the points that consensus should not be mocking with internals of the ledger state in order to accommodate ability to hard fork into some era, because it would be very fragile and cumbersome.
I also agree that Ledger should somehow accommodate such a use case, since it does seem useful for testing of different components and downstream tools and because it is ultimately that is ledger that normally dictates when the hard fork is initiated. I am just not sure of a good solution in mind at this moment.

So, I am pretty confident that this feature should definitely work like this:

• Any test net has a capability to submit a very light way request to ledger to bump the major protocol version at a particular epoch boundary
• If this protocol version bump results in a transition to the next era that is great, otherwise that would become an intra era hard fork and another request can be submitted until the desired era is reached.

Currently increasing the protocol version is extremely complicated and varies between eras, so I don't see it being a viable approach to solve this problem. Here are some ideas that come to mind that could serve as potential solutions, which I am not 100% sold on:

1. Ability to supply through Shelley genesis file a setting that would indicate to the Ledger at which epochs number should ledger transition to which protocol versions.
2. Specify an admin credential in the configuration file that could submit special type of transactions that could do administrative stuff like initialize a hard fork or update a protocol parameter at will.

Naturally, these features would be disabled for Mainnet, similarly to how we have injection of initial staking credentials through a genesis file as a feature that is disabled for Mainnet.

1st solution is a bit too static IMHO and would interfere with natural flow of ledger rules (eg. bumping protocol parameter at an arbitrary point will mess up any HardForkInitiation proposals that are in flight in surprising ways)
2nd solution is a more powerful one, but it would require quite a bit of work and planning.

I am sure there are other solutions, so I'll keep thinking.