Update HeaderWithProof Tracking Issue

[njgheorghita]

Updating the HeaderWithProof type

There's been an update to the specs for the HeaderWithProof type, which is breaking. All existing HeaderWithProofs in our prod dbs will need to be updated / removed. Also, trin needs support for ephemeral proofs.

@morph-dev & @njgheorghita are taking lead on this

TODOs

• Turn off history bridges (to prevent invalid data flowing into the network)
• Temporarily disable trin node's ability to store history types (to prevent further propagation of invalid data)
• fix: add temporary flag to disable storing headers #1668
• Write a script to update headers in prod db databases
  • Convert valid headers with proofs. delete any headers that don't have proofs
• None removal & SSZ Union removal in BlockHeaderWithProof
  • Update HeaderWithProof with latest spec changes #1672
• Update bridge to support new post-capella proof generation (eg. non-ephemeral)
  • @njgheorghita is working on this
• Update hive & associated test vectors
  • https://github.com/ethereum/portal-spec-tests/blob/master/tests/mainnet/history/hive/test_data_collection_of_forks_blocks.yaml
  • remove crates/validation/src/assets/hive/blocks.yml from trin and replace with link to portal-spec-tests
• Add support for generating / storing / updating ephemeral proofs
  • Investigate storage strategy (does this require a separate db?)
• Can we fix incorrect storage reporting metrics in Trin during this update?
• Enable full validation across all networks
  • Does this allow us to remove any code? Options? Weird hacks we used for partial validation?
• Deploy changes and restart bridges
• Once History Network supports full header validation with ephemeral support and bridging headers from 21M to latest revert these PRs
  • feat: disable state network history header validation #1667 to re-enable history network header validation for the state network
  • fix: add temporary flag to disable storing headers #1668 to remove temporary flag disabling storing headers
  • https://github.com/ethereum/cluster/pull/1148
  • Update HeaderWithProof with latest spec changes #1672 - update mock block proofs

Related Issues

• Cross-Client Implementation / Deployment Tracking Issue
• Spec PR for updated Header type
• Storage reporting issue in Trin