Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove protocol.Params.ProtocolVersion field #6955

Draft
wants to merge 2 commits into
base: feature/efm-recovery
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions access/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -314,9 +314,10 @@ type CompatibleRange struct {

// NodeVersionInfo contains information about node, such as semver, commit, sporkID, protocolVersion, etc
type NodeVersionInfo struct {
Semver string
Commit string
SporkId flow.Identifier
Semver string
Commit string
SporkId flow.Identifier
// TODO: should we replicate protocol state version here, or just remove this field?
ProtocolVersion uint64
Comment on lines +320 to 321
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we replicate protocol state version here, or omit this field?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment, we are using the protocol state synonymously with the version of the protocol, so it might be useful to report it here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep it and have it use the protocol state version. We should also include all other important versions here as well (e.g. execution engine version)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

regarding the already-existing ProtocolVersion, was just hoping to confirm: a version change from 37 (old versioning scheme) to 2 (new scheme) would be fine?

SporkRootBlockHeight uint64
NodeRootBlockHeight uint64
Expand Down
1 change: 0 additions & 1 deletion cmd/bootstrap/cmd/finalize.go
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,6 @@ func finalize(cmd *cobra.Command, args []string) {
result,
seal,
rootQC,
intermediaryData.ProtocolVersion,
func(epochStateID flow.Identifier) (protocol_state.KVStoreAPI, error) {
return kvstore.NewDefaultKVStore(
intermediaryData.FinalizationSafetyThreshold,
Expand Down
1 change: 0 additions & 1 deletion cmd/bootstrap/cmd/intermediary.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ type IntermediaryBootstrappingData struct {
// like the root block).
// This is used to pass data between the rootblock command and the finalize command.
type IntermediaryParamsData struct {
ProtocolVersion uint
FinalizationSafetyThreshold uint64
EpochExtensionViewCount uint64
}
Expand Down
20 changes: 12 additions & 8 deletions cmd/bootstrap/cmd/rootblock.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,14 @@ import (
)

var (
flagRootChain string
flagRootParent string
flagRootHeight uint64
flagRootTimestamp string
flagProtocolVersion uint
flagRootChain string
flagRootParent string
flagRootHeight uint64
flagRootTimestamp string
// Deprecated: Replaced by ProtocolState KVStore version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

following up on my suggestion for the previous PR #6953:

Suggested change
// Deprecated: Replaced by ProtocolState KVStore version
// Deprecated: Replaced by ProtocolStateVersion

// Historically, this flag set a spork-scoped version number, by convention equal to the major software version.
// Now that we have HCUs which change the major software version mid-spork, this is no longer useful.
deprecatedFlagProtocolVersion uint
flagFinalizationSafetyThreshold uint64
flagEpochExtensionViewCount uint64
flagCollectionClusters uint
Expand Down Expand Up @@ -92,14 +95,13 @@ func addRootBlockCmdFlags() {
rootBlockCmd.Flags().StringVar(&flagRootParent, "root-parent", "0000000000000000000000000000000000000000000000000000000000000000", "ID for the parent of the root block")
rootBlockCmd.Flags().Uint64Var(&flagRootHeight, "root-height", 0, "height of the root block")
rootBlockCmd.Flags().StringVar(&flagRootTimestamp, "root-timestamp", time.Now().UTC().Format(time.RFC3339), "timestamp of the root block (RFC3339)")
rootBlockCmd.Flags().UintVar(&flagProtocolVersion, "protocol-version", flow.DefaultProtocolVersion, "major software version used for the duration of this spork")
rootBlockCmd.Flags().UintVar(&deprecatedFlagProtocolVersion, "protocol-version", 0, "deprecated: this flag will be ignored and remove in a future release")
rootBlockCmd.Flags().Uint64Var(&flagFinalizationSafetyThreshold, "finalization-safety-threshold", 500, "defines finalization safety threshold")
rootBlockCmd.Flags().Uint64Var(&flagEpochExtensionViewCount, "epoch-extension-view-count", 100_000, "length of epoch extension in views, default is 100_000 which is approximately 1 day")

cmd.MarkFlagRequired(rootBlockCmd, "root-chain")
cmd.MarkFlagRequired(rootBlockCmd, "root-parent")
cmd.MarkFlagRequired(rootBlockCmd, "root-height")
cmd.MarkFlagRequired(rootBlockCmd, "protocol-version")
cmd.MarkFlagRequired(rootBlockCmd, "finalization-safety-threshold")
cmd.MarkFlagRequired(rootBlockCmd, "epoch-extension-view-count")

Expand Down Expand Up @@ -134,6 +136,9 @@ func rootBlock(cmd *cobra.Command, args []string) {
log.Fatal().Msg("cannot use both --partner-stakes and --partner-weights flags (use only --partner-weights)")
}
}
if deprecatedFlagProtocolVersion != 0 {
log.Warn().Msg("using deprecated flag --protocol-version; please remove this flag from your workflow, it is ignored and will be removed in a future release")
}

// validate epoch configs
err := validateEpochConfig()
Expand Down Expand Up @@ -226,7 +231,6 @@ func rootBlock(cmd *cobra.Command, args []string) {
intermediaryParamsData := IntermediaryParamsData{
FinalizationSafetyThreshold: flagFinalizationSafetyThreshold,
EpochExtensionViewCount: flagEpochExtensionViewCount,
ProtocolVersion: flagProtocolVersion,
}
intermediaryData := IntermediaryBootstrappingData{
IntermediaryEpochData: intermediaryEpochData,
Expand Down
2 changes: 0 additions & 2 deletions cmd/bootstrap/cmd/rootblock_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ const rootBlockHappyPathLogs = "collecting partner network and staking keys" +
`remove internal partner nodes` +
`removed 0 internal partner nodes` +
`checking constraints on consensus nodes` +

`assembling network and staking keys` +
`wrote file \S+/node-infos.pub.json` +
`running DKG for consensus nodes` +
Expand Down Expand Up @@ -68,7 +67,6 @@ func TestRootBlock_HappyPath(t *testing.T) {
flagNumViewsInStakingAuction = 50_000
flagNumViewsInDKGPhase = 2_000
flagFinalizationSafetyThreshold = 1_000
flagProtocolVersion = 42
flagUseDefaultEpochTargetEndTime = true
flagEpochTimingRefCounter = 0
flagEpochTimingRefTimestamp = 0
Expand Down
3 changes: 1 addition & 2 deletions cmd/scaffold.go
Original file line number Diff line number Diff line change
Expand Up @@ -949,8 +949,7 @@ func (fnb *FlowNodeBuilder) initMetrics() error {
// metrics enabled, report node info metrics as post init event
fnb.PostInit(func(nodeConfig *NodeConfig) error {
nodeInfoMetrics := metrics.NewNodeInfoCollector()
protocolVersion := fnb.RootSnapshot.Params().ProtocolVersion()
nodeInfoMetrics.NodeInfo(build.Version(), build.Commit(), nodeConfig.SporkID.String(), protocolVersion)
nodeInfoMetrics.NodeInfo(build.Version(), build.Commit(), nodeConfig.SporkID.String())
return nil
})
}
Expand Down
8 changes: 5 additions & 3 deletions engine/access/rpc/backend/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -311,10 +311,12 @@ func (b *Backend) Ping(ctx context.Context) error {
// GetNodeVersionInfo returns node version information such as semver, commit, sporkID, protocolVersion, etc
func (b *Backend) GetNodeVersionInfo(_ context.Context) (*access.NodeVersionInfo, error) {
sporkID := b.stateParams.SporkID()
protocolVersion := b.stateParams.ProtocolVersion()
sporkRootBlockHeight := b.stateParams.SporkRootBlockHeight()

nodeRootBlockHeader := b.stateParams.SealedRoot()
protocolSnapshot, err := b.state.Final().ProtocolState()
if err != nil {
return nil, fmt.Errorf("could not read finalized protocol kvstore: %w", err)
}

var compatibleRange *access.CompatibleRange

Expand All @@ -330,7 +332,7 @@ func (b *Backend) GetNodeVersionInfo(_ context.Context) (*access.NodeVersionInfo
Semver: build.Version(),
Commit: build.Commit(),
SporkId: sporkID,
ProtocolVersion: uint64(protocolVersion),
ProtocolVersion: protocolSnapshot.GetProtocolStateVersion(),
SporkRootBlockHeight: sporkRootBlockHeight,
NodeRootBlockHeight: nodeRootBlockHeader.Height,
CompatibleRange: compatibleRange,
Expand Down
1 change: 0 additions & 1 deletion integration/testnet/network.go
Original file line number Diff line number Diff line change
Expand Up @@ -1275,7 +1275,6 @@ func BootstrapNetwork(networkConf NetworkConfig, bootstrapDir string, chainID fl
result,
seal,
qc,
flow.DefaultProtocolVersion,
networkConf.KVStoreFactory,
)
if err != nil {
Expand Down
12 changes: 4 additions & 8 deletions module/metrics/node_info.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
package metrics

import (
"strconv"

"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
Expand All @@ -14,10 +12,9 @@ type NodeInfoCollector struct {
}

const (
sporkIDLabel = "spork_id"
versionLabel = "version"
commitLabel = "commit"
protocolVersionLabel = "protocol_version"
sporkIDLabel = "spork_id"
versionLabel = "version"
commitLabel = "commit"
)

func NewNodeInfoCollector() *NodeInfoCollector {
Expand All @@ -32,9 +29,8 @@ func NewNodeInfoCollector() *NodeInfoCollector {
return collector
}

func (sc *NodeInfoCollector) NodeInfo(version, commit, sporkID string, protocolVersion uint) {
func (sc *NodeInfoCollector) NodeInfo(version, commit, sporkID string) {
sc.nodeInfo.WithLabelValues(versionLabel, version)
sc.nodeInfo.WithLabelValues(commitLabel, commit)
sc.nodeInfo.WithLabelValues(sporkIDLabel, sporkID)
sc.nodeInfo.WithLabelValues(protocolVersionLabel, strconv.FormatUint(uint64(protocolVersion), 10))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishalchangrani do we use this for partner tracking, or just the version?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, nm. this isn't related to the ping service

}
7 changes: 2 additions & 5 deletions module/metrics/node_info_test.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package metrics

import (
"strconv"
"testing"

"github.com/prometheus/client_golang/prometheus"
Expand All @@ -19,8 +18,7 @@ func TestNodeInfoCollector_NodeInfo(t *testing.T) {
version := "0.29"
commit := "63cec231136914941e2358de2054a6ef71ea3c99"
sporkID := unittest.IdentifierFixture().String()
protocolVersion := uint(10076)
collector.NodeInfo(version, commit, sporkID, protocolVersion)
collector.NodeInfo(version, commit, sporkID)
metricsFamilies, err := reg.Gather()
require.NoError(t, err)

Expand All @@ -35,8 +33,7 @@ func TestNodeInfoCollector_NodeInfo(t *testing.T) {
assert.Failf(t, "metric not found", "except to find value %s", value)
}

protocolVersionAsString := strconv.FormatUint(uint64(protocolVersion), 10)
for _, value := range []string{version, commit, sporkID, protocolVersionAsString} {
for _, value := range []string{version, commit, sporkID} {
assertReported(value)
}
}
7 changes: 0 additions & 7 deletions state/protocol/badger/params.go
Original file line number Diff line number Diff line change
Expand Up @@ -131,12 +131,6 @@ func ReadGlobalParams(db *badger.DB) (*inmem.Params, error) {
return nil, fmt.Errorf("could not get spork root block height: %w", err)
}

var version uint
err = db.View(operation.RetrieveProtocolVersion(&version))
if err != nil {
return nil, fmt.Errorf("could not get protocol version: %w", err)
}

root, err := ReadFinalizedRoot(db) // retrieve root header
if err != nil {
return nil, fmt.Errorf("could not get root: %w", err)
Expand All @@ -147,7 +141,6 @@ func ReadGlobalParams(db *badger.DB) (*inmem.Params, error) {
ChainID: root.ChainID,
SporkID: sporkID,
SporkRootBlockHeight: sporkRootBlockHeight,
ProtocolVersion: version,
},
), nil
}
Expand Down
5 changes: 0 additions & 5 deletions state/protocol/badger/snapshot_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,6 @@ func TestSnapshot_Params(t *testing.T) {

expectedChainID := rootSnapshot.Params().ChainID()
expectedSporkID := rootSnapshot.Params().SporkID()
expectedProtocolVersion := rootSnapshot.Params().ProtocolVersion()

rootHeader, err := rootSnapshot.Head()
require.NoError(t, err)
Expand Down Expand Up @@ -142,10 +141,6 @@ func TestSnapshot_Params(t *testing.T) {
sporkID := snapshot.Params().SporkID()
assert.Equal(t, expectedSporkID, sporkID)
})
t.Run("should be able to get protocol version from snapshot", func(t *testing.T) {
protocolVersion := snapshot.Params().ProtocolVersion()
assert.Equal(t, expectedProtocolVersion, protocolVersion)
})
}
})
}
Expand Down
6 changes: 0 additions & 6 deletions state/protocol/badger/state.go
Original file line number Diff line number Diff line change
Expand Up @@ -586,12 +586,6 @@ func bootstrapSporkInfo(root protocol.Snapshot) func(*transaction.Tx) error {
return fmt.Errorf("could not insert spork root block height: %w", err)
}

version := params.ProtocolVersion()
err = operation.InsertProtocolVersion(version)(bdtx)
if err != nil {
return fmt.Errorf("could not insert protocol version: %w", err)
}

return nil
}
}
Expand Down
6 changes: 1 addition & 5 deletions state/protocol/inmem/convert.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,6 @@ func FromParams(from protocol.GlobalParams) (*Params, error) {
ChainID: from.ChainID(),
SporkID: from.SporkID(),
SporkRootBlockHeight: from.SporkRootBlockHeight(),
ProtocolVersion: from.ProtocolVersion(),
}
return &Params{params}, nil
}
Expand All @@ -69,12 +68,11 @@ func ClusterFromEncodable(enc EncodableCluster) (*Cluster, error) {
// root bootstrap state. This is used to bootstrap the protocol state for
// genesis or post-spork states.
func SnapshotFromBootstrapState(root *flow.Block, result *flow.ExecutionResult, seal *flow.Seal, qc *flow.QuorumCertificate) (*Snapshot, error) {
version := flow.DefaultProtocolVersion
safetyParams, err := protocol.DefaultEpochSafetyParams(root.Header.ChainID)
if err != nil {
return nil, fmt.Errorf("could not get default epoch commit safety threshold: %w", err)
}
return SnapshotFromBootstrapStateWithParams(root, result, seal, qc, version, func(epochStateID flow.Identifier) (protocol_state.KVStoreAPI, error) {
return SnapshotFromBootstrapStateWithParams(root, result, seal, qc, func(epochStateID flow.Identifier) (protocol_state.KVStoreAPI, error) {
return kvstore.NewDefaultKVStore(safetyParams.FinalizationSafetyThreshold, safetyParams.EpochExtensionViewCount, epochStateID)
})
}
Expand All @@ -86,7 +84,6 @@ func SnapshotFromBootstrapStateWithParams(
result *flow.ExecutionResult,
seal *flow.Seal,
qc *flow.QuorumCertificate,
protocolVersion uint,
kvStoreFactory func(epochStateID flow.Identifier) (protocol_state.KVStoreAPI, error),
) (*Snapshot, error) {
setup, ok := result.ServiceEvents[0].Event.(*flow.EpochSetup)
Expand Down Expand Up @@ -123,7 +120,6 @@ func SnapshotFromBootstrapStateWithParams(
ChainID: root.Header.ChainID, // chain ID must match the root block
SporkID: root.ID(), // use root block ID as the unique spork identifier
SporkRootBlockHeight: root.Header.Height, // use root block height as the spork root block height
ProtocolVersion: protocolVersion, // major software version for this spork
}

rootMinEpochState := EpochProtocolStateFromServiceEvents(setup, commit)
Expand Down
1 change: 0 additions & 1 deletion state/protocol/inmem/encodable.go
Original file line number Diff line number Diff line change
Expand Up @@ -119,5 +119,4 @@ type EncodableParams struct {
ChainID flow.ChainID
SporkID flow.Identifier
SporkRootBlockHeight uint64
ProtocolVersion uint
}
4 changes: 0 additions & 4 deletions state/protocol/inmem/params.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,3 @@ func (p Params) SporkID() flow.Identifier {
func (p Params) SporkRootBlockHeight() uint64 {
return p.enc.SporkRootBlockHeight
}

func (p Params) ProtocolVersion() uint {
return p.enc.ProtocolVersion
}
4 changes: 0 additions & 4 deletions state/protocol/invalid/params.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,3 @@ func (p Params) SporkID() flow.Identifier {
func (p Params) SporkRootBlockHeight() uint64 {
return 0
}

func (p Params) ProtocolVersion() uint {
return 0
}
18 changes: 0 additions & 18 deletions state/protocol/mock/global_params.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 0 additions & 18 deletions state/protocol/mock/params.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 0 additions & 4 deletions state/protocol/params.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,4 @@ type GlobalParams interface {
// If node uses a sealing segment for bootstrapping then this value will be carried over
// as part of snapshot.
SporkRootBlockHeight() uint64

// ProtocolVersion returns the protocol version, the major software version
// of the protocol software.
ProtocolVersion() uint
}
13 changes: 0 additions & 13 deletions storage/badger/operation/spork.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,3 @@ func InsertSporkRootBlockHeight(height uint64) func(*badger.Txn) error {
func RetrieveSporkRootBlockHeight(height *uint64) func(*badger.Txn) error {
return retrieve(makePrefix(codeSporkRootBlockHeight), height)
}

// InsertProtocolVersion inserts the protocol version for the present spork.
// A single database and protocol state instance spans at most one spork, and
// a spork has exactly one protocol version for its duration, so this is
// inserted exactly once, when bootstrapping the state.
func InsertProtocolVersion(version uint) func(*badger.Txn) error {
return insert(makePrefix(codeProtocolVersion), version)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we should also Deprecate the corresponding storage code:

codeProtocolVersion = 14

}

// RetrieveProtocolVersion retrieves the protocol version for the present spork.
func RetrieveProtocolVersion(version *uint) func(*badger.Txn) error {
return retrieve(makePrefix(codeProtocolVersion), version)
}
Loading
Loading