Skip to content

feat(server): add migration command #113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ require (
cosmossdk.io/math v1.5.3
cosmossdk.io/store v1.1.2
github.com/cometbft/cometbft v0.38.17
github.com/cometbft/cometbft-db v0.14.1
github.com/cosmos/cosmos-proto v1.0.0-beta.5
github.com/cosmos/cosmos-sdk v0.50.13
github.com/cosmos/gogoproto v1.7.0
Expand Down Expand Up @@ -87,7 +88,6 @@ require (
github.com/cockroachdb/pebble v1.1.2 // indirect
github.com/cockroachdb/redact v1.1.5 // indirect
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect
github.com/cometbft/cometbft-db v0.14.1 // indirect
github.com/containerd/cgroups v1.1.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.16.3 // indirect
Expand Down
File renamed without changes.
247 changes: 247 additions & 0 deletions server/migration_cmd.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@
package server

import (
"bytes"
"context"
"errors"
"fmt"
"path/filepath"

dbm "github.com/cometbft/cometbft-db"
abci "github.com/cometbft/cometbft/abci/types"
cometbftcmd "github.com/cometbft/cometbft/cmd/cometbft/commands"
cfg "github.com/cometbft/cometbft/config"
"github.com/cometbft/cometbft/libs/os"
cmtproto "github.com/cometbft/cometbft/proto/tendermint/types"
"github.com/cometbft/cometbft/state"
"github.com/cometbft/cometbft/store"
cometbfttypes "github.com/cometbft/cometbft/types"
"github.com/spf13/cobra"

rollkitstore "github.com/rollkit/rollkit/pkg/store"
rollkittypes "github.com/rollkit/rollkit/types"
)

var flagDaHeight = "da-height"

// MigrateToRollkitCmd returns a command that migrates the data from the CometBFT chain to Rollkit.
func MigrateToRollkitCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "rollkit-migration",
Short: "Migrate the data from the CometBFT chain to Rollkit",
Long: "Migrate the data from the CometBFT chain to Rollkit. This command should be used to migrate nodes or the sequencer.",
Args: cobra.ExactArgs(0),
RunE: func(cmd *cobra.Command, args []string) error {
config, err := cometbftcmd.ParseConfig(cmd)
if err != nil {
return err
}

cometBlockStore, cometStateStore, err := loadStateAndBlockStore(config)
if err != nil {
return err
}

cometBFTstate, err := cometStateStore.Load()
if err != nil {
return err
}

lastBlockHeight := cometBFTstate.LastBlockHeight

cmd.Printf("Last block height: %d\n", lastBlockHeight)

rollkitStore, err := loadRollkitStateStore(config.RootDir, config.DBPath)
if err != nil {
return err
}

daHeight, err := cmd.Flags().GetUint64(flagDaHeight)
if err != nil {
return fmt.Errorf("error reading %s flag: %w", flagDaHeight, err)
}
rollkitState, err := rollkitStateFromCometBFTState(cometBFTstate, daHeight)
if err != nil {
return err
}

if err = rollkitStore.UpdateState(
context.Background(), rollkitState,
); err != nil {
return err
}

// migrate all the blocks from the CometBFT block store to the rollkit store
// the migration is done in reverse order, starting from the last block
for height := lastBlockHeight; height > 0; height-- {
cmd.Printf("Migrating block %d...\n", height)
Comment on lines +76 to +77
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider iterating blocks in chronological order.

The current implementation iterates blocks backward from lastBlockHeight to 1. For consistency and potential dependencies between blocks, consider iterating forward instead.

-			for height := lastBlockHeight; height > 0; height-- {
+			for height := int64(1); height <= lastBlockHeight; height++ {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for height := lastBlockHeight; height > 0; height-- {
cmd.Printf("Migrating block %d...\n", height)
for height := int64(1); height <= lastBlockHeight; height++ {
cmd.Printf("Migrating block %d...\n", height)
🤖 Prompt for AI Agents
In server/migration_cmd.go around lines 68 to 69, the loop iterates blocks
backward from lastBlockHeight down to 1, which may cause issues with
dependencies between blocks. Change the loop to iterate forward from 1 up to
lastBlockHeight to process blocks in chronological order, ensuring consistency
and correct handling of dependencies.


block := cometBlockStore.LoadBlock(height)
if block == nil {
cmd.Printf("Block %d not found in CometBFT block store, skipping...\n", height)
continue
}

header, data, signature := cometBlockToRollkit(block)

if err = rollkitStore.SaveBlockData(context.Background(), header, data, &signature); err != nil {
return err
}

// Only save extended commit info if vote extensions are enabled
if enabled := cometBFTstate.ConsensusParams.ABCI.VoteExtensionsEnabled(block.Height); enabled {
extendedCommit := cometBlockStore.LoadBlockExtendedCommit(lastBlockHeight)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix incorrect height variable usage.

The code uses lastBlockHeight instead of the current iteration variable height.

-					extendedCommit := cometBlockStore.LoadBlockExtendedCommit(lastBlockHeight)
+					extendedCommit := cometBlockStore.LoadBlockExtendedCommit(height)
🤖 Prompt for AI Agents
In server/migration_cmd.go at line 80, replace the variable lastBlockHeight with
the current loop iteration variable height when calling LoadBlockExtendedCommit
to ensure the correct block height is used in each iteration.


extendedCommitInfo := abci.ExtendedCommitInfo{
Round: extendedCommit.Round,
}

for _, vote := range extendedCommit.ToExtendedVoteSet("", cometBFTstate.LastValidators).List() {
power := int64(0)
for _, v := range cometBFTstate.LastValidators.Validators {
if bytes.Equal(v.Address.Bytes(), vote.ValidatorAddress) {
power = v.VotingPower
break
}
}

extendedCommitInfo.Votes = append(extendedCommitInfo.Votes, abci.ExtendedVoteInfo{
Validator: abci.Validator{
Address: vote.ValidatorAddress,
Power: power,
},
VoteExtension: vote.Extension,
ExtensionSignature: vote.ExtensionSignature,
BlockIdFlag: cmtproto.BlockIDFlag(vote.CommitSig().BlockIDFlag),
})
}

_ = extendedCommitInfo
// rollkitStore.SaveExtendedCommit(context.Background(), header.Height(), &extendedCommitInfo)
panic("Saving extended commit info is not implemented yet")
}
Comment on lines +119 to +122
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Replace panic with proper error handling.

Using panic for unimplemented functionality is poor user experience. Return an error instead.

-					_ = extendedCommitInfo
-					// rollkitStore.SaveExtendedCommit(context.Background(), header.Height(), &extendedCommitInfo)
-					panic("Saving extended commit info is not implemented yet")
+					// TODO: Implement SaveExtendedCommit
+					// rollkitStore.SaveExtendedCommit(context.Background(), header.Height(), &extendedCommitInfo)
+					return fmt.Errorf("saving extended commit info is not implemented yet")
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
_ = extendedCommitInfo
// rollkitStore.SaveExtendedCommit(context.Background(), header.Height(), &extendedCommitInfo)
panic("Saving extended commit info is not implemented yet")
}
// TODO: Implement SaveExtendedCommit
// rollkitStore.SaveExtendedCommit(context.Background(), header.Height(), &extendedCommitInfo)
return fmt.Errorf("saving extended commit info is not implemented yet")
🤖 Prompt for AI Agents
In server/migration_cmd.go around lines 106 to 109, replace the panic call used
for unimplemented saving of extended commit info with proper error handling by
returning an error instead. Modify the function to return an error indicating
that saving extended commit info is not implemented yet, avoiding abrupt
termination and improving user experience.


cmd.Println("Block", height, "migrated")
}

cmd.Println("Migration completed successfully")
return errors.Join(rollkitStore.Close(), cometBlockStore.Close(), cometStateStore.Close())
},
}

cmd.Flags().Uint64(flagDaHeight, 1, "The DA height to set in the Rollkit state. Defaults to 1.")

return cmd
}

// cometBlockToRollkit converts a cometBFT block to a rollkit block
func cometBlockToRollkit(block *cometbfttypes.Block) (*rollkittypes.SignedHeader, *rollkittypes.Data, rollkittypes.Signature) {
var (
header *rollkittypes.SignedHeader
data *rollkittypes.Data
signature rollkittypes.Signature
)

// find proposer signature
for _, sig := range block.LastCommit.Signatures {
if bytes.Equal(sig.ValidatorAddress.Bytes(), block.ProposerAddress.Bytes()) {
signature = sig.Signature
break
}
}

header = &rollkittypes.SignedHeader{
Header: rollkittypes.Header{
BaseHeader: rollkittypes.BaseHeader{
Height: uint64(block.Height),
Time: uint64(block.Time.UnixNano()),
ChainID: block.ChainID,
},
Version: rollkittypes.Version{
Block: block.Version.Block,
App: block.Version.App,
},
LastHeaderHash: block.LastBlockID.Hash.Bytes(),
LastCommitHash: block.LastCommitHash.Bytes(),
DataHash: block.DataHash.Bytes(),
ConsensusHash: block.ConsensusHash.Bytes(),
AppHash: block.AppHash.Bytes(),
LastResultsHash: block.LastResultsHash.Bytes(),
ValidatorHash: block.ValidatorsHash.Bytes(),
ProposerAddress: block.ProposerAddress.Bytes(),
},
Signature: signature, // TODO: figure out this.
}

data = &rollkittypes.Data{
Metadata: &rollkittypes.Metadata{
ChainID: block.ChainID,
Height: uint64(block.Height),
Time: uint64(block.Time.UnixNano()),
LastDataHash: block.DataHash.Bytes(),
},
}

for _, tx := range block.Txs {
data.Txs = append(data.Txs, rollkittypes.Tx(tx))
}

return header, data, signature
}

func loadStateAndBlockStore(config *cfg.Config) (*store.BlockStore, state.Store, error) {
dbType := dbm.BackendType(config.DBBackend)

if !os.FileExists(filepath.Join(config.DBDir(), "blockstore.db")) {
return nil, nil, fmt.Errorf("no blockstore found in %v", config.DBDir())
}

// Get BlockStore
blockStoreDB, err := dbm.NewDB("blockstore", dbType, config.DBDir())
if err != nil {
return nil, nil, err
}
blockStore := store.NewBlockStore(blockStoreDB)

if !os.FileExists(filepath.Join(config.DBDir(), "state.db")) {
return nil, nil, fmt.Errorf("no statestore found in %v", config.DBDir())
}

// Get StateStore
stateDB, err := dbm.NewDB("state", dbType, config.DBDir())
if err != nil {
return nil, nil, err
}
stateStore := state.NewStore(stateDB, state.StoreOptions{
DiscardABCIResponses: config.Storage.DiscardABCIResponses,
})

return blockStore, stateStore, nil
}

func loadRollkitStateStore(rootDir, dbPath string) (rollkitstore.Store, error) {
baseKV, err := rollkitstore.NewDefaultKVStore(rootDir, dbPath, "rollkit")
if err != nil {
return nil, err
}

return rollkitstore.New(baseKV), nil
}

func rollkitStateFromCometBFTState(cometBFTState state.State, daHeight uint64) (rollkittypes.State, error) {
return rollkittypes.State{
Version: rollkittypes.Version{
Block: cometBFTState.Version.Consensus.Block,
App: cometBFTState.Version.Consensus.App,
},
ChainID: cometBFTState.ChainID,
InitialHeight: uint64(cometBFTState.LastBlockHeight), // The initial height is the migration height
Copy link
Contributor

@coderabbitai coderabbitai bot Jun 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

InitialHeight should reflect the chain's actual initial height.

Setting InitialHeight to LastBlockHeight seems incorrect. It should be the height at which the chain was initialized, not the current migration height.

Consider using the actual initial height from the CometBFT state or configuration.

🤖 Prompt for AI Agents
In server/migration_cmd.go at line 222, the InitialHeight is incorrectly set to
LastBlockHeight, which represents the current migration height. Instead, update
InitialHeight to use the chain's actual initial height from the CometBFT state
or configuration, ensuring it reflects the chain's true starting height rather
than the current block height.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is correct, as the start of the rollkit chain (technically a new chain), will the be the end of the last chain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@julienrbrt Thank you for the clarification! That makes perfect sense - for chain migration continuity, the new Rollkit chain should indeed start where the CometBFT chain left off. I understand now that this is an intentional design decision.


✏️ Learnings added
Learnt from: julienrbrt
PR: rollkit/go-execution-abci#113
File: server/migration_cmd.go:231-231
Timestamp: 2025-06-20T13:20:04.011Z
Learning: In the rollkit migration command (server/migration_cmd.go), InitialHeight is correctly set to LastBlockHeight because the Rollkit chain is technically a new chain that starts where the CometBFT chain ended, ensuring continuity during migration.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

LastBlockHeight: uint64(cometBFTState.LastBlockHeight),
LastBlockTime: cometBFTState.LastBlockTime,

DAHeight: daHeight,

LastResultsHash: cometBFTState.LastResultsHash,
AppHash: cometBFTState.AppHash,
}, nil
}
Loading