PROTO-3189 Removed ignite, GRPC/RPC, data model restructure, refactor and cleanup / PROTO-3037 #108
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
|
|
||
| # `lib` Package Documentation | ||
|
|
||
| ## Overview | ||
|
|
||
| The `lib` package provides core functionality for the Allora off-chain node, handling configurations, RPC/GRPC connections, wallet management, and metrics collection. It serves as the foundation layer for the application's infrastructure. | ||
|
|
||
| ## Core Components | ||
|
|
||
| ### Configuration | ||
|
|
||
| - `WalletConfig`: Manages wallet-related configurations including: | ||
| - Address management (key name, mnemonic) | ||
| - Gas settings | ||
| - Chain configuration | ||
| - RPC endpoints | ||
| - `WorkerConfig`: Handles worker-specific settings, including: | ||
| - Inference endpoints | ||
| - Forecast configurations | ||
| - Topic management | ||
| - `ReputerConfig`: Handles reputer-specific settings, including: | ||
| - Ground Truth endpoint | ||
| - IsNegative loss function endpoint | ||
| - Topic management | ||
| - `ChainConfig`: Handles chain-specific settings, including: | ||
| - Chain ID | ||
| - Chain endpoints | ||
| - Chain gas prices | ||
|
|
||
|
|
||
| ### Connection Management | ||
|
|
||
| The lib package performs queries through GRPC and transactions through RPC. | ||
|
|
||
| - `ConnectionManager`: Thread-safe manager for multiple RPC/GRPC connections | ||
| - Handles both query and transaction nodes separately | ||
| - Provides automatic failover and switching between nodes | ||
| - Manages connection lifecycle | ||
| - A ConnectionManager maintains a Wallet instance and a WalletConfig. | ||
| - `NodeConfig`: Individual node configuration and connection management. The same NodeConfig is used for both query and transaction nodes for simplicity, although they use different clients and their interfaces are different. | ||
|
|
||
| - GRPC/RPC operations include retry mechanisms, both intra-node and inter-node(switching between nodes). Different types of errors are handled differently to optimize retry usage via a comprehensive error handling mechanism via ABCI and HTTP error codes capturing and falling back to string matching. | ||
|
|
||
| ### Wallet Management | ||
|
|
||
| - Thread-safe wallet implementation. | ||
| - Once initialized, the wallet is expected to only change its sequence number, which is done in a thread-safe manner. | ||
|
|
||
| ### Metrics | ||
|
|
||
| - Singleton metrics collector for Prometheus integration | ||
| - Thread-safe counter management | ||
| - Custom metric registration | ||
|
|
||
| ### Subpackages | ||
|
|
||
| Subpackages are used to encapsulate functionality that is not directly Allora. Eg. rpcclient, grpcclient, transaction boilerplate code, etc. | ||
|
|
||
| ## Architecture Decisions | ||
|
|
||
| 1. Singleton pattern for metrics to ensure single source of truth | ||
| 2. Separation of query and transaction nodes for better reliability | ||
| 3. Thread-safe implementations for concurrent operations | ||
| 4. Clear separation between configuration and runtime components | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| package auth | ||
|
|
||
| import ( | ||
| "sync" | ||
|
|
||
| "github.com/cosmos/cosmos-sdk/codec" | ||
| codectypes "github.com/cosmos/cosmos-sdk/codec/types" | ||
| cryptocodec "github.com/cosmos/cosmos-sdk/crypto/codec" | ||
| ) | ||
|
|
||
| var ( | ||
| keyringCodec codec.Codec | ||
| keyringCodecOnce sync.Once | ||
| ) | ||
|
|
||
| // GetKeyringCodec returns a singleton instance of the keyring codec | ||
| func GetKeyringCodec() codec.Codec { | ||
| keyringCodecOnce.Do(func() { | ||
| registry := codectypes.NewInterfaceRegistry() | ||
| cryptocodec.RegisterInterfaces(registry) | ||
| keyringCodec = codec.NewProtoCodec(registry) | ||
| }) | ||
| return keyringCodec | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| package auth | ||
|
|
||
| import ( | ||
| errorsmod "cosmossdk.io/errors" | ||
| cryptotypes "github.com/cosmos/cosmos-sdk/crypto/types" | ||
| sdktypes "github.com/cosmos/cosmos-sdk/types" | ||
| proto "github.com/cosmos/gogoproto/proto" | ||
| "github.com/rs/zerolog/log" | ||
|
|
||
| keyring "github.com/cosmos/cosmos-sdk/crypto/keyring" | ||
|
|
||
| "github.com/cosmos/cosmos-sdk/types/tx/signing" | ||
| ) | ||
|
|
||
| // MarshallAndSignByPrivKey is a helper function to sign a message with a private key | ||
| func MarshallAndSignByPrivKey(payload proto.Message, privKey cryptotypes.PrivKey, address sdktypes.Address) (sig, pk []byte, err error) { | ||
| protoBytesIn, err := proto.Marshal(payload) | ||
| if err != nil { | ||
| return nil, nil, errorsmod.Wrapf(err, "error marshalling workerPayload") // nolint: exhaustruct | ||
|
There was a problem hiding this comment. Remove reference to workerPayload to make it generic? |
||
| } | ||
| sig, err = privKey.Sign(protoBytesIn) | ||
| if err != nil { | ||
| return nil, nil, errorsmod.Wrapf(err, "error signing the InferenceForecastsBundle message") | ||
|
There was a problem hiding this comment. Remove reference to InferenceForecastsBundle. Error msg should be generic. |
||
| } | ||
| return sig, privKey.PubKey().Bytes(), nil | ||
| } | ||
|
|
||
| // MarshallAndSignByKeyring is a helper function to sign a message with a keyring | ||
| func MarshallAndSignByKeyring(payload proto.Message, keyring keyring.Keyring, address sdktypes.Address) (sig, pk []byte, err error) { | ||
| protoBytesIn, err := MarshalProtoMessage(payload) | ||
| if err != nil { | ||
| return nil, nil, errorsmod.Wrapf(err, "error marshalling workerPayload") // nolint: exhaustruct | ||
|
There was a problem hiding this comment. Remove reference to workerPayload to make it generic? |
||
| } | ||
| var pubKey cryptotypes.PubKey | ||
| sig, pubKey, err = keyring.SignByAddress( | ||
| address, | ||
| protoBytesIn, | ||
| signing.SignMode_SIGN_MODE_DIRECT) | ||
| if err != nil { | ||
| return nil, nil, errorsmod.Wrapf(err, "error signing the InferenceForecastsBundle message") // nolint: exhaustruct | ||
|
There was a problem hiding this comment. Remove reference to InferenceForecastsBundle. Error msg should be generic. |
||
| } | ||
| return sig, pubKey.Bytes(), nil | ||
| } | ||
|
|
||
| // MarshalProtoMessage dynamically marshals anytype of Protobuf message. | ||
| // Attempts to use XXX_Marshal if it exists, otherwise falls back to the default proto.Marshal. | ||
| func MarshalProtoMessage(msg proto.Message) ([]byte, error) { | ||
| // Check if XXX_Marshal exists on the type. | ||
| if m, ok := msg.(interface { | ||
| XXX_Marshal([]byte, bool) ([]byte, error) | ||
| }); ok { | ||
| log.Info().Msgf("USING XXX_MARSHAL") | ||
|
There was a problem hiding this comment. Info() might be too verbose here |
||
| return m.XXX_Marshal([]byte{}, true) | ||
| } | ||
|
|
||
| log.Info().Msgf("USING DEFAULT MARSHAL") | ||
|
There was a problem hiding this comment. Info() might be too verbose here |
||
| // Fallback to the default proto.Marshal if XXX_Marshal doesn't exist | ||
| return proto.Marshal(msg) | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo Marshall -> Marshal (there are other cases)