-
Notifications
You must be signed in to change notification settings - Fork 147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Initial consensus node to block node communication #17769
Conversation
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
… block nodes Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Derek Riley <[email protected]>
Signed-off-by: Petar Tonev <[email protected]>
Signed-off-by: Petar Tonev <[email protected]>
# Conflicts: # hedera-node/configuration/dev/application.properties # hedera-node/hedera-app/src/jmh/java/com/hedera/node/app/blocks/BlockRetentionManagerBenchmark.java # hedera-node/hedera-app/src/main/java/com/hedera/node/app/Hedera.java # hedera-node/hedera-app/src/main/java/com/hedera/node/app/blocks/BlockRetentionManager.java # hedera-node/hedera-app/src/test/java/com/hedera/node/app/blocks/BlockRetentionManagerTest.java
Signed-off-by: Petar Tonev <[email protected]>
Signed-off-by: Petar Tonev <[email protected]>
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferencesCodacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more |
Signed-off-by: Petar Tonev <[email protected]>
hedera-node/hedera-app/src/main/java/com/hedera/node/app/ServicesMain.java
Outdated
Show resolved
Hide resolved
...irlds-platform-core/src/test/java/com/swirlds/platform/state/WritableQueueStateBaseTest.java
Outdated
Show resolved
Hide resolved
.../hedera-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConnection.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Petar Tonev <[email protected]>
# Conflicts: # hedera-node/hedera-config/src/main/java/com/hedera/node/config/data/BlockStreamConfig.java # hedera-node/test-clients/src/main/java/com/hedera/services/bdd/spec/utilops/pauses/HapiSpecWaitUntilNextBlock.java
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I had a lot of thoughts while reviewing this, but I also recognize that 1) this is not the final production code we'll be using, and 2) I haven't been very involved in writing/coordinating this code for the past couple weeks. As long as this code is working we're probably fine to push off quite a few of my comments, and revisit them later.
hedera-node/hedera-app/src/main/java/com/hedera/node/app/ServicesMain.java
Show resolved
Hide resolved
hedera-node/hedera-app/src/main/java/com/hedera/node/app/blocks/BlockStreamModule.java
Outdated
Show resolved
Hide resolved
...ra-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConfigExtractor.java
Show resolved
Hide resolved
.../hedera-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/GrpcBlockItemWriter.java
Outdated
Show resolved
Hide resolved
hedera-node/hedera-config/src/main/java/com/hedera/node/config/types/BlockStreamWriterMode.java
Outdated
Show resolved
Hide resolved
...-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConnectionManager.java
Outdated
Show resolved
Hide resolved
...-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConnectionManager.java
Show resolved
Hide resolved
...-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConnectionManager.java
Show resolved
Hide resolved
...-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConnectionManager.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Petar Tonev <[email protected]>
Signed-off-by: Petar Tonev <[email protected]>
Signed-off-by: Petar Tonev <[email protected]>
# Conflicts: # hedera-node/hedera-app/src/main/java/com/hedera/node/app/blocks/impl/GrpcBlockItemWriter.java # hedera-node/hedera-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/FileBlockItemWriter.java
Signed-off-by: Petar Tonev <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a few more comments as a reminder, but they can be addressed later (feel free to resolve them whenever). Thanks @petreze @MiroslavGatsanoga and @derektriley, this is a lot of work! 🙌
...ra-node/hedera-app/src/main/java/com/hedera/node/app/blocks/impl/BlockStreamManagerImpl.java
Show resolved
Hide resolved
...ra-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConfigExtractor.java
Show resolved
Hide resolved
...-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/BlockNodeConnectionManager.java
Show resolved
Hide resolved
...-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/FileAndGrpcBlockItemWriter.java
Show resolved
Hide resolved
hedera-node/hedera-config/src/main/java/com/hedera/node/config/data/BlockStreamConfig.java
Outdated
Show resolved
Hide resolved
...lients/src/main/java/com/hedera/services/bdd/junit/SharedNetworkLauncherSessionListener.java
Show resolved
Hide resolved
...clients/src/main/java/com/hedera/services/bdd/junit/hedera/subprocess/SubProcessNetwork.java
Show resolved
Hide resolved
# Conflicts: # hedera-node/hedera-app/src/main/java/com/hedera/node/app/blocks/impl/streaming/FileBlockItemWriter.java # hedera-node/hedera-config/src/main/java/com/hedera/node/config/data/BlockStreamConfig.java
Signed-off-by: Petar Tonev <[email protected]>
Description:
This PR improves and provides an implementation of gRPC streaming to block nodes. This implementation is subject to change and only provides a happy path for streaming.
Addition of Block Node Connection Management
New Protocol Buffers Definition: Added a new internal protocol buffers file
network/block_node_connections.proto
to define the configuration for connecting to block nodes. This includes messages for individual block node configurations and overall connection configurations.Configuration Files: Added a new JSON configuration file
hedera-node/configuration/dev/block-nodes.json
to specify block node connection details, as an example.Java Class Modifications
Hedera.java: Updated the
Hedera
class to initialize block node connections, handle their shutdown during different platform statuses, and ensure connections are established before starting the platform.HederaInjectionComponent.java: Added a new method to provide the
BlockNodeConnectionManager
.ServicesMain.java: Modified the main method to initialize block node connections before starting the platform.
Dependency Injection and Module Updates
BlockStreamModule
to provide theBlockNodeConnectionManager
and handle different block stream writer modes, including a new mode for both file and gRPC streaming.Related issue(s):
Fixes #17730
Notes for reviewer:
Checklist