Feat 2.0 #383

zajko · 2024-12-18T11:02:17Z

Merging condor feature branch to dev

…s read. It will be stored in the event_log table. (#244) Co-authored-by: Jakub Zajkowski <[email protected]>

* Moving rpc sidecar to event sidecar workspace. Both og those servers will work on one binary * Updating schemars version because the old one is having bugs (if there is a name collision for components schema only one will be saved) * Copying casper_types to this project. The reason is that casper_types in release <= 4.0.1 depends on old schemars * Copying casper_types for 2.0 release to this project. The reason is that rpc sidecar has to use the new types definitions, but for now they are not released (and it's not clear if they will be released prior to node 2.0 release). * Changing RpcError implementation to fix tests. Some alignments of codestyle to make clippy happy. * Moving casper-types dependencies to workspace level * Sync changes from node branch * Update the schema file * Delete protocol.md * Move a DbId fix * Change error message * Changes to versioning * Sync changes to types * Switch to having a single binary * Moving config files, fixing compilation issues * bump 'juliet' to '0.2.1' * Sync casper-types changes * Changing RPC sidecar config so that the rpc_server.node_client.exponential_backoff will take a new parameter called max_attempts. I tcan be either "infinite" or a positive, non-zero number. * Storing ApiVersion in event_log table. Removing is_big_integer_id config from DDLConfiguration because it's no longer needed (new version of sea_query handles the situation of defining big_integer and autoincrement) * Revert "Storing ApiVersion in event_log table. Removing is_big_integer_id con…" * Update for node review changes (#15) * Update for node changes * Fix lints * Cleanup * Cover all values in tag roundtrip tests * Moving admin server out from the sse sidecar. They are spinned up separately from sse events server. Also the database initialization happens separetely. Is sse events server is defined a storage definition is required. If rest api server is defined a storage definition is required. * Fix GlobalStateRequest::random * Changes explicit BoxFuture casting to calling 'boxed()' method --------- Co-authored-by: Jakub Zajkowski <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]> Co-authored-by: zajko <[email protected]>

…REST API endpoint with /transaction/deploy/(...) and /transaction/version1/(...) endpoints. (#245) Co-authored-by: Jakub Zajkowski <[email protected]>

* Move merkle proof to casper-types and cleanup * Remove casper types stuff

* Update for InformationRequest::Transaction change * Fix test failures due to version mismatches * Run formatter * Bump commit

#248) * Adding network_name as a field in event_log. It is fetched from `/status` endpoint of the node to which sidecar connects to. --------- Co-authored-by: Jakub Zajkowski <[email protected]>

… event. Instead we wrap the sse events in an envelope which looks like: (#252) ``` { "header": { "api_version": "2.0.0", "network_name": "casper" }, "payload": {(...)} } ``` Events in endpoints that return lists (like signatures for block) will also have each individual element of the list wrapped in such envelope. In the above the header fields: * "api_version" is the api version which was reported in the ApiVersion message for the node that we fetched the event from. * "network_name" is the "chainspec_name" field that was returned in the "/status" endpoint for the node that we fetched the event from. Co-authored-by: Jakub Zajkowski <[email protected]>

* Update usage of ReactorState * Update schema

In Condor, we introduced new bid records in global state that are stored under the `Key::BidAddr` key type. We need to include these records when creating the auction info. Signed-off-by: Alexandru Sardan <[email protected]>

rpc-sidecar: include new style bid records in auction info

* restructured metrics (moved to separate module) * added rest-api specific metrics (current connections, response times) * added db specific metrics (amount of data fetched from "raw" fields for events) * added rpc specific metrics Co-authored-by: Jakub Zajkowski <[email protected]>

* Fix auction state tests * Fix a clippy lint * Disable part of the workflow for now * Workaround for non-PRs

…so it's easier to fine-tune what should actually run. If configuration for all components is either missing or has `enable_server = false`, sidecar exits since it has nothing to do. Also did some refactoring of initialization code. (#262) Co-authored-by: Jakub Zajkowski <[email protected]>

* Bump casper types and refresh schema file * Fix test data

* Switch block reporting changes * Correct typo in name * Bump types

…ntation. (#264) Co-authored-by: Jakub Zajkowski <[email protected]>

… a package to fail (#265) * Removing lingering references to "casper-event-sidecar". The whole project should now be called "casper-sidecar" * Moving packaging configuration from "event-sidecar" module to "sidecar" which aggrgates the final binary Co-authored-by: Jakub Zajkowski <[email protected]>

… sidecar after 2.0 update (#266) Co-authored-by: Jakub Zajkowski <[email protected]>

…spicy-merge

Co-authored-by: Jakub Zajkowski <[email protected]>

…o tests (#366) Co-authored-by: Jakub Zajkowski <[email protected]>

* Aligning sidecar to latest changes in casper-node * Fixing schema tests --------- Co-authored-by: Jakub Zajkowski <[email protected]>

Implement keepalive checks

Co-authored-by: Jakub Zajkowski <[email protected]>

Validate network name if specified

Use local duplicate of `AuctionState` struct

Co-authored-by: Jakub Zajkowski <[email protected]>

Handle standard interrupt signals

* Bumping dependencies --------- Co-authored-by: Jakub Zajkowski <[email protected]>

* aligning with node changes --------- Co-authored-by: Jakub Zajkowski <[email protected]>

…d with newest node code (#385) Co-authored-by: Jakub Zajkowski <[email protected]>

* Introduced "state_get_auction_info_v2" json rpc method. --------- Co-authored-by: Jakub Zajkowski <[email protected]>

#393) * Refactoring "state_get_auction_info" and "state_get_auction_info_v2" to make less binary port requests --------- Co-authored-by: Jakub Zajkowski <[email protected]>

wojcik91 · 2025-01-15T15:10:20Z

.github/workflows/ci-casper-event-sidecar-rs.yml

@@ -13,6 +13,7 @@ on:
  pull_request:
    branches:
      - "dev"
+      - "feat-*"


Suggested change

- "feat-*"

wojcik91 · 2025-01-15T15:11:42Z

Cargo.toml

+tracing = { version = "0", default-features = false }
+tracing-subscriber = "0"
+serde = { version = "1", default-features = false }
+jsonschema = "0.26.2"


Suggested change

jsonschema = "0.26.2"

jsonschema = "0.26.2"

wojcik91 · 2025-01-16T14:55:26Z

event_sidecar/Cargo.toml

+colored = "2.0.0"
+futures-util = { workspace = true }
+once_cell = { workspace = true }
+pg-embed = { git = "https://github.com/zajko/pg-embed", branch = "bump_dependencies" }


Do we need a "release" of this before merging?

wojcik91 · 2025-01-16T14:56:43Z

event_sidecar/src/admin_server.rs

@@ -1,11 +1,13 @@
-use crate::types::config::AdminServerConfig;
+use crate::types::config::AdminApiServerConfig;


Combine imports here and below

wojcik91 · 2025-01-16T14:57:47Z

event_sidecar/src/admin_server.rs

@@ -28,7 +30,7 @@ impl AdminServer {
            .concurrency_limit(self.max_concurrent_requests as usize)
            .rate_limit(self.max_requests_per_second as u64, Duration::from_secs(1))
            .service(warp_service);
-
+        info!(address = %address, "started {} server", "Admin API");


Not sure about the purpose of this string formatting.

wojcik91 · 2025-01-20T14:11:02Z

sidecar/src/run.rs

+            Ok(ExitCode::SUCCESS)
+        },
+        res = do_run(config, components) => res.map_err(|component_error| {
+            error!("The server has exited with an error: {}", component_error);


Suggested change

error!("The server has exited with an error: {}", component_error);

error!("The server has exited with an error: {component_error}");

wojcik91 · 2025-01-20T14:12:46Z

types/src/legacy_sse_data/fixtures.rs

@@ -0,0 +1,893 @@
+use std::collections::{BTreeMap, BTreeSet};


Imports can be cleaned up

wojcik91 · 2025-01-20T14:28:38Z

types/src/legacy_sse_data/translate_execution_result.rs

+
+fn map_transform_v2(ex_ef: &TransformV2) -> Option<TransformKindV1> {
+    let maybe_transform_kind = match ex_ef.kind() {
+        casper_types::execution::TransformKindV2::Identity => Some(TransformKindV1::Identity),


Should probalby just import TransformKindV2

wojcik91 · 2025-01-20T14:29:19Z

types/src/legacy_sse_data/translate_execution_result.rs

+            }
+        }
+        // following variants will not be understood by old clients since they were introduced in 2.x
+        StoredValue::AddressableEntity(_) => None,


Combine those into a single match arm

wojcik91 · 2025-01-20T14:31:21Z

types/src/sse_data.rs

    pub fn should_include(&self, filter: &[EventFilter]) -> bool {
        match self {
            SseData::Shutdown => true,
-            SseData::ApiVersion(_) => filter.contains(&EventFilter::ApiVersion),
+            //Keeping the rest part as explicit match so that if a new variant is added, it will be caught by the compiler            SseData::Shutdown


Suggested change

//Keeping the rest part as explicit match so that if a new variant is added, it will be caught by the compiler SseData::Shutdown

// Keeping the rest part as explicit match so that if a new variant is added, it will be caught by the compiler

…de. (#394) * Added missing error variants to handle error codes returned by the node. * Removing catch-all statement

#395) * Refreshing casper-node dependency, refactoring tests for 'chain_get_era_info_by_switch_block' * applying CR remarks

zajko and others added 30 commits February 12, 2024 11:24

Storing ApiVersion for sse events of the node from which the event wa…

d1d5150

…s read. It will be stored in the event_log table. (#244) Co-authored-by: Jakub Zajkowski <[email protected]>

Replacing 'deploy' naming with 'transaction'. Replaced /deploy/(...) …

13accda

…REST API endpoint with /transaction/deploy/(...) and /transaction/version1/(...) endpoints. (#245) Co-authored-by: Jakub Zajkowski <[email protected]>

Move merkle proof to casper-types, fix compile errors and cleanup (#247)

ced3510

* Move merkle proof to casper-types and cleanup * Remove casper types stuff

Add an addressable entity endpoint (#249)

b552709

Bump versions (#250)

820fce5

Sync node fix (#251)

980c976

* Update for InformationRequest::Transaction change * Fix test failures due to version mismatches * Run formatter * Bump commit

Adding network_name as a field in event_log. It is fetched from `/sta… (

4406583

#248) * Adding network_name as a field in event_log. It is fetched from `/status` endpoint of the node to which sidecar connects to. --------- Co-authored-by: Jakub Zajkowski <[email protected]>

Update usage of ReactorState (#256)

458f708

* Update usage of ReactorState * Update schema

rpc-sidecar: include new style bid records in auction info

488332d

In Condor, we introduced new bid records in global state that are stored under the `Key::BidAddr` key type. We need to include these records when creating the auction info. Signed-off-by: Alexandru Sardan <[email protected]>

Merge pull request #259 from alsrdn/fix-bid-records-in-auction-info

379e920

rpc-sidecar: include new style bid records in auction info

Fix auction state tests (#260)

fdbc585

* Fix auction state tests * Fix a clippy lint * Disable part of the workflow for now * Workaround for non-PRs

Bump casper types and refresh test schema file (#263)

7003df5

* Bump casper types and refresh schema file * Fix test data

Switch block reporting changes (#261)

05626a7

* Switch block reporting changes * Correct typo in name * Bump types

Updated oauth definition of endpoints. Reworded some things in docume…

dbed884

…ntation. (#264) Co-authored-by: Jakub Zajkowski <[email protected]>

Bump casper types

3072c8a

Add support for the new speculative execution results

5a4b005

Use workspace-wide temporary patch for casper-types

f7b00f7

Refreshed documentation so it reflects how users should interact with…

b8b7fe2

… sidecar after 2.0 update (#266) Co-authored-by: Jakub Zajkowski <[email protected]>

Speculative execution no longer needs 'block_identifier'

12e9ddd

Merge remote-tracking branch 'upstream/feat-2.0' into track-feat-2.0-…

e0e98b6

…spicy-merge

Re-add temporarily commented code

67b9b02

Remove stray debug artifact

45bfc5e

Bumping dependencies of sidecar (#269)

3589624

Co-authored-by: Jakub Zajkowski <[email protected]>

Update RPC schema

2cd7a9c

Bump juliet version to 0.3

97b229a

zajko and others added 22 commits November 25, 2024 16:57

Aligning code with recent changes to casper-types, fixing auction_inf…

af91bc9

…o tests (#366) Co-authored-by: Jakub Zajkowski <[email protected]>

Merge branch 'feat-2.0' into keep-alive-checks

037f9fd

apply formatting

e8f2a69

Aligning sidecar to latest changes in casper-node (#367)

332b2d3

* Aligning sidecar to latest changes in casper-node * Fixing schema tests --------- Co-authored-by: Jakub Zajkowski <[email protected]>

fix tests

e9d12b0

Merge pull request #368 from igor-casper/keep-alive-checks

25ad1c3

Implement keepalive checks

Merge branch 'feat-2.0' into copy_auction_state_from_types

1c18a4c

Merge branch 'feat-2.0' into validate_network_name

2df3c85

post-merge fixes

5ee2086

update json schema

8bed099

Repointing casper-node to v2.0.0-rc5 (#369)

e9233e5

Co-authored-by: Jakub Zajkowski <[email protected]>

Merge branch 'feat-2.0' into copy_auction_state_from_types

cfca75c

Merge branch 'feat-2.0' into validate_network_name

6310c9e

Merge pull request #357 from teonite/validate_network_name

a1a0c11

Validate network name if specified

Merge pull request #364 from teonite/copy_auction_state_from_types

61ba931

Use local duplicate of `AuctionState` struct

Aligning code with recent changes in the node (#374)

4e2b764

Co-authored-by: Jakub Zajkowski <[email protected]>

handle standard interrupt signals

36cbd02

disable lint

e558c5c

Merge pull request #375 from teonite/handle_unix_signals

a9c6b66

Handle standard interrupt signals

Bumping dependencies (#378)

3597922

* Bumping dependencies --------- Co-authored-by: Jakub Zajkowski <[email protected]>

aligning with node changes (#382)

b8d19f7

* aligning with node changes --------- Co-authored-by: Jakub Zajkowski <[email protected]>

Repointing casper-node dependency to dev; applying fixes to be aligne…

6ed71c5

…d with newest node code (#385) Co-authored-by: Jakub Zajkowski <[email protected]>

zajko requested review from wojcik91 and darthsiroftardis January 2, 2025 14:14

zajko and others added 3 commits January 10, 2025 14:16

Cutting rc3 (#389)

cbce3a8

Introduced "state_get_auction_info_v2" json rpc method. (#392)

b28ca20

* Introduced "state_get_auction_info_v2" json rpc method. --------- Co-authored-by: Jakub Zajkowski <[email protected]>

Refactoring "state_get_auction_info" and "state_get_auction_info_v2" … (

de43fbc

#393) * Refactoring "state_get_auction_info" and "state_get_auction_info_v2" to make less binary port requests --------- Co-authored-by: Jakub Zajkowski <[email protected]>

wojcik91 requested changes Jan 20, 2025

View reviewed changes

zajko added 2 commits January 21, 2025 11:18

Added missing error variants to handle error codes returned by the no…

3b8c1fb

…de. (#394) * Added missing error variants to handle error codes returned by the node. * Removing catch-all statement

Refreshing casper-node dependency, refactoring tests for 'chain_get_e… (

7a22f99

#395) * Refreshing casper-node dependency, refactoring tests for 'chain_get_era_info_by_switch_block' * applying CR remarks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat 2.0 #383

Feat 2.0 #383

zajko commented Dec 18, 2024

wojcik91 Jan 15, 2025

wojcik91 Jan 15, 2025

wojcik91 Jan 16, 2025

wojcik91 Jan 16, 2025

wojcik91 Jan 16, 2025

wojcik91 Jan 20, 2025

wojcik91 Jan 20, 2025

wojcik91 Jan 20, 2025

wojcik91 Jan 20, 2025

wojcik91 Jan 20, 2025

		@@ -1,11 +1,13 @@
		use crate::types::config::AdminServerConfig;
		use crate::types::config::AdminApiServerConfig;

	error!("The server has exited with an error: {}", component_error);
	error!("The server has exited with an error: {component_error}");

		@@ -0,0 +1,893 @@
		use std::collections::{BTreeMap, BTreeSet};

	//Keeping the rest part as explicit match so that if a new variant is added, it will be caught by the compiler SseData::Shutdown
	// Keeping the rest part as explicit match so that if a new variant is added, it will be caught by the compiler

Feat 2.0 #383

Are you sure you want to change the base?

Feat 2.0 #383

Conversation

zajko commented Dec 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment