feat(observability): vector top, release candidate (#4886)

* initial relationships

Signed-off-by: Lee Benson <[email protected]>

* bidirectional

Signed-off-by: Lee Benson <[email protected]>

* tweaks

Signed-off-by: Lee Benson <[email protected]>

* update schema.json

Signed-off-by: Lee Benson <[email protected]>

* table

Signed-off-by: Lee Benson <[email protected]>

* table format

Signed-off-by: Lee Benson <[email protected]>

* wip

* api client refactor

Signed-off-by: Lee Benson <[email protected]>

* remove config

Signed-off-by: Lee Benson <[email protected]>

* wip

* wip

Signed-off-by: Lee Benson <[email protected]>

* retry_until lifetime

Signed-off-by: Lee Benson <[email protected]>

* Update top.rs

* source events

Signed-off-by: Lee Benson <[email protected]>

* all topology

Signed-off-by: Lee Benson <[email protected]>

* less matchy

Signed-off-by: Lee Benson <[email protected]>

* clippy love

Signed-off-by: Lee Benson <[email protected]>

* Update api.rs

* Update schema.json

* top events

Signed-off-by: Lee Benson <[email protected]>

* humanize

Signed-off-by: Lee Benson <[email protected]>

* Update top.rs

* up to date

Signed-off-by: Lee Benson <[email protected]>

* -mutex

Signed-off-by: Lee Benson <[email protected]>

* for now

Signed-off-by: Lee Benson <[email protected]>

* clean up

Signed-off-by: Lee Benson <[email protected]>

* better docs

Signed-off-by: Lee Benson <[email protected]>

* clippy

Signed-off-by: Lee Benson <[email protected]>

* topology events

Signed-off-by: Lee Benson <[email protected]>

* gen schema

Signed-off-by: Lee Benson <[email protected]>

* Update src/top.rs

Co-authored-by: Ana Hobden <[email protected]>

* human

Signed-off-by: Lee Benson <[email protected]>

* anymap

Signed-off-by: Lee Benson <[email protected]>

* import tidy up

Signed-off-by: Lee Benson <[email protected]>

* bounded chans

Signed-off-by: Lee Benson <[email protected]>

* Update top.rs

* Update top.rs

* wip

Signed-off-by: Lee Benson <[email protected]>

* mutex

Signed-off-by: Lee Benson <[email protected]>

* ArcSwap

* memory metrics

* swap metrics

Signed-off-by: Lee Benson <[email protected]>

* load averages

Signed-off-by: Lee Benson <[email protected]>

* network metrics

Signed-off-by: Lee Benson <[email protected]>

* filesystem + disk metrics

Signed-off-by: Lee Benson <[email protected]>

* clippy

Signed-off-by: Lee Benson <[email protected]>

* wip

* merge conflicts

Signed-off-by: Lee Benson <[email protected]>

* clippy

Signed-off-by: Lee Benson <[email protected]>

* fix conflicts

Signed-off-by: Lee Benson <[email protected]>

* wip

* tweaks

Signed-off-by: Lee Benson <[email protected]>

* minor improvements

* happy clippy

Signed-off-by: Lee Benson <[email protected]>

* -arc-swap

Signed-off-by: Lee Benson <[email protected]>

* windows support

Signed-off-by: Lee Benson <[email protected]>

* tweaks

Signed-off-by: Lee Benson <[email protected]>

* -termion

Signed-off-by: Lee Benson <[email protected]>

* wasm?

Signed-off-by: Lee Benson <[email protected]>

* fix find/replace

* fix find/replace

Signed-off-by: Lee Benson <[email protected]>

* commentary

Signed-off-by: Lee Benson <[email protected]>

* commentary

Signed-off-by: Lee Benson <[email protected]>

* is_tty

* throttle redraws

Signed-off-by: Lee Benson <[email protected]>

* Rename topology to components in GraphQL API

Signed-off-by: Luc Perkins <[email protected]>

* Regenerate schema.json

Signed-off-by: Luc Perkins <[email protected]>

* Update Vector client library to including s/topology/components name change

Signed-off-by: Luc Perkins <[email protected]>

* components

Signed-off-by: Lee Benson <[email protected]>

* wip

* component event metrics

Signed-off-by: Lee Benson <[email protected]>

* commentary

Signed-off-by: Lee Benson <[email protected]>

* clippy

Signed-off-by: Lee Benson <[email protected]>

* fix merge

Signed-off-by: Lee Benson <[email protected]>

* commentary

Signed-off-by: Lee Benson <[email protected]>

* errors, bytes

Signed-off-by: Lee Benson <[email protected]>

* ErrorsTotal

Signed-off-by: Lee Benson <[email protected]>

* Send + 'static

Signed-off-by: Lee Benson <[email protected]>

* it compiles

Signed-off-by: Lee Benson <[email protected]>

* wip

* filter_fn

Signed-off-by: Lee Benson <[email protected]>

* needs sorting

* ordered results

Signed-off-by: Lee Benson <[email protected]>

* wip

* Update metrics.rs

* wip

* Create component_events_processed_total.graphql

* fix subscription client; add test

Signed-off-by: Lee Benson <[email protected]>

* allow(clippy::float_cmp)

Signed-off-by: Lee Benson <[email protected]>

* streaming event metrics

Signed-off-by: Lee Benson <[email protected]>

* -println

Signed-off-by: Lee Benson <[email protected]>

* bytes

Signed-off-by: Lee Benson <[email protected]>

* fix SubscriptionClient

Signed-off-by: Lee Benson <[email protected]>

* Update dashboard.rs

* tweaks

Signed-off-by: Lee Benson <[email protected]>

* check-fmt

Signed-off-by: Lee Benson <[email protected]>

* fix pub/sub

Signed-off-by: Lee Benson <[email protected]>

* minor tweak

Signed-off-by: Lee Benson <[email protected]>

* -slab, -anymap

Signed-off-by: Lee Benson <[email protected]>

* single client

Signed-off-by: Lee Benson <[email protected]>

* reduced cloning

Signed-off-by: Lee Benson <[email protected]>

* ignore tests

Signed-off-by: Lee Benson <[email protected]>

* check-fmt

Signed-off-by: Lee Benson <[email protected]>

Co-authored-by: Ana Hobden <[email protected]>
Co-authored-by: Luc Perkins <[email protected]>

.graphqlconfig

@@ -0,0 +1,15 @@
+{
+  "name": "Vector API",
+  "schemaPath": "lib/vector-api-client/graphql/schema.json",
+  "extensions": {
+    "endpoints": {
+      "Default GraphQL Endpoint": {
+        "url": "http://localhost:8686/graphql",
+        "headers": {
+          "user-agent": "Headless GraphQL"
+        },
+        "introspect": false
+      }
+    }
+  }
+}

Cargo.toml

@@ -103,16 +103,16 @@ goauth = { version = "0.7.1", optional = true }
 smpl_jwt = { version = "0.5.0", optional = true }
 
 # API
-async-graphql = { version = "2.0.8", optional = true }
-async-graphql-warp = { version = "2.0.8", optional = true }
-slab = { version = "0.4.2", optional = true }
-anymap = {version = "0.12.1", optional = true }
+async-graphql = { version = "2.0.15", optional = true }
+async-graphql-warp = { version = "2.0.15", optional = true }
+itertools = { version = "0.9.0", optional = true }
 
 # API client
 human_format = { version = "1.0.3", optional = true }
 num-format = { version = "0.4.0", optional = true }
 crossterm = { version = "0.18.0", optional = true }
 tui = { version = "0.12.0", optional = true, default-features = false, features = ["crossterm"] }
+bus = { version = "2.2.3", optional = true }
 
 # Remap Lang
 remap = { package = "remap-lang", path = "lib/remap-lang" }
@@ -281,8 +281,7 @@ kubernetes = ["k8s-openapi", "evmap"]
 api = [
   "async-graphql",
   "async-graphql-warp",
-  "slab",
-  "anymap",
+  "itertools",
 ]
 
 # API client

lib/vector-api-client/graphql/queries/components.graphql

@@ -5,5 +5,8 @@ query ComponentsQuery {
     eventsProcessedTotal {
       eventsProcessedTotal
     }
+    bytesProcessedTotal {
+      bytesProcessedTotal
+    }
   }
 }