Skip to content

Commit

Permalink
Updated all WAX Hyperion Technical How To guides to v3.3.10-1 (#192)
Browse files Browse the repository at this point in the history 
* Update wax-intro-to-hyperion-full-history.md v3.3.10-1

* Update wax-build-hyperion-software-components.md v3.3.10-1

* Update wax-config-hyperion-software-components.md v.3.3.10-1

* Update wax-running-hyperion.md v3.3.10-1

* Update wax-clustering-elasticsearch-for-hyperion.md v3.3.10-1

* Update wax-rectify-missing-blocks-in-hyperion.md v3.3.10-1

* Update wax-hyperion-indexer-scaling.md v3.3.10-1

* Update wax-hyperion-streaming-load-balancer.md v3.3.10-1

* Update wax-hyperion-repair-indexed-data.md v3.3.10-1

* Update wax-hyperion-data-tiers.md - Fixed Screen Snips
  • Loading branch information
@Rossco99
Rossco99 authored Feb 10, 2025
1 parent cbdf3b6 commit 45d1da5
Show file tree
Hide file tree
Showing 10 changed files with 51 additions and 39 deletions.
27 changes: 18 additions & 9 deletions docs/en/operate/wax-hyperion/wax-build-hyperion-software-components.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Once again this Technical How To series will cover some of the EOS RIO same cont

![](https://miro.medium.com/v2/resize:fit:598/0*QSOXBoNEcm0pWwSl.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# Build WAX Hyperion Software Components

Expand All @@ -24,11 +24,11 @@ The process for building each of these primary building blocks is covered below:

## **WAX Software State-History (SHIP) Node**

The WAX Hyperion deployment requires access to a fully sync’d WAX State-History Node, the current recommend version is `v4.0.4wax02`. This build process is extensively covered in [WAX Technical How To #7](https://medium.com/eosphere/wax-technical-how-to-7-9ccc102efd9d).
The WAX Hyperion deployment requires access to a fully sync’d WAX State-History Node, the current recommend version is `v5.0.1wax01`. This build process is extensively covered in [WAX Technical How To #7](https://medium.com/eosphere/wax-technical-how-to-7-9ccc102efd9d).

## RabbitMQ

To install the latest RabbitMQ currently `3.12.6` be sure to check their latest [Cloudsmith Quick Start Script](https://www.rabbitmq.com/install-debian.html), this in our experience is the simplest way to ensure you are current and correctly built.
To install the latest RabbitMQ currently `4.0.5` and Erlang `27.2` be sure to check their latest [Cloudsmith Quick Start Script](https://www.rabbitmq.com/install-debian.html), this in our experience is the simplest way to ensure you are current and correctly built.

The summary process is below:

Expand Down Expand Up @@ -75,9 +75,16 @@ EOF
**Check Version**
> sudo rabbitmqctl version
```

If you performed an upgrade to RabbitMQ version 4 you may need to enable all stable feature flags as below:

```
> sudo rabbitmqctl enable_feature_flag all
```

## Redis

Our current WAX Hyperion deployment is running on the latest Redis stable version `v7.2.1` which is built as below:
Our current WAX Hyperion deployment is running on the latest Redis stable version `v7.3.1` which is built as below:

```
> sudo apt install lsb-release curl gpg
Expand All @@ -97,7 +104,7 @@ Our current WAX Hyperion deployment is running on the latest Redis stable versio

## Node.js

Hyperion requires Node.js v18 , our WAX Hyperion deployment is running the current LTS `v18.18.0` which is built below:
Hyperion requires Node.js v22 , our WAX Hyperion deployment is running the current LTS `v22.12.0` which is built below:

```
#Download and import the Nodesource GPG key#
Expand All @@ -110,7 +117,7 @@ Hyperion requires Node.js v18 , our WAX Hyperion deployment is running the curre
> curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
#Create .deb repository#
> NODE_MAJOR=18
> NODE_MAJOR=22
> echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
Expand All @@ -125,7 +132,7 @@ Hyperion requires Node.js v18 , our WAX Hyperion deployment is running the curre

## PM2

The latest public version is `5.3.0` and is built as below:
The latest public version is `5.4.3` and is built as below:

```
> sudo apt update
Expand All @@ -139,7 +146,7 @@ The latest public version is `5.3.0` and is built as below:

## Elasticsearch

Currently our WAX Hyperion is using Elasticsearch `8.5.2` with great results, however the current recommended Elasticsearch version is `8.9.0` which I expect will work just as well or better. Build the latest Elasticsearch `8.x` as below:
Currently our WAX Hyperion is using Elasticsearch `8.13.2` with great results, however the current recommended Elasticsearch version is `8.17.0` which I expect will work just as well or better. Build the latest Elasticsearch `8.x` as below:

```
> sudo apt update
Expand Down Expand Up @@ -183,7 +190,7 @@ The utilised Kibana version should be paired with the installed Elasticsearch ve

## **EOS RIO Hyperion Indexer and API**

Currently (September 2023) the most robust and production ready version of Hyperion from our experience is `3.3.9–8` and is used in our WAX Hyperion Fully History Service. The EOS RIO Team are constantly developing and improving their code, the best way to stay on top of the current recommend version is to join the [Hyperion Telegram Group](https://t.me/EOSHyperion). Build Hyperion from `main` as below:
Currently (December 2024) the most robust and production ready version of Hyperion from our experience is `3.3.10-1` and is used in our WAX Hyperion Fully History Service. The EOS RIO Team are constantly developing and improving their code, the best way to stay on top of the current recommend version is to join the [Hyperion Telegram Group](https://t.me/EOSHyperion). Build Hyperion from `main` as below:

```
> git clone https://github.com/eosrio/hyperion-history-api.git
Expand All @@ -197,6 +204,8 @@ Currently (September 2023) the most robust and production ready version of Hyper
> npm audit fix
```

_Hyperion version '3.5.0' is expected to release in the next few weeks, this guide will be updated accordingly._

After all Hyperion Software Components are built and provisioned you can now proceed to configuration.

The next **WAX Hyperion Full History** sub article will walk through the technical configuration of each component.
Expand Down
2 changes: 1 addition & 1 deletion docs/en/operate/wax-hyperion/wax-clustering-elasticsearch-for-hyperion.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Once again this Technical How To series may cover some of EOS RIO’s same conte

![](https://miro.medium.com/v2/resize:fit:598/0*SMEjl3XTe2BtL1W3.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

## Clustering Elasticsearch for WAX Hyperion Full History

Expand Down
22 changes: 11 additions & 11 deletions docs/en/operate/wax-hyperion/wax-config-hyperion-software-components.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

![](https://miro.medium.com/v2/resize:fit:598/0*e9la30mQaOGWxfmM.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# Configure WAX Hyperion Software Components

Expand Down Expand Up @@ -95,20 +95,20 @@ Debug Redis issues in the logs:

## Node.js

Nothing to configure here, however ensure you are running Node.js v16
Nothing to configure here, however ensure you are running Node.js v22

```
> node -v
v18.x.x
v22.x.x
```

## PM2

Nothing to configure here, check that you are running the latest PM2 version`5.3.0`
Nothing to configure here, check that you are running the latest PM2 version`5.4.3`

```
> pm2 --version
5.3.0
5.4.3
```

## Elasticsearch
Expand Down Expand Up @@ -194,15 +194,15 @@ Enter host password for user 'elastic':
{
"name" : "<YOUR NODE NAME>",
"cluster_name" : "<YOUR CLUSTER NAME>",
"cluster_uuid" : "exucKwVpRJubHGq5Jwu1_Q",
"cluster_uuid" : "36T00kPBT6qq8Tb1JYUgCw",
"version" : {
"number" : "8.10.1",
"number" : "8.13.2",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "a94744f97522b2b7ee8b5dc13be7ee11082b8d6b",
"build_date" : "2023-09-14T20:16:27.027355296Z",
"build_hash" : "16cc90cd2d08a3147ce02b07e50894bc060a4cbf",
"build_date" : "2024-04-05T14:45:26.420424304Z",
"build_snapshot" : false,
"lucene_version" : "9.7.0",
"lucene_version" : "9.10.0",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
Expand Down Expand Up @@ -483,7 +483,7 @@ Configure as below, take note of the **#UPDATE#** parameters
"root_only": false
},
"scaling": {
"readers": 2, #INCREASE READERS#
"readers": 1,
"ds_queues": 1,
"ds_threads": 1,
"ds_pool_size": 1,
Expand Down
11 changes: 6 additions & 5 deletions docs/en/operate/wax-hyperion/wax-hyperion-data-tiers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

[Learn more about EOS RIO Hyperion](https://eosrio.io/hyperion/)

![image](https://github.com/user-attachments/assets/2e4b17c5-8d89-49f1-bf97-76e39748ab91)
![image](https://github.com/user-attachments/assets/2ce31a7d-8a87-48a4-b85a-b5bb1e0685a8)

# Using Data Tiers with WAX Hyperion Full History

Expand Down Expand Up @@ -52,15 +52,16 @@ The indices we are most interested in are the `wax-action` and `wax-delta` i
> GET /_cat/indices/wax*
```

![image](https://github.com/user-attachments/assets/e5391d4a-0129-4348-a4a4-d96a04189cc1)
![image](https://github.com/user-attachments/assets/8c46dd0d-34d4-4a9c-a058-d09b8a071b07)


The specific settings below show the existing data tier, as expected `data_content` by default.

```
> GET /wax-action-v1-000023/_settings
```

![image](https://github.com/user-attachments/assets/684b0011-bc93-40fa-a772-80f7f637622f)
![image](https://github.com/user-attachments/assets/82830550-359f-4129-94a3-403ff4803d1b)

## Set Index Tier

Expand All @@ -73,7 +74,7 @@ Set the desired older index to `data_warm`.
}
```

![image](https://github.com/user-attachments/assets/a57ceafd-8036-4853-935d-10f00622fb01)
![image](https://github.com/user-attachments/assets/c9ced417-bb61-4e37-9a66-27c623c36f47)

## Observe Index Allocation

Expand All @@ -88,7 +89,7 @@ wax-action-v1-000023 wax-es-node-1 -> 10.125.0.89 fQeNr0UkSXuAnofq-rceoA wax-es-
wax-action-v1-000023 wax-es-node-1 -> 10.125.0.89 fQeNr0UkSXuAnofq-rceoA wax-es-node-3
```

![image](https://github.com/user-attachments/assets/a50ee046-b3a7-4956-882f-776b4d4e409b)
![image](https://github.com/user-attachments/assets/e68130ee-7215-4272-8451-83331242f6e7)

The move process may take some time and appears to use very little resources, which is obviously great for an in production cluster.

Expand Down
4 changes: 2 additions & 2 deletions docs/en/operate/wax-hyperion/wax-hyperion-indexer-scaling.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

![](https://miro.medium.com/v2/resize:fit:598/0*KXa0o0Fa-YGqItzy.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# WAX Hyperion Full History Indexer Scaling

Expand Down Expand Up @@ -141,7 +141,7 @@ Here are a few suggestions on starting your indexer for the first time and then

```
"scaling": {
"readers": 2,
"readers": 1,
"ds_queues": 2,
"ds_threads": 2,
"ds_pool_size": 6,
Expand Down
2 changes: 1 addition & 1 deletion docs/en/operate/wax-hyperion/wax-hyperion-repair-indexed-data.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

![](https://miro.medium.com/v2/resize:fit:598/0*GqDtvMh8QunDs6Mc.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# Repair WAX Hyperion Indexed Data

Expand Down
2 changes: 1 addition & 1 deletion docs/en/operate/wax-hyperion/wax-hyperion-streaming-load-balancer.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

![](https://miro.medium.com/v2/resize:fit:598/0*VJM9YUUmYPzi9zEc.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# WAX Hyperion Streaming with a WAX Load Balancer

Expand Down
7 changes: 4 additions & 3 deletions docs/en/operate/wax-hyperion/wax-intro-to-hyperion-full-history.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ EOSphere have extensive production experience with providing Full History servic

![](https://miro.medium.com/v2/resize:fit:598/1*ATnQKunF5rVtPEsL-sP0JA.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# Introduction to WAX Hyperion Full History

Expand Down Expand Up @@ -131,13 +131,14 @@ The recommendations below are for WAX Mainnet and WAX Testnet with Full History,
* Modern CPU, 3Ghz+, 8 Cores+
* 64GB RAM
* Enterprise Grade SSD/NVMe
_The current (September 2023) Elasticsearch Database is 26TB, I suggest provisioning 35–40TB across the cluster for Full History service longevity_
_The current (December 2024) Elasticsearch Database is 32TB, I suggest provisioning 40–45TB across the cluster for Full History service longevity_
* 1Gb/s+ LAN
* **WAX State-History**
* Modern CPU, 4Ghz+, 4 Cores
* 128GB RAM
* 256GB DISK 1 _(Enterprise Grade SSD/NVMe)_
* 16TB DISK 2 _(SAS or SATA are OK)_
* 18TB+ DISK 2 _(SAS or SATA are OK)_
* 1Gb/s+ LAN

**WAX Testnet**

Expand Down
4 changes: 2 additions & 2 deletions docs/en/operate/wax-hyperion/wax-rectify-missing-blocks-in-hyperion.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Rectify Missing Blocks in WAX Hyperion Full History
---

Initially indexing the complete WAX Mainnet from genesis will take time and it can be quite heavy on hardware resources causing pauses or stops in indexing, furthermore if you have taken our suggestion on batching block ranges to index you may encounter a scenario where you have gaps in your full history indexed blocks set.
This next sub-article in the series will go through the process of identifying gaps in your full history blocks set and how to manually correct them.
This next sub-article in the series will go through the process of identifying gaps in your full history blocks set and how to *manually* correct them.

EOS RIO have an excellent [Hyperion Documentation Repository](https://hyperion.docs.eosrio.io/) including details on how to run their Hyperion Full History product, however this article expands on their documentation specifically in relation to operating Hyperion using Elasticsearch 8.x

Expand All @@ -13,7 +13,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

![](https://miro.medium.com/v2/resize:fit:449/0*pY_GWzqlqGfAO7Kf.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

## Rectify Missing Blocks in WAX Hyperion Full History

Expand Down
9 changes: 5 additions & 4 deletions docs/en/operate/wax-hyperion/wax-running-hyperion.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Once again this Technical How To series will cover some of EOS RIO’s same cont

![](https://miro.medium.com/v2/resize:fit:598/0*K8f_xpc-l5NSUL7H.png)

_This article has been updated to reflect the current Hyperion deployment in September 2023._
_This article has been updated to reflect the current Hyperion deployment in December 2024._

# Running WAX Hyperion Full History

Expand Down Expand Up @@ -170,7 +170,7 @@ Below is the initial configuration used for `wax.config.json` :
"root_only": false
},
"scaling": {
"readers": 2, #INCREASE READERS#
"readers": 1,
"ds_queues": 1,
"ds_threads": 1,
"ds_pool_size": 1,
Expand Down Expand Up @@ -252,7 +252,7 @@ Make sure the following is configured or amended in the `wax.config.json` :
"live_reader": false, #DISABLED FOR BULK INDEXING#
"abi_scan_mode": false, #SET FOR INDEXING PHASE#
"scaling": { #CONSERVATIVE SETTINGS#
"readers": 2,
"readers": 1,
"ds_queues": 1,
"ds_threads": 2,
"ds_pool_size": 2,
Expand Down Expand Up @@ -366,8 +366,9 @@ In particular make sure `last_indexed_block` is equal to `total_indexed_block
```
> curl [http://<SERVER IP ADDRESS>:7000/v2/health](https://wax-testnet.eosphere.io/v2/health)
{"version":"3.3.9-8","version_hash":"b94f99d552a8fe85a3ab2c1cb5b84ccd6ded6af4","host":"wax-testnet.eosphere.io","health":[{"service":"RabbitMq","status":"OK","time":1695700845755},{"service":"NodeosRPC","status":"OK","service_data":{"head_block_num":268459315,"head_block_time":"2023-09-26T04:00:45.500","time_offset":210,"last_irreversible_block":268458983,"chain_id":"1064487b3cd1a897ce03ae5b6a865651747e2e152090f99c1d19d44e01aea5a4"},"time":1695700845710},{"service":"Elasticsearch","status":"OK","service_data":{"active_shards":"100.0%","head_offset":2,"first_indexed_block":2,"last_indexed_block":268459313,"total_indexed_blocks":268459311,"missing_blocks":0,"missing_pct":"0.00%"},"time":1695700845712}],"features":{"streaming":{"enable":true,"traces":true,"deltas":true},"tables":{"proposals":true,"accounts":true,"voters":true},"index_deltas":true,"index_transfer_memo":true,"index_all_deltas":true,"deferred_trx":false,"failed_trx":false,"resource_limits":false,"resource_usage":false},"cached":true,"query_time_ms":0.158,"last_indexed_block":268459318,"last_indexed_block_time":"2023-09-26T04:00:47.000"}
{"version":"3.3.10","version_hash":"3c24fea14d424eeef5fe2719238ef4e2ee65dc4e","host":"wax-testnet.eosphere.io","health":[{"service":"RabbitMq","status":"OK","time":1735014759377},{"service":"NodeosRPC","status":"OK","service_data":{"head_block_num":347074519,"head_block_time":"2024-12-24T04:32:39.500","time_offset":-165,"last_irreversible_block":347074187,"chain_id":"f16b1833c747c43682f4386fca9cbb327929334a762755ebec17f6f23c9b8a12"},"time":1735014759335},{"service":"Elasticsearch","status":"OK","service_data":{"active_shards":"100.0%","head_offset":3,"first_indexed_block":2,"last_indexed_block":347074516,"total_indexed_blocks":347074514,"missing_blocks":0,"missing_pct":"0.00%"},"time":1735014759338}],"features":{"streaming":{"enable":true,"traces":true,"deltas":true},"tables":{"proposals":true,"accounts":true,"voters":true},"index_deltas":true,"index_transfer_memo":true,"index_all_deltas":true,"deferred_trx":false,"failed_trx":false,"resource_limits":false,"resource_usage":false},"cached":true,"query_time_ms":0.323,"last_indexed_block":347074522,"last_indexed_block_time":"2024-12-24T04:32:41.000"}
```

Congratulations you have now successfully built, configured and are running a WAX Hyperion Full History Service, ready to be made publicly available from behind a SSL offloading Load Balancer such as HAProxy.

The next **WAX Hyperion Full History** sub article with cover missing blocks troubleshooting, Elasticsearch monitoring as well as ensuring Hyperion web services are available.
Expand Down

0 comments on commit 45d1da5

Please sign in to comment.