From 5f6913637e68e84257d47e2143d2661ef0a1e1cd Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Wed, 3 Apr 2024 12:19:17 +0300 Subject: [PATCH 1/8] added steps for downloading archives from GCP --- docs/integrators/deep-history-squad.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index c3da3920c..4e5b6a2e0 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -103,9 +103,10 @@ Never attach a non-pruned database to a regular observer (i.e. that does not hav Archives supporting historical lookup are available to download from a Google Cloud Storage bucket. -:::important -Documentation in this section is preliminary and subject to change. -::: + In order to download archives, you must have [gcloud CLI](https://cloud.google.com/sdk/docs/install) installed and an account on Google Cloud Platform with [billing](https://cloud.google.com/billing/docs/) enabled. +``` +gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_00000.tar ~/Downloads [--billing-project=BILLING_PROJECT] +``` [comment]: # (mx-context-auto) From 572da0cdfc80c17951c7b30abec137e41cfcb0e0 Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Wed, 3 Apr 2024 15:13:16 +0300 Subject: [PATCH 2/8] Added steps for downloading archives from GCP --- docs/integrators/deep-history-squad.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index 4e5b6a2e0..ab26d20e9 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -103,9 +103,29 @@ Never attach a non-pruned database to a regular observer (i.e. that does not hav Archives supporting historical lookup are available to download from a Google Cloud Storage bucket. - In order to download archives, you must have [gcloud CLI](https://cloud.google.com/sdk/docs/install) installed and an account on Google Cloud Platform with [billing](https://cloud.google.com/billing/docs/) enabled. +In order to avoid unintentional downloads and promote careful fetching of archives, we've enabled [requester pays](https://cloud.google.com/storage/docs/requester-pays). + +### Requirements + +1. **Google Cloud Platform Account**: An account on Google Cloud Platform with billing enabled is required. Find out how you can manage your billing account and modify your project here: +- https://cloud.google.com/billing/docs/how-to/manage-billing-account +- https://cloud.google.com/billing/docs/how-to/modify-project + +2. **Google Cloud SDK**: The Google Cloud SDK includes the `gcloud` command-line tool, which you'll use to interact with Google Cloud Storage. You can download it from the [Google Cloud SDK webpage](https://cloud.google.com/sdk/docs/install). + +### Downloading archives + +Once you have the Google Cloud SDK installed and you're [authenticated](https://cloud.google.com/docs/authentication/gcloud), you can download archives from the Google Cloud Storage bucket using the `gcloud storage cp` command. + +Here's an example command that downloads an archive from the `multiversx-deep-history-archives-mainnet/shard-0/` bucket: +``` +gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_00000.tar ~/DOWNLOAD_LOCATION --billing-project=BILLING_PROJECT +``` +Replace BILLING_PROJECT with the name of your billing project and ~/DOWNLOAD_LOCATION with the directory where the archives should be downloaded. + +The following example will download epochs starting with Epoch_00010 up to Epoch_00020, for a billing project called *multiversx*: ``` -gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_00000.tar ~/Downloads [--billing-project=BILLING_PROJECT] +gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_000{10..20}.tar ~/Downloads/ --billing-project=multiversx ``` [comment]: # (mx-context-auto) From dc41f6ee36edf55392a067e1861c37e873f3a9ea Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Wed, 3 Apr 2024 15:15:22 +0300 Subject: [PATCH 3/8] Added steps for downloading archives from GCP --- docs/integrators/deep-history-squad.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index ab26d20e9..63e83dce1 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -123,9 +123,9 @@ gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_00 ``` Replace BILLING_PROJECT with the name of your billing project and ~/DOWNLOAD_LOCATION with the directory where the archives should be downloaded. -The following example will download epochs starting with Epoch_00010 up to Epoch_00020, for a billing project called *multiversx*: +The following example will download epochs starting with Epoch_01000 up to Epoch_01300, for a billing project called *multiversx*: ``` -gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_000{10..20}.tar ~/Downloads/ --billing-project=multiversx +gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_0{1000..1300}.tar ~/Downloads/ --billing-project=multiversx ``` [comment]: # (mx-context-auto) From 6fe71dd57cf162c14bb21951c3d23f99ad3398b8 Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Wed, 3 Apr 2024 15:30:05 +0300 Subject: [PATCH 4/8] Added steps for downloading archives from GCP --- docs/integrators/deep-history-squad.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index 63e83dce1..08afd527a 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -121,9 +121,9 @@ Here's an example command that downloads an archive from the `multiversx-deep-hi ``` gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_00000.tar ~/DOWNLOAD_LOCATION --billing-project=BILLING_PROJECT ``` -Replace BILLING_PROJECT with the name of your billing project and ~/DOWNLOAD_LOCATION with the directory where the archives should be downloaded. +Replace **BILLING_PROJECT** with the name of your billing project and **~/DOWNLOAD_LOCATION** with the directory where the archives should be downloaded. -The following example will download epochs starting with Epoch_01000 up to Epoch_01300, for a billing project called *multiversx*: +The following example will download epochs starting with Epoch_01000.tar up to Epoch_01300.tar, for a billing project called **multiversx**: ``` gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_0{1000..1300}.tar ~/Downloads/ --billing-project=multiversx ``` From 7973fa2cd4042760487aa599449a9d282271b603 Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Wed, 3 Apr 2024 16:57:29 +0300 Subject: [PATCH 5/8] Added steps for downloading archives from GCP --- docs/integrators/deep-history-squad.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index 08afd527a..40afec6a2 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -101,9 +101,9 @@ Never attach a non-pruned database to a regular observer (i.e. that does not hav ## Downloading non-pruned database -Archives supporting historical lookup are available to download from a Google Cloud Storage bucket. +Archives supporting historical lookup are available to download from a Google Cloud Storage [bucket](https://console.cloud.google.com/storage/browser/multiversx-deep-history-archives-mainnet). -In order to avoid unintentional downloads and promote careful fetching of archives, we've enabled [requester pays](https://cloud.google.com/storage/docs/requester-pays). +In order to avoid unintentional downloads and promote careful fetching of archives, we've enabled the [requester pays](https://cloud.google.com/storage/docs/requester-pays) feature on the bucket that holds the deep-history archives for mainnet. ### Requirements @@ -111,13 +111,13 @@ In order to avoid unintentional downloads and promote careful fetching of archiv - https://cloud.google.com/billing/docs/how-to/manage-billing-account - https://cloud.google.com/billing/docs/how-to/modify-project -2. **Google Cloud SDK**: The Google Cloud SDK includes the `gcloud` command-line tool, which you'll use to interact with Google Cloud Storage. You can download it from the [Google Cloud SDK webpage](https://cloud.google.com/sdk/docs/install). +2. **Google Cloud SDK**: The Google Cloud SDK includes the `gcloud` command-line tool, which you'll use to interact with Google Cloud Storage. In order to install it, please follow the provided on the [Google Cloud SDK webpage](https://cloud.google.com/sdk/docs/install). ### Downloading archives Once you have the Google Cloud SDK installed and you're [authenticated](https://cloud.google.com/docs/authentication/gcloud), you can download archives from the Google Cloud Storage bucket using the `gcloud storage cp` command. -Here's an example command that downloads an archive from the `multiversx-deep-history-archives-mainnet/shard-0/` bucket: +Here's an example command that downloads an archive from the `multiversx-deep-history-archives-mainnet` bucket: ``` gcloud storage cp gs://multiversx-deep-history-archives-mainnet/shard-0/Epoch_00000.tar ~/DOWNLOAD_LOCATION --billing-project=BILLING_PROJECT ``` From fbcd3d563fea993a24966e785d29a984b21001f2 Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Wed, 3 Apr 2024 17:02:41 +0300 Subject: [PATCH 6/8] Added steps for downloading archives from GCP --- docs/integrators/deep-history-squad.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index 40afec6a2..35da1e59c 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -111,7 +111,7 @@ In order to avoid unintentional downloads and promote careful fetching of archiv - https://cloud.google.com/billing/docs/how-to/manage-billing-account - https://cloud.google.com/billing/docs/how-to/modify-project -2. **Google Cloud SDK**: The Google Cloud SDK includes the `gcloud` command-line tool, which you'll use to interact with Google Cloud Storage. In order to install it, please follow the provided on the [Google Cloud SDK webpage](https://cloud.google.com/sdk/docs/install). +2. **Google Cloud SDK**: The Google Cloud SDK includes the `gcloud` command-line tool, which you'll use to interact with Google Cloud Storage. In order to install it, please follow the instructions provided on the [Google Cloud SDK webpage](https://cloud.google.com/sdk/docs/install). ### Downloading archives From f2410b5e2e83bd3b94aa225461bfca20085d073a Mon Sep 17 00:00:00 2001 From: Daniel Drasovean Date: Thu, 4 Apr 2024 11:09:28 +0300 Subject: [PATCH 7/8] Move and update Historical VM queries section --- docs/integrators/deep-history-squad.md | 40 +++++++++++--------------- 1 file changed, 17 insertions(+), 23 deletions(-) diff --git a/docs/integrators/deep-history-squad.md b/docs/integrators/deep-history-squad.md index 35da1e59c..8d6d343ee 100644 --- a/docs/integrators/deep-history-squad.md +++ b/docs/integrators/deep-history-squad.md @@ -39,6 +39,23 @@ In the example above, the key `726573657276650000000a55544b2d326638306539` is de [comment]: # (mx-context-auto) +## Historical VM queries + +Starting with the [Sirius Patch 5](https://github.com/multiversx/mx-chain-mainnet-config/releases/tag/v1.6.18.0), deep-history observers can resolve historical VM queries. Such a query specifies the `blockNonce` parameter: + +``` +POST http://localhost:8080/vm-values/query?blockNonce={...} HTTP/1.1 +Content-Type: application/json + +{ + "scAddress": "...", + "funcName": "...", + "args": [ ... ] +} +``` + +[comment]: # (mx-context-auto) + ## MultiversX squad The observing squads backing the public Gateways, in addition to being full history squads (serving past blocks, transactions and events up until the Genesis), also act as 3-epochs deep-history squads. That is, for **mainnet**, one can use https://gateway.multiversx.com to resolve historical account (state) queries, for the last 3 days. This interval is driven by the configuration parameter `[StoragePruning.NumEpochsToKeep]`, which is set to `4`, by default. @@ -310,29 +327,6 @@ Once the **import-db** is over, the `db` folder can be attached to a deep-histor [comment]: # (mx-context-auto) -## Historical VM queries - -:::important -Documentation in this section is preliminary and subject to change. -::: - -Starting with the [Sirius Mainnet Upgrade](https://github.com/multiversx/mx-specs/blob/main/releases/protocol/release-specs-v1.6.0-Sirius.md), deep-history observers can resolve VM queries up to `[StoragePruning.NumActivePersisters]` (by default, 3) epochs in the past. Such a query specifies the `blockNonce` parameter: - -``` -POST http://localhost:8080/vm-values/query?blockNonce={...} HTTP/1.1 -Content-Type: application/json - -{ - "scAddress": "...", - "funcName": "...", - "args": [ ... ] -} -``` - -VM queries that refer older data aren't supported as of January 2024 (Sirius). This will be fixed in a future release. - -[comment]: # (mx-context-auto) - ## Starting a squad Suppose you have prepared the data for a deep-history squad beforehand, whether by downloading it or by reconstructing it locally. Then, the deep-history data root folder should look as follows: From 061c28b983d7e12f61701549b919ca83a4ef0b97 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andrei=20B=C4=83ncioiu?= Date: Thu, 4 Apr 2024 11:54:59 +0300 Subject: [PATCH 8/8] v13 cookbook: events, tx watcher. --- .../sdk-js/sdk-js-cookbook-v13.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/docs/sdk-and-tools/sdk-js/sdk-js-cookbook-v13.md b/docs/sdk-and-tools/sdk-js/sdk-js-cookbook-v13.md index 5330acfa9..39c9472e2 100644 --- a/docs/sdk-and-tools/sdk-js/sdk-js-cookbook-v13.md +++ b/docs/sdk-and-tools/sdk-js/sdk-js-cookbook-v13.md @@ -180,6 +180,19 @@ await Promise.all([ ]); ``` +In some circumstances, when awaiting for a transaction completion in order to retrieve its logs and events, +it's possible that these pieces of information are missing at the very moment the transaction is marked as completed - +they may not be immediately available. + +If that is an issue, you can configure the [`TransactionWatcher`](https://multiversx.github.io/mx-sdk-js-core/v13/classes/TransactionWatcher.html) to have additional **patience** +before returning the transaction object. Below, we're adding a patience of 8 seconds: + +``` +const watcherWithPatience = new TransactionWatcher(apiNetworkProvider, { patienceMilliseconds: 8000 }); +``` + +Alternatively, use [`TransactionWatcher.awaitAnyEvent()`](https://multiversx.github.io/mx-sdk-js-core/v13/classes/TransactionWatcher.html#awaitAnyEvent) or [`TransactionWatcher.awaitOnCondition()`](https://multiversx.github.io/mx-sdk-js-core/v13/classes/TransactionWatcher.html#awaitOnCondition) to customize the waiting strategy. + For a different awaiting strategy, also see [extending sdk-js](https://docs.multiversx.com/sdk-and-tools/sdk-js/extending-sdk-js). ## Token transfers @@ -742,6 +755,36 @@ const parsedOutcome = parser.parseExecute({ transactionOutcome }); console.log(parsedOutcome); ``` +### Decode transaction events + +Additionally, you might be interested into decoding the events emitted by a contract. +You can do so by means of the [`TransactionEventsParser`](https://multiversx.github.io/mx-sdk-js-core/v13/classes/TransactionEventsParser.html). + +Suppose we'd like to decode a `startPerformAction` event emitted by the [**multisig**](https://github.com/multiversx/mx-contracts-rs/tree/main/contracts/multisig) contract. + +Let's fetch [a previously-processed transaction](https://devnet-explorer.multiversx.com/transactions/05d445cdd145ecb20374844dcc67f0b1e370b9aa28a47492402bc1a150c2bab4), +to serve as an example, and convert it to a [`TransactionOutcome`](https://multiversx.github.io/mx-sdk-js-core/v13/classes/TransactionOutcome.html) (see above why): + +``` +const transactionOnNetworkMultisig = await apiNetworkProvider.getTransaction("05d445cdd145ecb20374844dcc67f0b1e370b9aa28a47492402bc1a150c2bab4"); +const transactionOutcomeMultisig = converter.transactionOnNetworkToOutcome(transactionOnNetworkMultisig); +``` + +Now, let's find and parse the event we are interested in: + +``` +import { TransactionEventsParser, findEventsByFirstTopic } from "@multiversx/sdk-core"; + +const eventsParser = new TransactionEventsParser({ + abi: abiMultisig +}); + +const [event] = findEventsByFirstTopic(transactionOutcomeMultisig, "startPerformAction"); +const parsedEvent = eventsParser.parseEvent({ event }); + +console.log(parsedEvent); +``` + ## Contract queries In order to perform Smart Contract queries, we recommend the use of [`SmartContractQueriesController`](https://multiversx.github.io/mx-sdk-js-core/v13/classes/SmartContractQueriesController.html).