Merge pull request #130 from sablier-labs/feat/subgraph-updates

feat: API (merkle, envio) and App (airstreams) updates

docs/api/01-overview.mdx

@@ -0,0 +1,88 @@
+---
+id: "overview"
+sidebar_position: 1
+title: "Overview"
+---
+
+import LinkPreview from "@site/src/components/LinkPreview";
+
+# The Sablier APIs
+
+Sablier relies on specific dependencies to source data or manage off-chain flows. All of these are either public or
+fully open-source, so feel free to roam around and suggest improvements or optimizations where needed.
+
+## Subgraphs and Indexers (GraphQL)
+
+As an alternative to reading data from the contracts or listening to onchain events, we use a set of subgraphs and
+indexers. These act as a middleware between the chain and our interfaces and allow for caching, formatting and querying
+data.
+
+### The Graph's Subgraphs
+
+[The Graph](https://thegraph.com/) has been the vendor of choice for the EVM space for the past few years. We've been
+integrating The Graph since 2019 with Sablier V1. For Sablier V2, we designed a special set of feature-oriented
+subgraphs used not only by us (in the official apps) but by 3rd party services (e.g. snapshot) and integrators alike.
+
+These subgraphs are hosted on The Graph Network, as well as on the The Graph Hosted Service. They can be used to query
+Sablier data from the official [endpoints](/api/subgraphs/endpoints). Every network has its own subgraph endpoints.
+
+Read more on the dedicated page about our `protocol` and `merkle` [subgraphs](/api/subgraphs/overview) or check out the
+[official vendor docs](https://thegraph.com/docs/en/quick-start/).
+
+<LinkPreview
+  href="https://github.com/sablier-labs/v2-subgraphs"
+  icon="github"
+  subtitle="@sablier/v2-subgraphs"
+  title="Github - apps/protocol, apps/merkle"
+/>
+
+### Envio's Indexers
+
+[Envio](https://envio.dev/) offers a suite of fast and flexible tools to access on-chain data. Their HyperIndex service
+provides for a similar GraphQL driven API to access cached data and serve it into our client interfaces.
+
+We designed a set of indexers that mimic the features and entities exposed by subgraphs and take advantage of the speed
+and optimized environment configured through HyperIndex. Read more on the dedicated page about our `protocol-envio` and
+`merkle-envio` [indexers](/api/indexers/overview).
+
+All networks share the same indexer endpoint (for one indexer type), as Envio promotes a cross-chain indexing
+architecture.
+
+:::tip
+
+Before diving into integrating Envio's indexer as a data source for your application, please read the
+[main differences](/api/indexers/differences) between subgraphs and indexers (generally as well as applied to the
+Sablier deployments).
+
+:::
+
+<LinkPreview
+  href="https://github.com/sablier-labs/v2-subgraphs"
+  icon="github"
+  subtitle="@sablier/v2-subgraphs"
+  title="Github - apps/protocol-envio, apps/merkle-envio"
+/>
+
+## Merkle API (Rust)
+
+To support our Merkle distribution (a.k.a. [Airstreams](/apps/features#airstreams)) system we developed a backend
+service called `@sablier/v2-merkle-api`. This backend deals with the validation, creation and management of Merkle
+trees, used to define eligibility and claiming rules for Sablier's Airstream campaigns.
+
+This service is open-source and can be used by integrators as a plug-n-play solution to support (and even deploy)
+similar campaigns from their own dashboards. Read more on the dedicated page about the `merkle-api`
+[backend](/api/merkle-api/overview).
+
+<LinkPreview
+  href="https://github.com/sablier-labs/v2-merkle-api"
+  icon="github"
+  subtitle="@sablier/v2-merkle-api"
+  title="Github - Rust based API to generate Merkle distribution"
+/>
+
+## Explorer
+
+To preview the GraphQL APIs for subgraphs and indexers you can plug them into
+[Hasura's](https://cloud.hasura.io/public/graphiql) online explorer.
+[Here](https://cloud.hasura.io/public/graphiql?endpoint=https%3A%2F%2Fapi.thegraph.com%2Fsubgraphs%2Fname%2Fsablier-labs%2Fsablier-v2-experimental)'s
+an example setup for the `sablier-v2` mainnet subgraph inside Hasura's explorer.

docs/api/indexers/01-endpoints.mdx

@@ -0,0 +1,79 @@
+---
+id: "endpoints"
+sidebar_position: 1
+title: "Endpoints"
+---
+
+import LinkPreview from "@site/src/components/LinkPreview";
+
+:::tip Chains: Individual vs. Aggregated
+
+While subgraphs have a chain-focused approach (one subgraph endpoint/deployment per chain), it is important to note that
+Envio will index all chains within the same deployment. This makes it so data from all chains can be queried from the
+same place.
+
+:::
+
+:::warning Note
+
+The Sablier Envio indexers are still under development so links may change in time. Please reach out if the endpoints
+don't work any more
+
+:::
+
+## Sablier V2 - Protocol-Envio
+
+The `protocol-envio` indexer tracks events mainly emitted by the
+[`@sablier/v2-core`](/contracts/v2/reference/overview#core) contracts. It deals with core protocol actions like
+creating, withdrawing or transferring streams.
+
+### Endpoints
+
+| Chain            | Endpoint (Hosted Network)                                                 |
+| ---------------- | ------------------------------------------------------------------------- |
+| All-Networks[^1] | [https://indexer.bigdevenergy.link/[...]/v1/graphql]([endpoint-protocol]) |
+
+Use [Hasura's online explorer](https://cloud.hasura.io/public/graphiql?) to view the entities and query API.
+
+### Code
+
+<LinkPreview
+  href="https://github.com/sablier-labs/v2-subgraphs/tree/main/apps/protocol-envio"
+  icon="github"
+  subtitle="Github - sablier-labs/v2-subgraphs"
+  title="Sablier V2 protocol subgraph (Envio)"
+/>
+
+## Sablier V2 - Merkle-Envio (Airstreams)
+
+This subgraph tracks events emitted by the [`@sablier/v2-periphery`](/contracts/v2/reference/overview#periphery)
+contracts, specifically the
+[`Merkle Factory`](/contracts/v2/reference/periphery/contract.SablierV2MerkleStreamerFactory). It deals with airstream
+specific actions like the factory creating a campaign, admin clawbacks or users claiming streams as defined in the
+attached Merkle tree.
+
+### Endpoints
+
+| Chain        | Endpoint (Hosted Network)                                               |
+| ------------ | ----------------------------------------------------------------------- |
+| All-Networks | [https://indexer.bigdevenergy.link/[...]/v1/graphql]([endpoint-merkle]) |
+
+Use [Hasura's online explorer](https://cloud.hasura.io/public/graphiql?) to view the entities and query API.
+
+### Code
+
+<LinkPreview
+  href="https://github.com/sablier-labs/v2-subgraphs/tree/main/apps/protocol"
+  icon="github"
+  subtitle="Github - sablier-labs/v2-subgraphs"
+  title="Sablier V2 protocol subgraph (The Graph)"
+/>
+
+[^1]:
+    All-Networks = most of the supported networks from Sablier's app e.g. Optimism, Polygon, Sepolia, Mainnet. The full
+    list can be found in
+    [this](https://github.com/sablier-labs/v2-subgraphs/blob/main/apps/protocol-envio/src/constants/chains/index.ts)
+    configuration file.
+
+[endpoint-protocol]: https://indexer.bigdevenergy.link/be2052e/v1/graphql
+[endpoint-merkle]: https://indexer.bigdevenergy.link/61cf6be/v1/graphql

docs/api/indexers/02-differences.mdx

@@ -0,0 +1,130 @@
+---
+id: "differences"
+sidebar_position: 1
+title: "Differences"
+---
+
+# Differences (The Graph vs. Envio)
+
+:::info
+
+If you're looking to support both subgraphs (The Graph) and indexers (Envio) into your apps, please make sure to scroll
+down to the "Different Results" section to understand how results from both APIs may differ for the same entities.
+
+:::
+
+## General
+
+In contrast to The Graph, Envio offers a slightly different Developer Experience (DX) in terms of indexer
+implementation. While for consumers, the query specifics don't change a lot, here are a few items to consider before
+developing on top of the Sablier indexers/subgraphs:
+
+### Querying Language
+
+Both solutions create a **GraphQL** API for consumers to read data from. It is important to note that The Graph uses a
+customized subset of GraphQL operations, which makes is impossible to use the same queries between both indexers and
+subgraphs, whereas Envio deploys a GraphQL API over a Postgres DB. Some examples:
+
+| Example              | The Graph                             | Envio                                         |
+| -------------------- | ------------------------------------- | --------------------------------------------- |
+| Pagination           | `first`, `skip`                       | `limit`, `offset`                             |
+| Filter by id         | `stream(id)`                          | `Stream(where: {id: {_eq: 1}})`               |
+| Single vs. Plural    | `stream(id){}`, `streams{}`           | `Stream(where...){}` for both                 |
+| Nested items         | `campaigns{ id, asset: {id, symbol}}` | `Campaign{ asset, assetObject: {id, symbol}}` |
+| Filter similar items | `assets(id: $id)`                     | `Asset(where: {id: {_iregex: $id}})`          |
+
+#### Important Notes
+
+For the 3rd example, querying for a single item vs. a collection will have separate keywords for The Graph. With Envio,
+you'll have to identify within the app itself if the query is supposed to yield one or multiple items. As you may tell,
+with Envio you'll query for `Stream` (capitalized) while with The Graph you'll query for `stream` or `streams`.
+
+For the 4th example, in Envio indexers the name of an object will yield its `id`, while the `<name>Object` label will
+ask for the object itself (e.g. `asset` vs `assetObject`).
+
+:::warning
+
+This `asset` vs `assetObject` discrepancy may change in future versions of Envio. Please checks the docs accordingly
+before developing features on top of the Sablier V2 indexers.
+
+:::
+
+### Handler Language
+
+- The Graph: uses Assembly script
+- Envio: we've chosen Typescript, but you can use JS, TS or Rescript
+
+For Envio, one important mental model is in the concept of loaders vs. handlers. To optimize querying speeds, Envio will
+ask you to write 2 methods: a `loader`, where you can express which existing entities you're expecting to use and
+mutate, and a `handler` where you manage these entities and/or create new ones.
+
+Example: In the case of a Withdraw event, we'll **pre-load** the `Stream` entity and maybe the `Watcher`, while in the
+second part we'll **handle** the creation of a new `Action` of type Withdraw, we'll update the `Stream`'s
+`withdrawnAmount` and increase the `Watcher`'s index.
+
+## Specifics
+
+### Initializer Contracts
+
+We've architected this indexer around a set of pre-configured contracts.
+
+Similar to The Graph, we start by pre-configuring a set of contracts. While Envio's indexer doesn't have the same
+[requirement](https://discord.com/channels/438038660412342282/438070183794573313/1153155902933831811) of pre-configuring
+contracts to listen to, we'll keep this feature to ensure we can query against those entities, even if they'll be empty
+at start.
+
+We'll ensure contracts have been initialized (see the `watcher.ts` helper) by making a call against the initializer at
+the start of each method. It should only come into play within the create handlers.
+
+### Versioning and Lockup Flavors
+
+While for The Graph's subgraphs we track flavor-first (see `subgraph.template.yaml` for the configuration of
+`SablierV2LockupLinear` and `SablierV2LockupDynamic`), for Envio's indexers we'll have a version-first approach.
+
+Therefore, `LockupLinear` and `LockupDynamic` will be bundled under the same `Lockup<Version>` contract tracker (see
+`./config.template.mustache`). Different versions of the protocol will be tracked separately, which is why we have
+`Lockup_V20` (v2.0) and `Lockup_V21` (v2.1) in our configuration. Later on, inside the handler logic, we'll separate
+contracts by flavor.
+
+```tree title="Envio: Contract configuration tree (version-first)"
+└── contracts/
+    ├── LockupV20/
+    │   ├── event: CreateLockupDynamicStream
+    │   └── event: CreateLockupLinearStream
+    └── LockupV21/
+        ├── event: CreateLockupDynamicStream
+        └── event: CreateLockupLinearStream
+```
+
+```tree title="The Graph: Contract configuration tree (flavor-first)"
+└── contracts/
+    ├── LockupDynamic/
+    │   ├── event: CreateLockupDynamicStreamV20
+    │   └── event: CreateLockupDynamicStreamV21
+    └── LockupLinear/
+        ├── event: CreateLockupLinearStreamV20
+        └── event: CreateLockupLinearStreamV21
+```
+
+## Different Results (\*)
+
+Due to the cross-chain indexing aspect, entities in Envio will need to have a chainId suffix attached to them to ensure
+they're unique across the board. At the same time, there are some minor features missing, which will cause some
+differences listed below.
+
+- For Envio indexers, some entities will have different identifiers (from what The Graph's subgraph have):
+  1.  `protocol-envio`: the `Action`, `Asset`, `Batch`, `Batcher`, `Contract` have a `-chainId` appended to the ID
+  2.  `merkle-envio`: the `Action`, `Asset` and `Factory` have a `-chainId` appended to the ID
+
+:::tip
+
+To avoid writing separate systems when assigning variables to queries, you can use slightly different filters. For
+example, given the different `id`s of an `Asset` (`address` in The Graph and `address-chainId` in Envio) you can query
+for certain assets with:
+
+1. `asset(id: $assetId)` for The Graph
+2. `Asset(where: {id: {_iregex: $assetId}})`
+
+With `assetId` in both cases being assigned to the Asset's `address`.
+
+:::

docs/api/indexers/05-development.mdx

@@ -0,0 +1,62 @@
+---
+id: "development"
+sidebar_position: 5
+title: "Development"
+---
+
+import LinkPreview from "@site/src/components/LinkPreview";
+
+## Integrating
+
+To integrate the Sablier V2 indexers into your own product, start from the official `@sablier/v2-subgraphs` repository.
+We recommend using:
+
+- [GraphQL Code Generator](https://the-guild.dev/graphql/codegen/docs/getting-started) to create types from your GraphQL
+  fragments and queries
+- [TanStack Query](https://tanstack.com/query) to fetch results from the exposed endpoints.
+
+If you're integrated the Sablier V2 subgraphs, we recommend using request and response wrappers/middleware to format
+results coming from Envio's indexer in the same shape as those from The Graph. Suggestions:
+
+1. Write queries in both systems (based on the
+   [querying language specifics](/api/indexers/differences#querying-language)) using the same `operationName` and swap
+   query strings between vendors based on it
+2. Write response middleware that converts results into similar shapes (see these
+   [important notes](api/indexers/differences#important-notes))
+
+<LinkPreview
+  href="https://github.com/sablier-labs/v2-subgraphs"
+  icon="github"
+  subtitle="GitHub"
+  title="Github - sablier-labs/v2-subgraphs: Sablier V2 Indexers"
+/>
+
+## Contributing
+
+To contribute to the V2 indexers (Envio x Sablier) or deploy your own, head over to the GitHub
+[`v2-subgraphs`](https://github.com/sablier-labs/v2-subgraphs) repository. Get started by reviewing the
+`apps/protocol-envio` directory.
+
+To bootstrap an indexer project, run the following commands. They will install the requisite dependencies and generate
+the code for a multi-chain deployment.
+
+```bash
+cd ./packages/core
+pnpm install
+pnpm run setup
+
+# OR
+
+pnpm dev
+```
+
+You'll find a local Hasura process at `http://localhost:8080` which also includes a `GraphQL` endpoint to query locally.
+To deploy without hosting yourself, have a look at the Envio
+[Hosted Service](https://docs.envio.dev/docs/hosted-service) docs.
+
+:::info
+
+Prior to starting the `Envio` process (`pnpm dev`) please make sure your Docker app is running. For installation see the
+official docs at https://docs.envio.dev/.
+
+:::

docs/api/indexers/_category_.json

@@ -0,0 +1,5 @@
+{
+  "collapsed": false,
+  "label": "Indexers (Envio)",
+  "position": 2
+}

docs/api/indexers/merkle/01-entities.mdx

@@ -0,0 +1,22 @@
+---
+id: "entities"
+sidebar_position: 1
+title: "Entities (+)"
+---
+
+:::info
+
+For an introduction into the base Sablier primitives and some architectural choices, make sure to check out the
+[protocol subgraph](/api/subgraphs/protocol/entities) before reading about the merkle indexer.
+
+The same (or similar) entities are used throughout both The Graph and Envio setups, as well as protocol and merkle
+deployments.
+
+:::
+
+## Entity Architecture
+
+Entities mirror those used inside the `merkle` subgraph. Read more in the
+[dedicated section](/api/subgraphs/merkle/entities).
+
+These are also defined in the `schema.graphql` file of each indexer.