feat(ponder-ens): introduce plugin architecture #3

tk-o · 2024-12-31T06:14:44Z

This change allows defining simple ponder plugins per supported subnames. Only one plugin can be active during indexing, and to select it, please set the ACTIVE_PLUGIN env var to the name of one of the available plugins (see src/plugins directory). For example:

ACTIVE_PLUGIN=eth
ACTIVE_PLUGIN=base.eth

Also, please make sure to know how to use the DATABASE_SCHEMA env var.

Examples

Running indexer with the requested subname plugin for ponder:

ACTIVE_PLUGIN=base.eth DATABASE_SCHEMA=my_base_eth pnpm start

This change allows defining simple ponder plugins per supported ENS blockchain which can be loaded conditionally in the runtime. Requesting a chain to be indexed requires passing a chain ID in the `CHAINS_TO_INDEX` env var and comma-separated chain ID values.

lightwalker-eth

@tk-o Hey thanks for all the efforts on this 🙌 Excited to see the "plugins" start to come to life! Will defer to @shrugs on more detailed review for the proposed plugin architecture.

A more general feedback from my side. For each PR we make, let's please also consider what documentation updates might be relevant and include them in the PR. Appreciate this takes time, but good documentation for other devs in the ENS community will be key to our success.

For example, ideas that would be great to include as documentation in the README.md with this PR might include:

Overall introduction and framing for what we're working to do here. For example: enable any permutation of chain-specific ENS indexers to be dynamically activated via environment variables.
Communicate known oversimplifications in our current approach and identify that our roadmap will remove these oversimplifications in time. For example:
1. At the moment we're making oversimplifications where indexing "mainnet" represents indexing the legacy ENS Subgraph, indexing "base" represents only indexing subnames of "base.eth" on Base, and indexing "linea" represents only indexing subnames of "linea.eth" on Linea. Etc.
2. At the moment we're making oversimplifications where we assume that all the delegation relationships of subname issuance from an L1 resolver (using ENSIP-10 and EIP-3668) to an offchain gateway server that reads state from a L2 subname registry is immutable. In other words, we're currently assuming that the current L2 subname registry for subnames of "base.eth" on Base is guaranteed to continue serving in that role.
Perhaps an .env.example file that includes comments documenting the environment variables we use and different example values that might be set. Am I correct to assume that there's also some ponder-related environment variables that we should be documenting in an .env.example file too? For example: RPC endpoints for different chains? Let's write our documentation under the assumption that the reader has never used Ponder before and should not need to familiarize themselves with Ponder if they just want to get started and run their own instance.
A "getting started" section that explains step-by-step how to go from cloning the repo to having a running instance of our indexer.
Any other ideas you suggest will be useful 👍 😄

src/ponder-ens-plugins/ethereum/ponder.config.ts

lightwalker-eth

@tk-o Hey nice updates! I like the direction you're moving in 🚀 Reviewed and shared feedback 👍

src/ponder-ens-plugins/main.ts

src/ponder-ens-plugins/base/ponder.config.ts

src/ponder-ens-plugins/chain.ts

src/ponder-ens-plugins/base/README.md

src/lib/ens-helpers.ts

shrugs

an abi diff between mainnet and base would be helpful for identifying any logic discrepancies

please configure your editor to use biome to format files — the columnWidth being changed from 80 to 100 is causing a lot of formatting-related changes

ponder.config.ts

src/lib/ens-helpers.ts

src/ponder-ens-plugins/base/handlers/Resolver.ts

src/ponder-ens-plugins/base/ponder.config.ts

…handler lib

Refactor/plugin logic

This event is emitted by the `BaseRegistrar` during a registration from Bases RegistrarControllers

src/ponder-ens-plugins/eth/ponder.config.ts

This one includes a fix for duplicated log entries error

Make the root domain name based of the env var

renames NAMEHASH to NODE

lightwalker-eth

@tk-o Hey great to see the updates here. 👍 Reviewed and shared feedback 😄

src/handlers/Registry.ts

src/handlers/Resolver.ts

src/lib/plugins.ts

src/ponder-ens-plugins/eth/ponder.config.ts

src/ponder-ens-plugins/eth.base/ponder.config.ts

lightwalker-eth · 2025-01-06T12:39:13Z

src/ponder-ens-plugins/eth.base/ponder.config.ts

+export const ns = createNs(nestedNamespace);
+export type NsType<T extends string> = NsReturnType<T, typeof nestedNamespace>;
+
+const START_BLOCK = undefined; // 17607350;


Can we please document the idea here? Why setting these to undefined?

shrugs · 2025-01-06T19:32:45Z

i much prefer the root argument passed to makeRegistryHandlers() because it makes clear that the registry handlers are universal except for root that nodes in those handlers are relative to. it also means that in the future v2 this shared code should be relatively identical. using the constant is less clear and requires more mental overhead for the reader

The generic handlers have now the inlined args types to avoid importing subname-specifc event types. Also, we now use the `INDEX_SUBNAME` env var to specify the subname to index, i.e. `eth` or `base.eth` (note: no starting dot character).

shrugs

excellent documentation & typescript type work! i only have nitpicks in here, i think, so it's looking very close to merge

shrugs · 2025-01-07T19:40:53Z

src/handlers/Registrar.ts

+/**
+ * A factory function that returns Ponder indexing handlers for a specified index name/subname.
+ */
+export const makeRegistryHandlers = (indexedSubname: `${string}eth`) => {


now that i understand the nomenclature better, let's call this indexedSubname variable managedSubname (and managedSubnameNode) so that people know that these Registrar handlers are in the context of a registrar that manages the provided subname

shrugs · 2025-01-07T19:41:14Z

src/handlers/Registrar.ts

    }

    await context.db.update(registrations, { id: label }).set({ labelName: name, cost });
  }

  return {
-    get baseNameNode() {
-      return baseNameNode;
+    get indexedSubnameNode() {


why provide this back, it will be known to the caller because they provided it as an input agument

We need this value in the caller for some inline indexing handlers:
https://github.com/namehash/ens-multichain-indexer/blob/aad651cf532f1d89b214695994bb2d8e2574b7e3/src/ponder-ens-plugins/eth.base/handlers/Registrar.ts#L8-L17
https://github.com/namehash/ens-multichain-indexer/blob/aad651cf532f1d89b214695994bb2d8e2574b7e3/src/ponder-ens-plugins/eth.base/handlers/Registrar.ts#L31

ah, hmm. i don't love this but it's ok for the moment 👍 we'll refactor this in v2

src/handlers/Registrar.ts

shrugs · 2025-01-07T19:43:30Z

src/handlers/Registrar.ts

@@ -114,12 +119,16 @@ export const makeRegistryHandlers = (baseName: `.${`${string}`}eth`) => {
      args: { tokenId, from, to },
    }: {
      context: Context;
-      args: Event<NsType<"BaseRegistrar:Transfer">>["args"];
+      args: {


and i think for each of these functions in Registrar.ts we should standardize the api to be like a ponder handler: it always accepts context and event, and args is within the event. we need not do this in this pr, i can manage it once we've merged if you'd like

shrugs · 2025-01-07T19:45:43Z

src/handlers/Registry.ts

+/**
+ * Initialize the ENS root node with the zeroAddress as the owner.
+ */
+export async function handleRootNodeCreation({ context }: { context: Context }) {


agreed with lightwalker's naming concern, and apologies for the nitpick, but how about calling this setupRootNode

src/handlers/Resolver.ts

shrugs · 2025-01-07T19:48:22Z

src/lib/ponder-plugin-utils.ts

+
+type PonderNsPath<T extends PonderNsPath = "/"> = `` | `/${string}` | `/${string}${T}`;
+
+type EthSubname = `${string}eth`;


very nice type work in this file 🫡

@tk-o Yeah nice work here! 💪

When referring to a variable holding any of: `eth`, `base.eth`, `linea.eth`

shrugs · 2025-01-07T22:28:26Z

looks like biome got misconfigured or something — some line width formatting related changes in the recent commits

Allows to automatically catch code-related issues on the CI

lightwalker-eth

@tk-o Thanks for your updates here! Very happy to see docs improving. Let's continue to push hard on making our docs and use of language properly aligned.

Reviewed and shared feedback 👍

lightwalker-eth · 2025-01-08T11:35:53Z

.env.local.example

+PONDER_RPC_URL_8453=https://base-rpc.publicnode.com
+PONDER_RPC_URL_59144=https://linea-rpc.publicnode.com
+
+# ponder indexer ens deployment configuration


@tk-o This is still an open feedback

lightwalker-eth · 2025-01-08T11:37:21Z

.env.local.example

+# ponder indexer database configuration
+
+## one schema name per chain ID, i.e. ponder_ens_${INDEX_BASENAME}
+DATABASE_SCHEMA=ponder_ens_base.eth


@tk-o This remains an open feedback item.

I'm also concerned to read "chain id" here. I'm very worried we're getting key ideas confused. In my understanding, "chain id" should not be THE key consideration in our configurations. The INDEX_SUBNAME should be THE number 1 consideration, and then everything else should flow downstream from that.

For example, the chain to index is determined by the subnames to be indexed. Not the other way around.

lightwalker-eth · 2025-01-08T11:42:43Z

src/ponder-ens-plugins/eth.base/ponder.config.ts

 import { BaseRegistrar } from "./abis/BaseRegistrar";
 import { EarlyAccessRegistrarController } from "./abis/EARegistrarController";
 import { L2Resolver } from "./abis/L2Resolver";
 import { RegistrarController } from "./abis/RegistrarController";
 import { Registry } from "./abis/Registry";

-export const baseName = ".base.eth" as const;
+export const managedSubname = "base.eth" as const;


Could you help me understand the use of the word "managed" here? Is there a special reason to use that and not "indexed"?

What does it mean to be "managed"?

@shrugs could you share your view here?

@lightwalker-eth it got lost in the github comments but: the registrar handlers are in the context of the subname that the registrar 'manages' — the ETH Registrar manages .eth, base's l2 registrar deployment manages .base.eth subnames, etc. perhaps there's a better word but i don't like 'indexed' in this case, since the argument is specifying which parent node all of the events in this registrar are relative to

lightwalker-eth · 2025-01-08T11:46:07Z

src/ponder-ens-plugins/eth.base/ponder.config.ts

-
-const START_BLOCK = undefined; // 17607350;
-const END_BLOCK = undefined; // 17607351;
+export const ponderNamespace = createPonderNamespace(managedSubname);


Is there a special reason for us to put lots of emphasis on "ponder" when we name things?

It's true that we're building on the Ponder framework, but I feel attracted to more general naming of these ideas. For example:

"ponderNamespace" -> "pluginNamespace"

"createPonderNamespace" -> "createPluginNamespace"

Very open to other suggestions. Perhaps I'm missing something?

I might suggest contractNamespace

lightwalker-eth · 2025-01-08T11:47:50Z

src/ponder-ens-plugins/eth/ponder.config.ts

@@ -12,11 +12,9 @@ import { Resolver } from "./abis/Resolver";

 const RESOLVER_ABI = mergeAbis([LegacyPublicResolver, Resolver]);

-export const baseName = ".eth" as const;
+export const managedSubname = "eth";


Please see my other comment about the use of the word "managed" here.

I've reverted back to indexedSubname

lightwalker-eth · 2025-01-08T12:11:17Z

src/lib/ponder-plugin-utils.ts

+export function createPonderNamespace<Subname extends EthSubname>(subname: Subname) {
+  const path = transformDomain(subname) satisfies PonderNsPath;
+
+  /** Creates a name-spaced contract name */


Suggested change

/** Creates a name-spaced contract name */

/** Creates a namespaced contract name */

namespace / namespaced is one word, no hyphen.

Please search for other instances of "name-spaced" and replace them. Thanks.

lightwalker-eth · 2025-01-08T12:13:32Z

src/lib/ponder-plugin-utils.ts

+ *
+ * However, in some cases, we may want to create a name-spaced contract name to distinguish between contracts having the same name, but handling different implementations.
+ *
+ * Let's say we have two contracts named `Registry`. One handles the `eth` name, and the other handles the `base.eth` subname. We need to create a name-spaced contract name to avoid conflicts.


Suggested change

* Let's say we have two contracts named `Registry`. One handles the `eth` name, and the other handles the `base.eth` subname. We need to create a name-spaced contract name to avoid conflicts.

* Let's say we have two contracts named `Registry`. One handles `eth` subnames, and the other handles `base.eth` subnames. We need to create a namespaced contract name to avoid conflicts.

lightwalker-eth · 2025-01-08T12:17:05Z

src/lib/ponder-plugin-utils.ts

@@ -0,0 +1,64 @@
+/**


@tk-o Great job on the docs here! 🚀 🚀 This makes a big impact for our goals! Docs like this do a lot to help encourage other devs to adopt our indexer because they can more easily make sense of what we're doing and why 😄

One small suggestion is to break to a new line around approximately 80ish characters. This helps with line wrapping when viewing code on platforms such as GitHub.

lightwalker-eth · 2025-01-08T12:18:27Z

src/ponder-ens-plugins/eth.base/handlers/Registrar.ts

What do you think about renaming the ponder-ens-plugins directory to just plugins? Seems more clean 😄

Assuming we do rename, please be sure to do it in the way that it's properly tracked by git as just renames, and not a bunch of file deletes / creations.

lightwalker-eth · 2025-01-08T12:20:31Z

.github/workflows/static-analysis.yml

@@ -0,0 +1,31 @@
+name: Static Analysis


Great to see this!

Parser is not required, as we do the validation in the ponder.config.ts and throw an error if the provided value was not expected.

This allows to define a name managed by the given plugin.

lightwalker-eth

@tk-o Hey super awesome updates! 🚀 🚀 🚀

lightwalker-eth reviewed Dec 31, 2024

View reviewed changes

tk-o commented Jan 2, 2025

View reviewed changes

src/ponder-ens-plugins/ethereum/ponder.config.ts Outdated Show resolved Hide resolved

tk-o added 3 commits January 3, 2025 14:12

feat(ponder-ens-plugin): update abis and contract configs

a5bfb00

fix(ens-base-plugin): enalbe domain insert setup event

2578b0f

feat(ponder-ens-plugin): include early adopter registrar contract

fae9508

lightwalker-eth requested changes Jan 3, 2025

View reviewed changes

feat(ponder-ens-plugin): focus plugins around domain names

e1dad39

shrugs reviewed Jan 3, 2025

View reviewed changes

shrugs and others added 10 commits January 3, 2025 12:51

checkpoint: pull shared logic for registry and resolvers into shared …

1931ec7

…handler lib

feat: refactor plugin logic, DRY handler code

3d181f9

chore: inline some addreses

2d079a7

refactor: DRY up registrar code

32912d5

Merge pull request #4 from namehash/refactor/plugin-logic

7f84cff

Refactor/plugin logic

feat(ponder-ens-plugin): use domain name path as a namespace

a88933e

chore(ponder-ens-plugin): use biomejs formatter

7e5af0e

fix: tidy registrar handlers

cd152a6

fix: handle preminted domain in base node

dc0bfdd

fix(base.eth): handle NameRegisteredWithRecord event

77c1ec2

This event is emitted by the `BaseRegistrar` during a registration from Bases RegistrarControllers

shrugs reviewed Jan 4, 2025

View reviewed changes

src/ponder-ens-plugins/eth/ponder.config.ts Outdated Show resolved Hide resolved

tk-o added 2 commits January 5, 2025 19:53

fix(deps): update ponder version

ff0b2af

This one includes a fix for duplicated log entries error

fix: use selected root domain name

b3c17e5

Make the root domain name based of the env var

tk-o force-pushed the feat/ponder-ens-plugins branch from 8c25e59 to 75cd66b Compare January 6, 2025 11:04

refactor: rename consts to apply feedback

0e35dd8

renames NAMEHASH to NODE

tk-o force-pushed the feat/ponder-ens-plugins branch from 75cd66b to 0e35dd8 Compare January 6, 2025 11:08

docs(base.eth): update readme

e5d7541

lightwalker-eth requested changes Jan 6, 2025

View reviewed changes

refactor(ponder-plugins): simplify architecture

067f570

The generic handlers have now the inlined args types to avoid importing subname-specifc event types. Also, we now use the `INDEX_SUBNAME` env var to specify the subname to index, i.e. `eth` or `base.eth` (note: no starting dot character).

shrugs reviewed Jan 7, 2025

View reviewed changes

tk-o added 2 commits January 7, 2025 22:56

refactor(ponder-plugins): use managed subname term

e3b62c0

When referring to a variable holding any of: `eth`, `base.eth`, `linea.eth`

refactor(ponder-plugins): make generic handlers to use ponder types

aad651c

tk-o added 2 commits January 8, 2025 07:45

chore(code-style): apply biome format --fix

00c9935

fix(ci): run static analysis

a39200f

Allows to automatically catch code-related issues on the CI

lightwalker-eth requested changes Jan 8, 2025

View reviewed changes

tk-o added 7 commits January 8, 2025 15:58

refactor(plugins): rename plugins directory

9eb9eef

refactor(plugins): rename plugin helpers

728ec62

refactor(plugins): rename plugin helpers file

7c4bc1e

refactor(plugins): rename ponder plugins to just plugins

9891ed0

refactor(plugins): drop the ACTIVE_PLUGIN env var parser

7b81161

Parser is not required, as we do the validation in the ponder.config.ts and throw an error if the provided value was not expected.

refactor(plugins): use ownedSubname name

1dbe2a9

This allows to define a name managed by the given plugin.

refactor(plugins): rename ownedSubname to ownedName

cc6f2c3

lightwalker-eth approved these changes Jan 8, 2025

View reviewed changes

tk-o merged commit 7bdc380 into main Jan 8, 2025
1 check passed

shrugs deleted the feat/ponder-ens-plugins branch January 9, 2025 18:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(ponder-ens): introduce plugin architecture #3

feat(ponder-ens): introduce plugin architecture #3

tk-o commented Dec 31, 2024 •

edited

Loading

lightwalker-eth left a comment

lightwalker-eth left a comment

shrugs left a comment

lightwalker-eth left a comment

lightwalker-eth Jan 6, 2025

shrugs commented Jan 6, 2025

shrugs left a comment

shrugs Jan 7, 2025

shrugs Jan 7, 2025

tk-o Jan 7, 2025

shrugs Jan 8, 2025

shrugs Jan 7, 2025

shrugs Jan 7, 2025

shrugs Jan 7, 2025

lightwalker-eth Jan 8, 2025

shrugs commented Jan 7, 2025

lightwalker-eth left a comment

lightwalker-eth Jan 8, 2025

lightwalker-eth Jan 8, 2025

lightwalker-eth Jan 8, 2025

tk-o Jan 8, 2025

shrugs Jan 8, 2025

lightwalker-eth Jan 8, 2025

shrugs Jan 8, 2025

lightwalker-eth Jan 8, 2025

tk-o Jan 8, 2025

tk-o Jan 8, 2025

lightwalker-eth Jan 8, 2025

lightwalker-eth Jan 8, 2025

lightwalker-eth Jan 8, 2025

lightwalker-eth Jan 8, 2025

lightwalker-eth Jan 8, 2025

lightwalker-eth left a comment


		type PonderNsPath<T extends PonderNsPath = "/"> = `` \| `/${string}` \| `/${string}${T}`;

		type EthSubname = `${string}eth`;

	/** Creates a name-spaced contract name */
	/** Creates a namespaced contract name */

	* Let's say we have two contracts named `Registry`. One handles the `eth` name, and the other handles the `base.eth` subname. We need to create a name-spaced contract name to avoid conflicts.
	* Let's say we have two contracts named `Registry`. One handles `eth` subnames, and the other handles `base.eth` subnames. We need to create a namespaced contract name to avoid conflicts.

feat(ponder-ens): introduce plugin architecture #3

feat(ponder-ens): introduce plugin architecture #3

Conversation

tk-o commented Dec 31, 2024 • edited Loading

Examples

lightwalker-eth left a comment

Choose a reason for hiding this comment

lightwalker-eth left a comment

Choose a reason for hiding this comment

shrugs left a comment

Choose a reason for hiding this comment

lightwalker-eth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shrugs commented Jan 6, 2025

shrugs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shrugs commented Jan 7, 2025

lightwalker-eth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lightwalker-eth left a comment

Choose a reason for hiding this comment

tk-o commented Dec 31, 2024 •

edited

Loading