Create Go supervisor

[torives]

📚 Context

Running a node right now requires the orchestration of several Docker containers. For testing, running locally, and/or deploying in a VPS a simpler option is desirable.

Original issue/discussion: #26

✔️ Solution

Create a single binary to execute all of our services. All service configuration should be done via environment variables, as the binary will have just a single parameter defining its execution mode: reader, validator, full (reader+validator), or host.

It will forward each service's stdout and stderr to its own stdout and stderr, similar to what is currently done when running the node using the docker-compose.yml file.

Although server-manager is not a service built in this repository, it will also be executed by the binary: it will assume it is installed in the system and available in the PATH.

It will also assume the Redis, PostgreSQL, and Hardhat services are already running and ready to connect by the time it starts.

📈 Subtasks

• Create a single node image #65
• Create the base Go project #91
• Run services using goroutines #96
• Setup the CI to build and test Go projects #108
• Configure logs #116
• Simplify Node config #88
• Add indexer service #125
• Setup environment for testing #97
• Add remaining services #139
• Run server-manager as a service #140
• Change DAPP_DEPLOY_BLOCK_HASH to DAPP_DEPLOY_BLOCK_NUMBER #180
• Map services variables #179
• Add description tag to Config.toml #177
• Setup health check endpoint #193
• Create SupervisorService #221
• Setup HTTP proxy to internal endpoints #241
  • /graphql, /inspect, /rollups (host-mode), /metrics (dispatcher)
  • See https://github.com/elazarl/goproxy
  • We should expose only the proxy port once we have the proxy. Addresses other than the port should be 127.0.0.1.
  • The proxy address should be 127.0.0.1 by default, and we should have an env var to configure it. This way, the node doesn't bind to a public address, which would be an unsafe default.
• Make All HTTP ports configurable #198
• Implement graceful shutdown #222
  • See: https://github.com/gligneul/eggroll/blob/main/internal/dev/signal.go#L14 and https://github.com/gligneul/eggroll/blob/main/internal/dev/dev.go#L27
• Add a feature flag to use custom Redis instead of starting it #200

[gligneul]

We could rename host mode to backendless mode, following the idea of sunodo.

[endersonmaia]

Create a single binary to execute all of our services....

It will forward each service's stdout and stderr to its own stdout and stderr ...

Are you planning to create another process/service supervisor? Why not reuse some existing software that already does this?

For sunodo/rollups-node, we're using s6-overlay,, but there are a lot of other options if you think s6 is not a fit.

The choice for s6 was 'cause it's made for running inside containers and it's very light.

[endersonmaia]

Since you're considering migrating some code to golang, maybe this supervisord golang implementation is an option.

[gligneul]

We want to do something much simpler than a supervisor. And, as we rewrite code in Go, we will replace the external binaries with Go packages. So, in the end, there won't be any "supervisor" code left.

[gligneul]

Suggestion for the supervisor code using goroutines: https://gist.github.com/gligneul/dafa46f3e778888e108da13aa2d89d17