feat: add some basic open telemetry setup and metrics

[ltucker]

• sets up open telemetry if requested according to environment config
• records a few basic measurements of rpc calls, ingestions and startups

etc

This pull request introduces significant enhancements to telemetry and metrics collection in the diode-server project. The changes include adding telemetry setup and metrics recording to the ingester and reconciler components, updating the Docker configuration to support telemetry, and updating dependencies to include OpenTelemetry packages.

Telemetry and Metrics Enhancements:

• Telemetry Setup and Metrics Recording:

  • diode-server/cmd/ingester/main.go: Added telemetry setup, startup counter metric, and Prometheus metrics server initialization. [1] [2]
  • diode-server/cmd/reconciler/main.go: Added telemetry setup, startup counter metric, ingestion processor metrics, and Prometheus metrics server initialization. [1] [2] [3]
  • diode-server/ingester/component.go: Integrated metrics recording in the Ingest method and added telemetry attributes. [1] [2] [3] [4] [5] [6] [7]

• Docker Configuration Updates:

  • diode-server/docker/docker-compose.yaml: Added telemetry environment variables and metrics endpoints for Prometheus. [1] [2] [3] [4]
  • diode-server/docker/sample.env: Added sample telemetry environment variables.

Dependency Updates:

• Go Module Dependencies:
  • diode-server/go.mod: Updated dependencies to include OpenTelemetry and Prometheus packages. [1] [2]

Testing Enhancements:

• Test Updates:
  • diode-server/ingester/component_test.go: Updated tests to include telemetry configuration and metrics setup. [1] [2] [3]

These changes collectively improve the observability and monitoring capabilities of the diode-server components, ensuring better insights into their operation and performance.

[github-actions]

Go test coverage

STATUS	ELAPSED	PACKAGE	COVER	PASS	FAIL	SKIP
🟢 PASS	0.01s	github.com/netboxlabs/diode/diode-server/dbstore/postgres	0.0%	0	0	0
🟢 PASS	1.24s	github.com/netboxlabs/diode/diode-server/ingester	75.0%	9	0	0
🟢 PASS	1.05s	github.com/netboxlabs/diode/diode-server/migrator	70.4%	4	0	0
🟢 PASS	1.08s	github.com/netboxlabs/diode/diode-server/netbox	0.4%	4	0	0
🟢 PASS	1.64s	github.com/netboxlabs/diode/diode-server/netboxdiodeplugin	78.9%	42	0	0
🟢 PASS	1.42s	github.com/netboxlabs/diode/diode-server/reconciler	69.7%	43	0	0
🟢 PASS	1.07s	github.com/netboxlabs/diode/diode-server/reconciler/applier	81.8%	1	0	0
🟢 PASS	0.00s	github.com/netboxlabs/diode/diode-server/reconciler/changeset	0.0%	0	0	0
🟢 PASS	1.29s	github.com/netboxlabs/diode/diode-server/reconciler/differ	74.2%	92	0	0
🟢 PASS	1.02s	github.com/netboxlabs/diode/diode-server/server	85.7%	14	0	0
🟢 PASS	0.00s	github.com/netboxlabs/diode/diode-server/telemetry	0.0%	0	0	0
🟢 PASS	1.01s	github.com/netboxlabs/diode/diode-server/version	100.0%	1	0	0

Total coverage: 17.5%

[leoparente]

Nice. LGTM