From ce7f9360b1c8a7faf2c66b7315020c32df072636 Mon Sep 17 00:00:00 2001 From: Robert Lin Date: Wed, 27 Mar 2024 18:09:08 +0900 Subject: [PATCH] managed-services: regenerate with msp-testbed, telemetry-gateway README --- .../engineering/managed-services/cloud-ops.md | 8 +++- .../managed-services/cloud-relay.md | 13 ++++++- .../managed-services/cody-analytics.md | 13 ++++++- .../engineering/managed-services/entitler.md | 8 +++- .../managed-services/gatekeeper.md | 13 ++++++- .../engineering/managed-services/index.md | 4 +- .../managed-services/msp-testbed.md | 15 +++++++- .../engineering/managed-services/pings.md | 8 +++- .../managed-services/releaseregistry.md | 13 ++++++- .../engineering/managed-services/sams.md | 8 +++- .../managed-services/sourcegraph-accounts.md | 8 +++- .../managed-services/support-integration.md | 13 ++++++- .../managed-services/telemetry-gateway.md | 38 ++++++++++++++++++- 13 files changed, 148 insertions(+), 14 deletions(-) diff --git a/content/departments/engineering/managed-services/cloud-ops.md b/content/departments/engineering/managed-services/cloud-ops.md index 222b496d6259..fcfd4972fa2c 100644 --- a/content/departments/engineering/managed-services/cloud-ops.md +++ b/content/departments/engineering/managed-services/cloud-ops.md @@ -3,7 +3,7 @@ @@ -17,9 +17,15 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | -------------------------------------------------------------------------------------------------------------------------- | | Service ID | [`cloud-ops`](https://github.com/sourcegraph/managed-services/blob/main/services/cloud-ops/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|----------------------------------------------------------------------------------------------------------------------------| +| Service ID | `cloud-ops` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/cloud-ops/service.yaml)) | +>>>>>>> Stashed changes | Owners | **cloud** | | Service kind | Cloud Run service | | Environments | [prod](#prod) | diff --git a/content/departments/engineering/managed-services/cloud-relay.md b/content/departments/engineering/managed-services/cloud-relay.md index a42254390032..f47fe87b3dbd 100644 --- a/content/departments/engineering/managed-services/cloud-relay.md +++ b/content/departments/engineering/managed-services/cloud-relay.md @@ -3,7 +3,7 @@ @@ -17,6 +17,7 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | -------------------------------------------------------------------------------------------------------------------- | | Service ID | [`cloud-relay`](https://github.com/sourcegraph/managed-services/blob/main/services/cloud-relay/service.yaml) | @@ -25,6 +26,16 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] | Environments | [prod](#prod) | | Docker image | `us-central1-docker.pkg.dev/control-plane-5e9ee072/docker/cloud-relay` | | Source code | [`https://github.com/sourcegraph/cloud-relay` - `.`](https://https://github.com/sourcegraph/cloud-relay/tree/HEAD/.) | +======= +| PROPERTY | DETAILS | +|--------------|------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `cloud-relay` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/cloud-relay/service.yaml)) | +| Owners | **cloud** | +| Service kind | Cloud Run service | +| Environments | [prod](#prod) | +| Docker image | `us-central1-docker.pkg.dev/control-plane-5e9ee072/docker/cloud-relay` | +| Source code | [`https://github.com/sourcegraph/cloud-relay` - `.`](https://https://github.com/sourcegraph/cloud-relay/tree/HEAD/.) | +>>>>>>> Stashed changes ## Environments diff --git a/content/departments/engineering/managed-services/cody-analytics.md b/content/departments/engineering/managed-services/cody-analytics.md index f0c6a78c435f..330f00a3fe50 100644 --- a/content/departments/engineering/managed-services/cody-analytics.md +++ b/content/departments/engineering/managed-services/cody-analytics.md @@ -3,7 +3,7 @@ @@ -17,6 +17,7 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | ------------------------------------------------------------------------------------------------------------------ | | Service ID | [`cody-analytics`](https://github.com/sourcegraph/managed-services/blob/main/services/cody-analytics/service.yaml) | @@ -25,6 +26,16 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] | Environments | [dev](#dev), [prod](#prod) | | Docker image | `us-central1-docker.pkg.dev/sourcegraph-dev/cody-analytics/service` | | Source code | [`github.com/sourcegraph/cody-analytics` - `.`](https://github.com/sourcegraph/cody-analytics/tree/HEAD/.) | +======= +| PROPERTY | DETAILS | +|--------------|------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `cody-analytics` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/cody-analytics/service.yaml)) | +| Owners | **cody-strat** | +| Service kind | Cloud Run service | +| Environments | [dev](#dev), [prod](#prod) | +| Docker image | `us-central1-docker.pkg.dev/sourcegraph-dev/cody-analytics/service` | +| Source code | [`github.com/sourcegraph/cody-analytics` - `.`](https://github.com/sourcegraph/cody-analytics/tree/HEAD/.) | +>>>>>>> Stashed changes ## Environments diff --git a/content/departments/engineering/managed-services/entitler.md b/content/departments/engineering/managed-services/entitler.md index 1e87431abee6..9f25d092a516 100644 --- a/content/departments/engineering/managed-services/entitler.md +++ b/content/departments/engineering/managed-services/entitler.md @@ -3,7 +3,7 @@ @@ -17,9 +17,15 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | --------------------------------------------------------------------------------------------------------------------------------------- | | Service ID | [`entitler`](https://github.com/sourcegraph/managed-services/blob/main/services/entitler/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|-----------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `entitler` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/entitler/service.yaml)) | +>>>>>>> Stashed changes | Owners | **security** | | Service kind | Cloud Run service | | Environments | [prod](#prod) | diff --git a/content/departments/engineering/managed-services/gatekeeper.md b/content/departments/engineering/managed-services/gatekeeper.md index 540f9953d0b2..8d9696282a04 100644 --- a/content/departments/engineering/managed-services/gatekeeper.md +++ b/content/departments/engineering/managed-services/gatekeeper.md @@ -3,7 +3,7 @@ @@ -17,6 +17,7 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | ---------------------------------------------------------------------------------------------------------- | | Service ID | [`gatekeeper`](https://github.com/sourcegraph/managed-services/blob/main/services/gatekeeper/service.yaml) | @@ -25,6 +26,16 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] | Environments | [prod](#prod) | | Docker image | `us.gcr.io/sourcegraph-dev/abuse-ban-bot` | | Source code | [`github.com/sourcegraph/abuse-ban-bot` - `.`](https://github.com/sourcegraph/abuse-ban-bot/tree/HEAD/.) | +======= +| PROPERTY | DETAILS | +|--------------|----------------------------------------------------------------------------------------------------------------------------| +| Service ID | `gatekeeper` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/gatekeeper/service.yaml)) | +| Owners | **cody-services** | +| Service kind | Cloud Run job | +| Environments | [prod](#prod) | +| Docker image | `us.gcr.io/sourcegraph-dev/abuse-ban-bot` | +| Source code | [`github.com/sourcegraph/abuse-ban-bot` - `.`](https://github.com/sourcegraph/abuse-ban-bot/tree/HEAD/.) | +>>>>>>> Stashed changes ## Environments diff --git a/content/departments/engineering/managed-services/index.md b/content/departments/engineering/managed-services/index.md index f682f8e079bb..61cf315a9e12 100644 --- a/content/departments/engineering/managed-services/index.md +++ b/content/departments/engineering/managed-services/index.md @@ -3,11 +3,11 @@ -These pages contain generated operational guidance for the infrastructure of [Managed Services Platform (MSP)](../teams/core-services/managed-services/platform.md) services. +These pages contain generated operational guidance for the infrastructure of the 12 [Managed Services Platform (MSP)](../teams/core-services/managed-services/platform.md) services (across 18 environments) currently in operation at Sourcegraph. This includes information about each service, configured environments, Entitle requests, common tasks, monitoring, etc. In addition to service-specific guidance, [General guidance](#general-guidance) is also available. diff --git a/content/departments/engineering/managed-services/msp-testbed.md b/content/departments/engineering/managed-services/msp-testbed.md index dd02c4823edb..3f5286b1b225 100644 --- a/content/departments/engineering/managed-services/msp-testbed.md +++ b/content/departments/engineering/managed-services/msp-testbed.md @@ -3,7 +3,7 @@ @@ -17,15 +17,28 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | -------------------------------------------------------------------------------------------------------------------------------- | | Service ID | [`msp-testbed`](https://github.com/sourcegraph/managed-services/blob/main/services/msp-testbed/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|----------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `msp-testbed` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/msp-testbed/service.yaml)) | +>>>>>>> Stashed changes | Owners | **core-services** | | Service kind | Cloud Run service | | Environments | [test](#test), [robert](#robert) | | Docker image | `us.gcr.io/sourcegraph-dev/msp-example` | | Source code | [`github.com/sourcegraph/sourcegraph` - `cmd/msp-example`](https://github.com/sourcegraph/sourcegraph/tree/HEAD/cmd/msp-example) | + + +This is a test environment used by the Core Services team for experimenting with MSP infrastructure changes. +Each Core Services teammate generally focuses their experiments on an individual environment of this service. + ## Environments ### test diff --git a/content/departments/engineering/managed-services/pings.md b/content/departments/engineering/managed-services/pings.md index ead843f63c38..4e488c29d6a5 100644 --- a/content/departments/engineering/managed-services/pings.md +++ b/content/departments/engineering/managed-services/pings.md @@ -3,7 +3,7 @@ @@ -17,9 +17,15 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | -------------------------------------------------------------------------------------------------------------------- | | Service ID | [`pings`](https://github.com/sourcegraph/managed-services/blob/main/services/pings/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|----------------------------------------------------------------------------------------------------------------------| +| Service ID | `pings` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/pings/service.yaml)) | +>>>>>>> Stashed changes | Owners | **core-services** | | Service kind | Cloud Run service | | Environments | [prod](#prod) | diff --git a/content/departments/engineering/managed-services/releaseregistry.md b/content/departments/engineering/managed-services/releaseregistry.md index ad7b208abc40..8945b47b2fca 100644 --- a/content/departments/engineering/managed-services/releaseregistry.md +++ b/content/departments/engineering/managed-services/releaseregistry.md @@ -3,7 +3,7 @@ @@ -17,6 +17,7 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | -------------------------------------------------------------------------------------------------------------------- | | Service ID | [`releaseregistry`](https://github.com/sourcegraph/managed-services/blob/main/services/releaseregistry/service.yaml) | @@ -25,6 +26,16 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] | Environments | [prod](#prod), [dev](#dev) | | Docker image | `us.gcr.io/sourcegraph-dev/releaseregistry` | | Source code | [`github.com/sourcegraph/releaseregistry` - `.`](https://github.com/sourcegraph/releaseregistry/tree/HEAD/.) | +======= +| PROPERTY | DETAILS | +|--------------|--------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `releaseregistry` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/releaseregistry/service.yaml)) | +| Owners | **dev-experience** | +| Service kind | Cloud Run service | +| Environments | [prod](#prod), [dev](#dev) | +| Docker image | `us.gcr.io/sourcegraph-dev/releaseregistry` | +| Source code | [`github.com/sourcegraph/releaseregistry` - `.`](https://github.com/sourcegraph/releaseregistry/tree/HEAD/.) | +>>>>>>> Stashed changes ## Environments diff --git a/content/departments/engineering/managed-services/sams.md b/content/departments/engineering/managed-services/sams.md index 73ce6ac7ebdd..8561449d3197 100644 --- a/content/departments/engineering/managed-services/sams.md +++ b/content/departments/engineering/managed-services/sams.md @@ -3,7 +3,7 @@ @@ -17,9 +17,15 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | | Service ID | [`sams`](https://github.com/sourcegraph/managed-services/blob/main/services/sams/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `sams` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/sams/service.yaml)) | +>>>>>>> Stashed changes | Owners | **cody-plg** | | Service kind | Cloud Run service | | Environments | [dev](#dev), [prod](#prod) | diff --git a/content/departments/engineering/managed-services/sourcegraph-accounts.md b/content/departments/engineering/managed-services/sourcegraph-accounts.md index 6411536d6e38..3fe9ebafc9f0 100644 --- a/content/departments/engineering/managed-services/sourcegraph-accounts.md +++ b/content/departments/engineering/managed-services/sourcegraph-accounts.md @@ -3,7 +3,7 @@ @@ -17,9 +17,15 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | Service ID | [`sourcegraph-accounts`](https://github.com/sourcegraph/managed-services/blob/main/services/sourcegraph-accounts/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `sourcegraph-accounts` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/sourcegraph-accounts/service.yaml)) | +>>>>>>> Stashed changes | Owners | **core-services** | | Service kind | Cloud Run service | | Environments | [dev](#dev), [prod](#prod) | diff --git a/content/departments/engineering/managed-services/support-integration.md b/content/departments/engineering/managed-services/support-integration.md index 39e16d6c7bf4..53ab9d91513c 100644 --- a/content/departments/engineering/managed-services/support-integration.md +++ b/content/departments/engineering/managed-services/support-integration.md @@ -3,7 +3,7 @@ @@ -17,6 +17,7 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | ---------------------------------------------------------------------------------------------------------------------------- | | Service ID | [`support-integration`](https://github.com/sourcegraph/managed-services/blob/main/services/support-integration/service.yaml) | @@ -25,6 +26,16 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] | Environments | [prod](#prod) | | Docker image | `index.docker.io/sourcegraph/support-integration` | | Source code | [`github.com/sourcegraph/support-integration` - `.`](https://github.com/sourcegraph/support-integration/tree/HEAD/.) | +======= +| PROPERTY | DETAILS | +|--------------|----------------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `support-integration` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/support-integration/service.yaml)) | +| Owners | **Customer Support** | +| Service kind | Cloud Run service | +| Environments | [prod](#prod) | +| Docker image | `index.docker.io/sourcegraph/support-integration` | +| Source code | [`github.com/sourcegraph/support-integration` - `.`](https://github.com/sourcegraph/support-integration/tree/HEAD/.) | +>>>>>>> Stashed changes ## Environments diff --git a/content/departments/engineering/managed-services/telemetry-gateway.md b/content/departments/engineering/managed-services/telemetry-gateway.md index 386c40d2b7fd..876e9354b78a 100644 --- a/content/departments/engineering/managed-services/telemetry-gateway.md +++ b/content/departments/engineering/managed-services/telemetry-gateway.md @@ -3,7 +3,7 @@ @@ -17,15 +17,51 @@ If you need assistance with MSP infrastructure, reach out to the [Core Services] ## Service overview +<<<<<<< Updated upstream | PROPERTY | DETAILS | | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | | Service ID | [`telemetry-gateway`](https://github.com/sourcegraph/managed-services/blob/main/services/telemetry-gateway/service.yaml) | +======= +| PROPERTY | DETAILS | +|--------------|----------------------------------------------------------------------------------------------------------------------------------------------| +| Service ID | `telemetry-gateway` ([specification](https://github.com/sourcegraph/managed-services/blob/main/services/telemetry-gateway/service.yaml)) | +>>>>>>> Stashed changes | Owners | **core-services** | | Service kind | Cloud Run service | | Environments | [dev](#dev), [prod](#prod) | | Docker image | `index.docker.io/sourcegraph/telemetry-gateway` | | Source code | [`github.com/sourcegraph/sourcegraph` - `cmd/telemetry-gateway`](https://github.com/sourcegraph/sourcegraph/tree/HEAD/cmd/telemetry-gateway) | + + +The Telemetry Gateway service is the service that ingests [telemetry v2 events](https://sourcegraph.com/doc/dev/background-information/telemetry) from all Sourcegraph instances, as well as other managed services. + +- For Sourcegraph instances that prior to 5.2.0, no events are exported to Telemetry Gateway, though legacy mechanisms may exist, e.g. for Cloud instances. +- As of 5.2.0, [certain flags can be configured](https://docs.sourcegraph.com/dev/background-information/telemetry#enabling-telemetry-export) to export events that have been instrumented with the new APIs to Telemetry Gateway. +- As of 5.2.1, for existing licenses, export is enabled by default for Cody events only - for new licenses, export is enabled for all events. Some license tags can be configured to disable telemetry export in various degrees - see the original [Telemetry Export rollout plan](https://docs.google.com/document/d/1Z1Yp7G61WYlQ1B4vO5-mIXVtmvzGmD7PqYHNBQV-2Ik/edit). + +For discussion around telemetry V2 adoption, please reach out to #wg-v2-telemetry. +For discussion around the Telemetry Gateway service, please reach out to #discuss-core-services. +For more information, also see: + +- Service API: [`telemetrygateway.proto`](https://github.com/sourcegraph/sourcegraph/blob/main/internal/telemetrygateway/v1/telemetrygateway.proto) +- [Docs: Admin: Telemetry](https://sourcegraph.com/docs/admin/telemetry#telemetry) +- [Docs: Dev: Background Information: Telemetry](https://sourcegraph.com/docs/dev/background-information/telemetry) +- [Docs: Dev: How to set up Telemetry Gateway locally](https://sourcegraph.com/docs/dev/how-to/telemetry_gateway) + +### Querying events + +Please reach out to #discuss-analytics for assistance in querying the dataset - Telemetry Gateway only handles ingestion and forwarding data to pipelines operated by the Data Analytics team. + +### Debugging missing Sourcegraph instance events + +1. Check for a license tag on the instance's license that disables events - see the original [Telemetry Export rollout plan](https://docs.google.com/document/d/1Z1Yp7G61WYlQ1B4vO5-mIXVtmvzGmD7PqYHNBQV-2Ik/edit). + 1. Note that [`external_url` export](https://github.com/sourcegraph/sourcegraph/pull/59014) was not added until 5.2.6+ - finding events for older instances require searching events by instance ID. +2. Check for pings, as that mechanism has not changed, and validate that the instance is is on 5.2.1+ +3. If the above don't reveal anything, reach out to #discuss-core-services for further debugging at the Telemetry Gateway level. + ## Environments ### dev