Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC-10895 Cloud 2.0 update export logs and metrics for Azure Monitor #18841

Merged
merged 9 commits into from
Aug 23, 2024
168 changes: 149 additions & 19 deletions src/current/cockroachcloud/export-logs-advanced.md

Large diffs are not rendered by default.

170 changes: 16 additions & 154 deletions src/current/cockroachcloud/export-logs.md

Large diffs are not rendered by default.

84 changes: 65 additions & 19 deletions src/current/cockroachcloud/export-metrics-advanced.md

Large diffs are not rendered by default.

61 changes: 11 additions & 50 deletions src/current/cockroachcloud/export-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ docs_area: manage
cloud: true
---

CockroachDB {{ site.data.products.standard }} users can use the [Cloud API]({% link cockroachcloud/cloud-api.md %}) to configure metrics export to [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/), [Datadog](https://www.datadoghq.com/), [Prometheus](https://prometheus.io/), or [Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-platform-metrics). Once the export is configured, metrics will flow from all nodes in all regions of your cluster to your chosen cloud metrics sink.
CockroachDB {{ site.data.products.standard }} users can use the [Cloud API]({% link cockroachcloud/cloud-api.md %}) to configure metrics export to [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/), [Datadog](https://www.datadoghq.com/), or [Prometheus](https://prometheus.io/). Once the export is configured, metrics will flow from all [regions]({% link cockroachcloud/regions.md %}) of your CockroachDB {{ site.data.products.standard }} cluster to your chosen cloud metrics sink.

{{site.data.alerts.callout_success}}
CockroachDB {{ site.data.products.standard }} clusters use [Cloud Console]({% link cockroachcloud/cluster-management.md %}#view-clusters-page) instead of [DB Console]({% link {{site.current_cloud_version}}/ui-overview.md %}), and DB Console is disabled. To export metrics from a CockroachDB {{ site.data.products.core }} cluster, refer to [Monitoring and Alerting](https://www.cockroachlabs.com/docs/{{site.current_cloud_version}}/monitoring-and-alerting) instead of this page.
CockroachDB {{ site.data.products.standard }} clusters use [Cloud Console]({% link cockroachcloud/cluster-management.md %}#view-clusters-page) instead of [DB Console]({% link {{site.current_cloud_version}}/ui-overview.md %}), and DB Console is disabled. To export metrics from a CockroachDB {{ site.data.products.core }} cluster, refer to [Monitoring and Alerting]({% link {{site.current_cloud_version}}/monitoring-and-alerting.md %}) instead of this page.
{{site.data.alerts.end}}

Exporting metrics to Amazon CloudWatch is available only on CockroachDB {{ site.data.products.standard }} clusters that are hosted on AWS. Exporting metrics to Azure Monitor is available only on CockroachDB {{ site.data.products.standard }} clusters that are hosted on Azure. Metrics export to Datadog and Prometheus is supported on all CockroachDB {{ site.data.products.standard }} clusters.
Exporting metrics to Amazon CloudWatch is available only on CockroachDB {{ site.data.products.standard }} clusters that are hosted on AWS. Metrics export to Datadog and Prometheus is supported on all CockroachDB {{ site.data.products.standard }} clusters.

<a id="the-metricexport-endpoint"></a>

Expand All @@ -22,10 +22,9 @@ To configure and manage metrics export for your CockroachDB {{ site.data.product

Cloud metrics sink | Metrics export endpoint
------------------ | ----------------------------------------------------
Amazon CloudWatch | `https://cockroachlabs.cloud/api/v1/clusters/{your_cluster_id}/metricexport/cloudwatch`
Amazon CloudWatch | `https://cockroachlabs.cloud/api/v1/clusters/{your_cluster_id}/metricexport/cloudwatch`
Datadog | `https://cockroachlabs.cloud/api/v1/clusters/{your_cluster_id}/metricexport/datadog`
Prometheus | `https://cockroachlabs.cloud/api/v1/clusters/{your_cluster_id}/metricexport/prometheus`
Azure Monitor | `https://cockroachlabs.cloud/api/v1/clusters/{your_cluster_id}/metricexport/azuremonitor`

Access to the `metricexport` endpoints requires a valid CockroachDB {{ site.data.products.cloud }} [service account]({% link cockroachcloud/managing-access.md %}#manage-service-accounts) with the appropriate permissions (`admin` privilege or Cluster Admin role).

Expand All @@ -35,7 +34,7 @@ Method | Required permissions | Description
--- | --- | ---
`GET` | `ADMIN`, `EDIT`, or `READ` | Returns the current status of the metrics export configuration.
`POST` | `ADMIN` or `EDIT` | Enables metrics export, or updates an existing metrics export configuration.
`DELETE` | `ADMIN` | Disables metrics export, halting all metrics export to Amazon CloudWatch, Datadog, Prometheus, or Azure Monitor.
`DELETE` | `ADMIN` | Disables metrics export, halting all metrics export to Amazon CloudWatch, Datadog, or Prometheus.

See [Service accounts]({% link cockroachcloud/managing-access.md %}#manage-service-accounts) for instructions on configuring a service account with these required permissions.

Expand All @@ -45,7 +44,6 @@ See [Service accounts]({% link cockroachcloud/managing-access.md %}#manage-servi
<button class="filter-button" data-scope="aws-metrics-export">Amazon CloudWatch</button>
<button class="filter-button" data-scope="datadog-metrics-export">Datadog</button>
<button class="filter-button" data-scope="prometheus-metrics-export">Prometheus</button>
<button class="filter-button" data-scope="azure-monitor-metrics-export">Azure Monitor</button>
</div>

<section class="filter-content" markdown="1" data-scope="aws-metrics-export">
Expand All @@ -54,7 +52,7 @@ See [Service accounts]({% link cockroachcloud/managing-access.md %}#manage-servi
{% include_cached feature-phases/preview.md %}
{{site.data.alerts.end}}

Exporting metrics to Amazon CloudWatch is only available on CockroachDB {{ site.data.products.dedicated }} clusters that are hosted on AWS. If your CockroachDB {{ site.data.products.dedicated }} cluster is hosted on Azure, you can export metrics to [Azure Monitor]({% link cockroachcloud/export-metrics.md %}?filters=azure-monitor-metrics-export). If your CockroachDB {{ site.data.products.dedicated }} cluster is hosted on GCP, you can export metrics to [Datadog]({% link cockroachcloud/export-metrics.md %}?filters=datadog-metrics-export) or [Prometheus]({% link cockroachcloud/export-metrics.md %}?filters=prometheus-metrics-export) instead.
Exporting metrics to Amazon CloudWatch is only available on CockroachDB {{ site.data.products.standard }} clusters which are hosted on AWS. If your CockroachDB {{ site.data.products.standard }} cluster is hosted on GCP, you can [export metrics to Datadog]({% link cockroachcloud/export-metrics.md %}?filters=datadog-metrics-export) or [Prometheus]({% link cockroachcloud/export-metrics.md %}?filters=prometheus-metrics-export) instead.

{{site.data.alerts.callout_info}}
Enabling metrics export will send around 90 metrics per [region]({% link cockroachcloud/regions.md %}) to Amazon CloudWatch. Review the [Amazon CloudWatch documentation](https://aws.amazon.com/cloudwatch/pricing/) to gauge how this adds to your Amazon CloudWatch spend.
Expand Down Expand Up @@ -173,7 +171,7 @@ Perform the following steps to enable metrics export from your CockroachDB {{ si
{% include_cached feature-phases/preview.md %}
{{site.data.alerts.end}}

To enable metrics export for your CockroachDB {{ site.data.products.dedicated }} cluster to Datadog, you can either enable the Datadog integration in your CockroachDB {{ site.data.products.dedicated }} Cloud Console, or on the command line via the [Cloud API]({% link cockroachcloud/cloud-api.md %}):
To enable metrics export for your CockroachDB {{ site.data.products.standard }} cluster to Datadog, you can either:

- Use the Cloud Console, following the [Monitor CockroachDB {{ site.data.products.cloud }} with Datadog]({% link cockroachcloud/tools-page.md %}#monitor-cockroachdb-cloud-with-datadog) instructions.

Expand Down Expand Up @@ -337,7 +335,6 @@ A subset of CockroachDB metrics require that you explicitly [enable percentiles]
<button class="filter-button" data-scope="aws-metrics-export">Amazon CloudWatch</button>
<button class="filter-button" data-scope="datadog-metrics-export">Datadog</button>
<button class="filter-button" data-scope="prometheus-metrics-export">Prometheus</button>
<button class="filter-button" data-scope="azure-monitor-metrics-export">Azure Monitor</button>
</div>

<section class="filter-content" markdown="1" data-scope="aws-metrics-export">
Expand Down Expand Up @@ -394,24 +391,6 @@ Where:

</section>

<section class="filter-content" markdown="1" data-scope="azure-monitor-metrics-export">

To check the status of an existing Azure Monitor metrics export configuration, use the following Cloud API command:

{% include_cached copy-clipboard.html %}
~~~shell
curl --request GET \
--url https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/metricexport/azuremonitor \
--header "Authorization: Bearer {secret_key}"
~~~

Where:

- `{cluster_id}` is your CockroachDB {{ site.data.products.dedicated }} cluster's cluster ID, which can be found in the URL of your [Cloud Console](https://cockroachlabs.cloud/clusters/) for the specific cluster you wish to configure, resembling `f78b7feb-b6cf-4396-9d7f-494982d7d81e`.
- `{secret_key}` is your CockroachDB {{ site.data.products.dedicated }} API key. See [API Access]({% link cockroachcloud/managing-access.md %}) for instructions on generating this key.

</section>

## Update an existing metrics export configuration

To update an existing CockroachDB {{ site.data.products.standard }} metrics export configuration, make any necessary changes to your cloud provider configuration (e.g., Amazon CloudWatch, Datadog, or Prometheus), then issue the same `POST` Cloud API command as shown in the [Enable metrics export](#enable-metrics-export) instructions for your cloud provider with the desired updated configuration. Follow the [Monitor the status of a metrics export configuration](#monitor-the-status-of-a-metrics-export-configuration) instructions to ensure the update completes successfully.
Expand All @@ -422,12 +401,11 @@ To update an existing CockroachDB {{ site.data.products.standard }} metrics expo
<button class="filter-button" data-scope="aws-metrics-export">Amazon CloudWatch</button>
<button class="filter-button" data-scope="datadog-metrics-export">Datadog</button>
<button class="filter-button" data-scope="prometheus-metrics-export">Prometheus</button>
<button class="filter-button" data-scope="azure-monitor-metrics-export">Azure Monitor</button>
</div>

<section class="filter-content" markdown="1" data-scope="aws-metrics-export">

To disable an existing Amazon CloudWatch metrics export configuration, and stop sending metrics to Cloudwatch, use the following Cloud API command:
To disable an existing Amazon CloudWatch metrics export configuration, and stop sending metrics to CloudWatch, use the following Cloud API command:

{% include_cached copy-clipboard.html %}
~~~shell
Expand Down Expand Up @@ -479,26 +457,9 @@ Where:

</section>

<section class="filter-content" markdown="1" data-scope="azure-monitor-metrics-export">

To disable an existing Azure Monitor metrics export configuration, and stop sending metrics to Azure Monitor, use the following Cloud API command:

{% include_cached copy-clipboard.html %}
~~~shell
curl --request DELETE \
--url https://cockroachlabs.cloud/api/v1/clusters/{cluster_id}/metricexport/azuremonitor \
--header "Authorization: Bearer {secret_key}"
~~~

Where:

- `{cluster_id}` is your CockroachDB {{ site.data.products.dedicated }} cluster's cluster ID, which can be found in the URL of your [Cloud Console](https://cockroachlabs.cloud/clusters/) for the specific cluster you wish to configure, resembling `f78b7feb-b6cf-4396-9d7f-494982d7d81e`.
- `{secret_key}` is your CockroachDB {{ site.data.products.dedicated }} API key. See [API Access]({% link cockroachcloud/managing-access.md %}) for instructions on generating this key.
</section>

## Limitations

- Metrics export to Amazon CloudWatch is available only on CockroachDB {{ site.data.products.standard }} clusters which are hosted on AWS. Metrics export to Azure is available only on CockroachDB {{ site.data.products.dedicated }} clusters which are hosted on Azure. If your CockroachDB {{ site.data.products.standard }} cluster is hosted on GCP, you can [export metrics to Datadog](export-metrics.html?filters=datadog-metrics-export) or [Prometheus]({% link cockroachcloud/export-metrics.md %}?filters=prometheus-metrics-export) instead.
- Metrics export to Amazon CloudWatch is available only on CockroachDB {{ site.data.products.standard }} clusters which are hosted on AWS. If your CockroachDB {{ site.data.products.standard }} cluster is hosted on GCP, you can [export metrics to Datadog]({% link cockroachcloud/export-metrics.md %}?filters=datadog-metrics-export) or [Prometheus]({% link cockroachcloud/export-metrics.md %}?filters=prometheus-metrics-export) instead.
- Amazon CloudWatch does not currently support histograms. Any histogram-type metrics emitted from your CockroachDB {{ site.data.products.standard }} cluster are dropped by CloudWatch. See [Prometheus metric type conversion](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-Prometheus-metrics-conversion.html) for more information, and [Logging dropped Prometheus metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-Prometheus-troubleshooting-EKS.html#ContainerInsights-Prometheus-troubleshooting-droppedmetrics) for instructions on tracking dropped histogram metrics in CloudWatch.

## Troubleshooting
Expand All @@ -509,9 +470,9 @@ Be sure you are providing **your own** AWS Account ID as shown on the AWS [IAM p

If you are using an existing AWS role, or are otherwise using a role name different from the example name used in this tutorial, be sure to use your own role name in step 8 in place of `CockroachCloudMetricsExportRole`.

Your CockroachDB {{ site.data.products.standard }} cluster must be running on AWS (not GCP or Azure) to make use of metrics export to Amazon CloudWatch. If your CockroachDB {{ site.data.products.standard }} cluster is hosted on Azure, you can export metrics to [Azure Monitor]({% link cockroachcloud/export-metrics.md %}?filters=azure-monitor-metrics-export). If your CockroachDB {{ site.data.products.standard }} cluster is hosted on GCP, you can [export metrics to Datadog](export-metrics.html?filters=datadog-metrics-export) or [Prometheus]({% link cockroachcloud/export-metrics.md %}?filters=prometheus-metrics-export) instead.
Your CockroachDB {{ site.data.products.standard }} cluster must be running on AWS (not GCP) to make use of metrics export to Amazon CloudWatch. If your CockroachDB {{ site.data.products.standard }} cluster is hosted on GCP, you can [export metrics to Datadog]({% link cockroachcloud/export-metrics.md %}?filters=datadog-metrics-export) or [Prometheus]({% link cockroachcloud/export-metrics.md %}?filters=prometheus-metrics-export) instead.

## See Also

- [Export Metrics From a CockroachDB Advanced Cluster]({% link cockroachcloud/export-metrics-advanced.md %})
- [Export Metrics From a CockroachDB {{ site.data.products.advanced }} Cluster]({% link cockroachcloud/export-metrics-advanced.md %})
- [Differences in Metrics between Third-Party Monitoring Integrations and DB Console]({% link {{site.current_cloud_version}}/differences-in-metrics-between-third-party-monitoring-integrations-and-db-console.md %})
8 changes: 4 additions & 4 deletions src/current/v23.1/cockroachdb-feature-availability.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ General availability (GA) | Feature is production-ready and
{{site.data.alerts.end}}

### Export logs to Azure Monitor
[Exporting logs to Azure Monitor]({% link cockroachcloud/export-logs.md %}?filters=azure-monitor-log-export) from your CockroachDB {{ site.data.products.dedicated }} cluster hosted on Azure is in limited access. Once the export is configured, logs will flow from all nodes in all regions of your CockroachDB {{ site.data.products.dedicated }} cluster to Azure Monitor. To express interest and try it out, contact [Support](https://support.cockroachlabs.com/hc).
[Exporting logs to Azure Monitor]({% link cockroachcloud/export-logs-advanced.md %}?filters=azure-monitor-log-export) from your CockroachDB {{ site.data.products.advanced }} cluster hosted on Azure is in limited access. Once the export is configured, logs will flow from all nodes in all regions of your CockroachDB {{ site.data.products.advanced }} cluster to Azure Monitor. To express interest and try it out, contact [Support](https://support.cockroachlabs.com/hc).

### Export metrics to Azure Monitor
[Exporting Metrics to Azure Monitor]({% link cockroachcloud/export-metrics.md %}?filters=azure-monitor-metrics-export) from a CockroachDB {{ site.data.products.dedicated }} cluster hosted on Azure is in limited access. Once the export is configured, metrics will flow from all nodes in all regions of your CockroachDB {{ site.data.products.dedicated }} cluster to your chosen cloud metrics sink. To express interest and try it out, contact [Support](https://support.cockroachlabs.com/hc).
[Exporting Metrics to Azure Monitor]({% link cockroachcloud/export-metrics-advanced.md %}?filters=azure-monitor-metrics-export) from a CockroachDB {{ site.data.products.advanced }} cluster hosted on Azure is in limited access. Once the export is configured, metrics will flow from all nodes in all regions of your CockroachDB {{ site.data.products.advanced }} cluster to your chosen cloud metrics sink. To express interest and try it out, contact [Support](https://support.cockroachlabs.com/hc).

### AWS PrivateLink for CockroachDB Serverless
[Connecting privately to a multi-region CockroachDB {{ site.data.products.serverless }} cluster using AWS PrivateLink]({% link cockroachcloud/aws-privatelink.md %}?filters=serverless) is in limited access. This can help your organization meet its security requirements and reduce your cluster's exposure to public networks. To express interest and try it out, contact [Support](https://support.cockroachlabs.com/hc).
Expand All @@ -50,8 +50,8 @@ General availability (GA) | Feature is production-ready and

The [**Custom Metrics Chart** page]({% link cockroachcloud/custom-metrics-chart-page.md %}) for CockroachDB {{ site.data.products.cloud }} clusters allows you to create custom charts showing the time series data for an available metric or combination of metrics.

### Export metrics from CockroachDB {{ site.data.products.dedicated }} clusters
[Exporting metrics from CockroachDB {{ site.data.products.dedicated }}]({% link cockroachcloud/export-metrics.md %}) to [AWS CloudWatch](https://aws.amazon.com/cloudwatch/) or [Datadog](https://www.datadoghq.com/) using the [Cloud API]({% link cockroachcloud/cloud-api.md %}) is in preview. Once the export is configured, metrics will flow from all nodes in all regions of your CockroachDB {{ site.data.products.dedicated }} cluster to your chosen cloud metrics sink.
### Export metrics from CockroachDB {{ site.data.products.advanced }} clusters
[Exporting metrics from CockroachDB {{ site.data.products.advanced }}]({% link cockroachcloud/export-metrics-advanced.md %}) to [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/) or [Datadog](https://www.datadoghq.com/) using the [Cloud API]({% link cockroachcloud/cloud-api.md %}) is in preview. Once the export is configured, metrics will flow from all nodes in all regions of your CockroachDB {{ site.data.products.advanced }} cluster to your chosen cloud metrics sink.

{{site.data.alerts.callout_info}}
Exporting metrics to Azure Monitor is in limited access. Refer to [Exporting metrics to Azure Monitor](#export-metrics-to-azure-monitor).
Expand Down
Loading
Loading