Skip to content

Commit

Permalink
Merge pull request #277 from k6io/feat/prometheus-remote-write-integr…
Browse files Browse the repository at this point in the history
…ation

Add Prometheus remote write integration docs
  • Loading branch information
mostafa authored May 19, 2021
2 parents 1334ce3 + b03f8dd commit fa9a21c
Show file tree
Hide file tree
Showing 3 changed files with 141 additions and 7 deletions.
14 changes: 8 additions & 6 deletions src/data/markdown/docs/03 cloud/03 Integrations/05 Cloud APM.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,11 @@ k6 Cloud platform supports exporting metrics to APM platforms, thereby enabling

Each supported APM platform is called a provider in Cloud APM. As you'll see in each platform's respective section, the `provider` is a key passed to the APM configuration object and its value should match the providers listed below. Also, each provider has a separate set of configuration parameters. Therefore you need to visit your provider's page:

| Provider | Platform page | Supported Regions |
| -------------- | ------------------------------------------------------------ | ---------------------------------------------------------------------------------------- |
| `datadog` | [DataDog](/cloud/integrations/cloud-apm/datadog) | [DataDog supported regions](/cloud/integrations/cloud-apm/datadog#supported-regions) |
| `azuremonitor` | [Azure Monitor](/cloud/integrations/cloud-apm/azure-monitor) | [Azure supported regions](/cloud/integrations/cloud-apm/azure-monitor#supported-regions) |
| Provider | Platform page | Supported Regions |
| -------------- | -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| `datadog` | [DataDog](/cloud/integrations/cloud-apm/datadog) | [DataDog supported regions](/cloud/integrations/cloud-apm/datadog#supported-regions) |
| `azuremonitor` | [Azure Monitor](/cloud/integrations/cloud-apm/azure-monitor) | [Azure supported regions](/cloud/integrations/cloud-apm/azure-monitor#supported-regions) |
| `prometheus` | [Prometheus Remote Write](/cloud/integrations/cloud-apm/prometheus-remote-write) | Not applicable |

This list will be expanded in the future. Please [contact us](https://k6.io/contact) if you would like an integration that isn't currently listed.

Expand Down Expand Up @@ -83,5 +84,6 @@ For the metric export to function properly, you have to fulfill the following:

## Limitations

1. APM data export is supported for tests that are up to 1 hour long. Longer tests are currently not supported.
2. The data exported in near real-time may appear incorrect until the test is finished and the 2nd pass export has completed.
1. APM data export is supported for tests that are up to 1 hour long (3600 seconds plus 30 seconds of `gracefulStop`). Longer tests are currently not supported.
2. The data exported in near real-time may appear incorrect until the test is finished and the 2nd pass export has completed. The Prometheus Remote Write integration doesn't have a 2nd pass export.
3. Duplicate APM configuration parameters of the same provider is not allowed. For example, you cannot export metrics to two Prometheus Remote Write servers at the same time. This also applies to the other providers.
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ These are the supported regions for Azure Monitor integration:

## Azure Monitor Setup

For sending custom metrics from your test run Azure Monitor, follow these instructions:
For sending custom metrics from your test run to Azure Monitor, follow these instructions:

1. Make sure you have an active subscription on Azure. Take note of your subscription ID for the configuration object.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
---
title: 'Prometheus Remote Write'
excerpt: 'How to export metrics from k6 Cloud to a Prometheus server using Remote Write'
---

## Configuration Parameters

The configuration parameters for sending metrics to Prometheus are as follows:

| Name | Description |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `provider` | Any APM provider name available in the [supported APM provider](/cloud/integrations/cloud-apm#supported-apm-providers)'s table. |
| `remoteWriteURL` | The `remoteWriteURL` provided by your Prometheus Remote Write provider or the URL of your own self-hosted Prometheus instance, for example: `http://monitoring.example.com:9090/api/v1/write`. |
| `credentials` | The `credentials` provided by your Prometheus Remote Write provider. Currently only bearer token and HTTP basic authentication mechanism are supported, as [listed](/cloud/integrations/cloud-apm/prometheus-remote-write#supported-authentication-mechanisms) below. This field is optional. |
| `metrics` | List of built-in and custom metrics to be exported. Metric names will be validated against the Prometheus [metric name conventions](https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels) and non-conforming metrics will be ignored. |
| `includeDefaultMetrics` | If set, the export will include the default metrics. Default is `true`. |
| `resampleRate` | The rate by which the metrics are resampled and sent to the APM provider in seconds. Default is 3 and acceptable values are integers between 1 and 10. |
| `includeTestRunId` | If set, the `test_run_id` will be exported per each metric as an extra tag. Default is `false`. |

## Supported Authentication Mechanisms

Currently only bearer token and HTTP basic authentication mechanisms are supported. To configure your authentication mechanism, populate `credentials` with either of these configuration parameters:

### HTTP basic authentication

| Name | Description |
| ---------- | ------------------------------------------ |
| `username` | The username in HTTP basic authentication. |
| `password` | The password in HTTP basic authentication. |

```javascript
credentials: {
username: "<username>",
password: "<password>"
}
```

### Bearer token authentication

| Name | Description |
| ------- | ----------------------------------------- |
| `token` | The bearer token without the prefix/type. |

```javascript
credentials: {
token: "<token>"
}
```

## Example Configuration Object

All the above configuration parameters are passed like this in your test run.

```javascript
export let options = {
ext: {
loadimpact: {
apm: [
{
provider: "prometheus",
remoteWriteURL: "<Remote Write URL>", // This can include query-string parameters
credentials: {
token: "<token>" // Optional
},
metrics: ["http_req_sending", "my_rate", "my_gauge", ...],
includeDefaultMetrics: true,
includeTestRunId: false
},
]
},
},
};
```

## New Relic Setup

For sending custom metrics from your test run to New Relic's Prometheus remote write integration, follow the [instructions](https://docs.newrelic.com/docs/integrations/prometheus-integrations/install-configure-remote-write/set-your-prometheus-remote-write-integration/) on their documentation.

The `prometheus_server` parameter should be included in the `remoteWriteURL` configuration parameter. The bearer token can be included either as `credentials.token` (APM configuration parameter) or as part of the `remoteWriteURL` using the `X-License-Key` parameter, as mentioned in their documentation.

So an example configuration for New Relic might look like this, with `X-License-Key` used as `token` in `credentials` would be:

```javascript
export let options = {
ext: {
loadimpact: {
apm: [
{
provider: "prometheus",
remoteWriteURL: "https://metric-api.newrelic.com/prometheus/v1/write?prometheus_server=<YOUR_DATA_SOURCE_NAME>",
credentials: {
token: "<YOUR_LICENSE_KEY>"
},
metrics: ["http_req_sending", "my_rate", "my_gauge", ...],
includeDefaultMetrics: true,
includeTestRunId: false
},
]
},
},
};
```

## Grafana Cloud Setup

For sending custom metrics from your test run to Grafana Cloud's Prometheus remote write integration, follow the [instructions](https://grafana.com/docs/grafana-cloud/metrics/prometheus/#sending-data-from-prometheus) on their documentation.

As also mentioned in their documentation, you can find the `remoteWriteURL`, username and password for your metrics endpoint by clicking on `Details` in the Prometheus card of the Grafana [Cloud Portal](https://grafana.com/docs/grafana-cloud/cloud-portal/). When you are [creating an API key](https://grafana.com/docs/grafana-cloud/cloud-portal/create-api-key/) on Grafana Cloud Portal to be used as password, make sure to use `MetricsPublisher` as role.

An example configuration for Grafana Cloud might look like this:

```javascript
export let options = {
ext: {
loadimpact: {
apm: [
{
provider: "prometheus",
remoteWriteURL: "https://prometheus-us-central1.grafana.net/api/prom/push",
credentials: {
username: "<Your Metrics instance ID>",
password: "<Your Grafana.com API Key>"
},
metrics: ["http_req_sending", "my_rate", "my_gauge", ...],
includeDefaultMetrics: true,
includeTestRunId: false
},
]
},
},
};
```

0 comments on commit fa9a21c

Please sign in to comment.