-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[receiver/valkey]: Init new component #37005
base: main
Are you sure you want to change the base?
Changes from 24 commits
d8e1a23
df723f7
df456b4
bd9ffef
613dd31
60963de
7d7655d
9898ae9
480f82c
d7ceb6d
66856f7
8ec3e78
4e81208
f8c2c52
a64d888
c48db20
1644903
3c08b49
fc0f24b
1a76c23
85454e4
018a5e4
3269428
db230b1
7b4e7d7
123aa3e
8309e1c
1c23a7c
7b7df5c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# Use this changelog template to create an entry for release notes. | ||
|
||
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' | ||
change_type: 'new_component' | ||
|
||
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver) | ||
component: valkeyreceiver | ||
|
||
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). | ||
note: Init code for Valkey receiver | ||
|
||
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. | ||
issues: [33787] | ||
|
||
# (Optional) One or more lines of additional information to render under the primary note. | ||
# These lines will be padded with 2 spaces and then inserted directly into the document. | ||
# Use pipe (|) for multiline entries. | ||
subtext: | ||
|
||
# If your change doesn't affect end users or the exported elements of any package, | ||
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label. | ||
# Optional: The change log or logs in which this entry should be included. | ||
# e.g. '[user]' or '[user, api]' | ||
# Include 'user' if the change is relevant to end users. | ||
# Include 'api' if there is a change to a library API. | ||
# Default: '[user]' | ||
change_logs: [] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
include ../../Makefile.Common |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# Valkey Receiver | ||
|
||
<!-- status autogenerated section --> | ||
| Status | | | ||
| ------------- |-----------| | ||
| Stability | [development]: metrics | | ||
| Distributions | [contrib] | | ||
| Issues | [![Open issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aopen%20label%3Areceiver%2Fvalkey%20&label=open&color=orange&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aopen+is%3Aissue+label%3Areceiver%2Fvalkey) [![Closed issues](https://img.shields.io/github/issues-search/open-telemetry/opentelemetry-collector-contrib?query=is%3Aissue%20is%3Aclosed%20label%3Areceiver%2Fvalkey%20&label=closed&color=blue&logo=opentelemetry)](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues?q=is%3Aclosed+is%3Aissue+label%3Areceiver%2Fvalkey) | | ||
| [Code Owners](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#becoming-a-code-owner) | [@dmitryax](https://www.github.com/dmitryax), [@rogercoll](https://www.github.com/rogercoll) | | ||
|
||
[development]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/component-stability.md#development | ||
[contrib]: https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib | ||
<!-- end autogenerated section --> | ||
|
||
The Valkey receiver is designed to retrieve Valkey INFO data from a single | ||
Valkey instance, build metrics from that data, and send them to the next consumer at a | ||
configurable interval. | ||
|
||
## Details | ||
|
||
The Valkey INFO command returns information and statistics about a Valkey | ||
server (see [https://valkey.io/commands/info/](https://valkey.io/commands/info/) for | ||
details). The Valkey receiver extracts values from the result and converts them to open | ||
telemetry metrics. Details about the metrics produced by the Valkey receiver | ||
can be found by browsing [the metadata file](./metadata.yaml). | ||
|
||
## Configuration | ||
|
||
The following settings are required: | ||
|
||
- `endpoint` (no default): The hostname and port of the Valkey instance, | ||
separated by a colon. | ||
|
||
The following settings are optional: | ||
|
||
- `collection_interval` (default = `10s`): This receiver runs on an interval. | ||
Each time it runs, it queries Valkey, creates metrics, and sends them to the | ||
next consumer. The `collection_interval` configuration option tells this | ||
receiver the duration between runs. This value must be a string readable by | ||
Golang's `ParseDuration` function (example: `1h30m`). Valid time units are | ||
`ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. | ||
- `tls`: see [TLS Configuration Settings](https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/README.md#tls-configuration-settings) for the full set of available options. | ||
|
||
Example: | ||
|
||
```yaml | ||
receivers: | ||
valkey: | ||
endpoint: "localhost:6379" | ||
collection_interval: 10s | ||
``` | ||
|
||
The full list of settings exposed for this receiver are documented [here](./config.go). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
package valkeyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/valkeyreceiver" | ||
|
||
import ( | ||
"context" | ||
"fmt" | ||
"strings" | ||
|
||
"github.com/valkey-io/valkey-go" | ||
) | ||
|
||
var attrDelimiter = "\r\n" | ||
|
||
// Interface for a Valkey client. Implementation can be faked for testing. | ||
type client interface { | ||
// retrieves a string of key/value pairs of valkey metadata | ||
retrieveInfo(context.Context) (map[string]string, error) | ||
// close release Valkey client connection pool | ||
close() error | ||
} | ||
|
||
// Wraps a real Valkey client, implements `client` interface. | ||
type valkeyClient struct { | ||
client valkey.Client | ||
} | ||
|
||
var _ client = (*valkeyClient)(nil) | ||
|
||
// Creates a new real Valkey client from the passed-in valkey.Options. | ||
func newValkeyClient(options valkey.ClientOption) (client, error) { | ||
innerClient, err := valkey.NewClient(options) | ||
if err != nil { | ||
return nil, err | ||
} | ||
return &valkeyClient{innerClient}, nil | ||
} | ||
|
||
// Retrieve Valkey INFO. We retrieve all of the 'sections'. | ||
func (c *valkeyClient) retrieveInfo(ctx context.Context) (map[string]string, error) { | ||
str, err := c.client.Do(ctx, c.client.B().Info().Build()).ToString() | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
return parseRawDataMap(str) | ||
} | ||
|
||
func parseRawDataMap(data string) (map[string]string, error) { | ||
attrs := make(map[string]string) | ||
lines := strings.Split(data, attrDelimiter) | ||
for _, line := range lines { | ||
if len(line) == 0 || strings.HasPrefix(line, "#") { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The lines with a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since Although I like the idea of grouping the data by section, I would suggest keeping this feature for a future PR if needed. The current exported client metrics are not affected by the tested version (integration test 7.2)." Wdyt? |
||
continue | ||
} | ||
key, value, found := strings.Cut(line, ":") | ||
if !found { | ||
return nil, fmt.Errorf("could not cut line %q using \":\" as delimiter", line) | ||
} | ||
attrs[key] = value | ||
} | ||
return attrs, nil | ||
} | ||
|
||
// close client to release connection pool. | ||
func (c *valkeyClient) close() error { | ||
c.client.Close() | ||
return nil | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package valkeyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/valkeyreceiver" | ||
|
||
import ( | ||
"fmt" | ||
"net" | ||
|
||
"go.opentelemetry.io/collector/config/confignet" | ||
"go.opentelemetry.io/collector/config/configtls" | ||
"go.opentelemetry.io/collector/receiver/scraperhelper" | ||
|
||
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/valkeyreceiver/internal/metadata" | ||
) | ||
|
||
type Config struct { | ||
scraperhelper.ControllerConfig `mapstructure:",squash"` | ||
|
||
// The target endpoint. | ||
confignet.AddrConfig `mapstructure:",squash"` | ||
|
||
TLS configtls.ClientConfig `mapstructure:"tls,omitempty"` | ||
|
||
MetricsBuilderConfig metadata.MetricsBuilderConfig `mapstructure:",squash"` | ||
} | ||
|
||
// configInfo holds configuration information to be used as resource/metrics attributes. | ||
type configInfo struct { | ||
Address string | ||
Port string | ||
} | ||
|
||
func newConfigInfo(cfg *Config) (configInfo, error) { | ||
address, port, err := net.SplitHostPort(cfg.Endpoint) | ||
if err != nil { | ||
return configInfo{}, fmt.Errorf("invalid endpoint %q: %w", cfg.Endpoint, err) | ||
} | ||
return configInfo{Address: address, Port: port}, nil | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
//go:generate ../../.tools/mdatagen metadata.yaml | ||
|
||
package valkeyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/valkeyreceiver" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
[comment]: <> (Code generated by mdatagen. DO NOT EDIT.) | ||
|
||
# valkey | ||
|
||
## Default Metrics | ||
|
||
The following metrics are emitted by default. Each of them can be disabled by applying the following configuration: | ||
|
||
```yaml | ||
metrics: | ||
<metric_name>: | ||
enabled: false | ||
``` | ||
|
||
### valkey.client.connection.count | ||
|
||
The number of connections that are currently in state described by the state attribute | ||
|
||
| Unit | Metric Type | Value Type | | ||
| ---- | ----------- | ---------- | | ||
| {connection} | Gauge | Int | | ||
|
||
#### Attributes | ||
|
||
| Name | Description | Values | | ||
| ---- | ----------- | ------ | | ||
| valkey.client.connection.state | The state of a connection in the pool | Str: ``used``, ``blocked``, ``tracking`` | | ||
|
||
## Resource Attributes | ||
|
||
| Name | Description | Values | Enabled | | ||
| ---- | ----------- | ------ | ------- | | ||
| server.address | Valkey server's address | Any Str | false | | ||
| server.port | Valkey server's port | Any Str | false | | ||
| valkey.version | Valkey server's version. | Any Str | true | |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package valkeyreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/valkeyreceiver" | ||
|
||
import ( | ||
"context" | ||
"time" | ||
|
||
"go.opentelemetry.io/collector/component" | ||
"go.opentelemetry.io/collector/config/confignet" | ||
"go.opentelemetry.io/collector/config/configtls" | ||
"go.opentelemetry.io/collector/consumer" | ||
"go.opentelemetry.io/collector/receiver" | ||
"go.opentelemetry.io/collector/receiver/scraperhelper" | ||
"go.opentelemetry.io/collector/scraper" | ||
|
||
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/valkeyreceiver/internal/metadata" | ||
) | ||
|
||
// NewFactory creates a factory for Valkey receiver. | ||
func NewFactory() receiver.Factory { | ||
return receiver.NewFactory( | ||
metadata.Type, | ||
createDefaultConfig, | ||
receiver.WithMetrics(createMetricsReceiver, metadata.MetricsStability)) | ||
} | ||
|
||
func createDefaultConfig() component.Config { | ||
scs := scraperhelper.NewDefaultControllerConfig() | ||
scs.CollectionInterval = 10 * time.Second | ||
return &Config{ | ||
AddrConfig: confignet.AddrConfig{ | ||
Transport: confignet.TransportTypeTCP, | ||
}, | ||
TLS: configtls.ClientConfig{ | ||
Insecure: true, | ||
}, | ||
ControllerConfig: scs, | ||
MetricsBuilderConfig: metadata.DefaultMetricsBuilderConfig(), | ||
} | ||
} | ||
|
||
func createMetricsReceiver( | ||
_ context.Context, | ||
set receiver.Settings, | ||
cfg component.Config, | ||
consumer consumer.Metrics, | ||
) (receiver.Metrics, error) { | ||
oCfg := cfg.(*Config) | ||
|
||
scrp, err := newValkeyScraper(oCfg, set) | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
scraper, err := scraper.NewMetrics( | ||
scraper.ScrapeMetricsFunc(scrp.scrape), | ||
scraper.WithShutdown(scrp.shutdown), | ||
) | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
return scraperhelper.NewScraperControllerReceiver(&oCfg.ControllerConfig, set, consumer, scraperhelper.AddScraper(metadata.Type, scraper)) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Small nit: I think maybe it's not reasonable to set up a too shot collection interval to prevent too much stress on Valkey.
Therefore, we can set up to minimum duration check in code to validate it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable to me, which low bound would you propose? Maybe 1 second?
Do we have any other scraper that sets a minimum collection interval value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I'm not sure about this, but 1s looks good to me, even though receiver can fetch metric the at such granularity, but I think in most cases, the user won't store or query the metrics in that granularity. For me I often query the metric in the granularity of one minute. It's enough for me in most cases.