Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Splunk observability scaler #6534

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

sschimper-splunk
Copy link

Provide a description of what has been changed

Checklist

  • When introducing a new scaler, I agree with the scaling governance policy
  • I have verified that my change is according to the deprecations & breaking changes policy
  • Tests have been added
  • Changelog has been updated and is aligned with our changelog requirements
  • [] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
  • A PR is opened to update the documentation on (repo) (if applicable)
  • Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #

Relates to #

@sschimper-splunk sschimper-splunk requested a review from a team as a code owner February 7, 2025 13:29
Copy link
Member

@JorTurFer JorTurFer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution! Some comments inline
Apart from them, please execute go mod tidy & go mod vendor and commit the output

@@ -0,0 +1,207 @@
//go:build e2e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a cloud service so we will need to create an account. Is Splunk Free enough to exceute this e2e test?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, no. The Splunk Observability Cloud (former SignalFX) is a different product.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would need an account then, @sschimper-splunk is this something you can help us arrange? We have done that with other service providers.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @sschimper-splunk, thanks for the heads-up. I will see how I can help!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could perhaps mock the response with a web server and override the endpoint being passed into the splunk-observability scaler?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with that approach is that any change in the cloud service will not be detected and will require mocks update. Honestly, I prefer to not use mocks for e2e tests. Does this service run as docker image by chance?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with @JorTurFer here

Realm string `keda:"name=realm, order=authParams"`
Query string `keda:"name=query, order=triggerMetadata"`
Duration int `keda:"name=duration, order=triggerMetadata"`
TargetValue float64 `keda:"name=targetValue, order=triggerMetadata"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Align this line please


s.logger.V(1).Info("Started MTS stream.")

time.Sleep(time.Duration(s.metadata.Duration * int(time.Second)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sleep required?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SignalFlow Go Client library that I am using to integrate with Splunk O11y does not let you query a precise value, instead it provides a stream value series for a specified timeframe. I use this sleep to say "let the stream open for <s.metadata.Duration> seconds and then close it.
There might be some more elegant ways, but I am still fairly new to Golang and would very much appreciate feedback. An example that uses Goroutines can be found here: https://github.com/signalfx/signalflow-client-go/blob/main/signalflow/example_test.go
But I found this a bit over-kill

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, please, we should avoid using Sleep

}

func (s *splunkObservabilityScaler) getQueryResult() (float64, error) {
comp, err := s.apiClient.Execute(context.Background(), &signalflow.ExecuteRequest{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's propagate the context until here.

s.logger.V(1).Info("Started MTS stream.")

time.Sleep(time.Duration(s.metadata.Duration * int(time.Second)))
if err := comp.Stop(context.Background()); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's propagate the context until here.

Comment on lines 57 to 58
{&testSplunkObservabilityMetadata[0], 0, "demo-trans-latency"},
{&testSplunkObservabilityMetadata[0], 1, "demo-trans-latency"},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this test is failing because the metric name must contain the index (it's done by GenerateMetricNameWithIndex)

Suggested change
{&testSplunkObservabilityMetadata[0], 0, "demo-trans-latency"},
{&testSplunkObservabilityMetadata[0], 1, "demo-trans-latency"},
{&testSplunkObservabilityMetadata[0], 0, "s0-signalfx"},
{&testSplunkObservabilityMetadata[0], 1, "s1-signalfx"},

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants