Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reflect] sdk/log/xlog: Add FilterProcessor and EnabledParameters #6286

Closed
wants to merge 40 commits into from
Closed
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
96a79ee
mv sdk/log/internal/x sdk/xlog
pellared Feb 5, 2025
6282e71
Introduce xlog module with FilterProcessor and EnabledParameters
pellared Feb 5, 2025
69ff66d
Update changelog
pellared Feb 5, 2025
bf61d44
Add trace context fields
pellared Feb 5, 2025
5b3c8ca
Test FilterProcessor functionality
pellared Feb 5, 2025
f24fc64
Update changelog
pellared Feb 5, 2025
fffc077
go mod tidy
pellared Feb 5, 2025
55adcc4
Merge branch 'main' into sdk/xlogs
pellared Feb 5, 2025
008cf4d
Merge branch 'main' into sdk/xlogs
pellared Feb 6, 2025
4df6669
Remove span context fields from EnabledParameters
pellared Feb 6, 2025
d31f329
Simplify doc comments
pellared Feb 6, 2025
fe5ae63
go mod tidy
pellared Feb 6, 2025
4e54827
Move sdk/xlog to sdk/log/xlog
pellared Feb 6, 2025
abe7833
Fix hyperlink
pellared Feb 6, 2025
d5817c6
Update README.md
pellared Feb 6, 2025
c23714b
Merge branch 'main' into sdk/xlogs
pellared Feb 6, 2025
cb91d27
Merge branch 'main' into sdk/xlogs
pellared Feb 7, 2025
71376d1
Update sdk/log/example_test.go
pellared Feb 7, 2025
e6b34ae
Update sdk/log/logger.go
pellared Feb 7, 2025
4ff41e1
Format comment
pellared Feb 7, 2025
628f916
Add missing period
pellared Feb 7, 2025
98d6811
Do not release xlog
pellared Feb 7, 2025
5cf1eaf
Improve Go doc comments
pellared Feb 7, 2025
9037771
sdk/log to use internal copy of interface instead of xlog
pellared Feb 7, 2025
3a07761
Cleanup go mod
pellared Feb 7, 2025
c56e9da
code snippet: TestReflectFilterProcessorPlayground
pellared Feb 7, 2025
91d84a9
Support xlog with duck typing using reflect
pellared Feb 7, 2025
685a417
Remove TestReflectFilterProcessorPlayground
pellared Feb 7, 2025
0547924
Fix typos
pellared Feb 7, 2025
7075931
Update CHANGELOG.md
pellared Feb 7, 2025
648e21f
Update CHANGELOG.md
pellared Feb 7, 2025
2941fb3
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 7, 2025
2a93c9f
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 10, 2025
9da340a
Apply suggestions from code review
pellared Feb 11, 2025
d55948e
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 11, 2025
08ac146
Remove usage of xlog in SDK
pellared Feb 11, 2025
9475ed8
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 12, 2025
25937e0
Add otel-experimental depguard rule
pellared Feb 12, 2025
e4d1ebf
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 12, 2025
2ac7f71
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- The `go.opentelemetry.io/otel/semconv/v1.30.0` package.
The package contains semantic conventions from the `v1.30.0` version of the OpenTelemetry Semantic Conventions.
See the [migration documentation](./semconv/v1.30.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.28.0`(#6240)
- Make the initial release of `go.opentelemetry.io/otel/sdk/log/xlog`.
This new module contains experimental features of the OpenTelemetry Logs SDK.
It replaces `go.opentelemetry.io/otel/sdk/log/internal/x`.
This module is unstable and breaking changes may be introduced.
See our [versioning policy](VERSIONING.md) for more information about these stability guarantees. (#6286)
- Add `FilterProcessor` and `EnabledParameters` in `go.opentelemetry.io/otel/sdk/log/xlog` .
Compared to previous version it additionally gives the possibility to filter by resource and instrumentation scope. (#6286)

### Changed

Expand Down
2 changes: 2 additions & 0 deletions exporters/otlp/otlplog/otlploggrpc/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,5 @@ replace go.opentelemetry.io/otel/log => ../../../../log
replace go.opentelemetry.io/otel/trace => ../../../../trace

replace go.opentelemetry.io/otel/metric => ../../../../metric

replace go.opentelemetry.io/otel/sdk/log/xlog => ../../../../sdk/log/xlog
2 changes: 2 additions & 0 deletions exporters/otlp/otlplog/otlploghttp/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,5 @@ replace go.opentelemetry.io/otel/sdk => ../../../../sdk
replace go.opentelemetry.io/otel/metric => ../../../../metric

replace go.opentelemetry.io/otel/log => ../../../../log

replace go.opentelemetry.io/otel/sdk/log/xlog => ../../../../sdk/log/xlog
2 changes: 2 additions & 0 deletions exporters/stdout/stdoutlog/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,5 @@ replace go.opentelemetry.io/otel/trace => ../../../trace
replace go.opentelemetry.io/otel/sdk => ../../../sdk

replace go.opentelemetry.io/otel/metric => ../../../metric

replace go.opentelemetry.io/otel/sdk/log/xlog => ../../../sdk/log/xlog
12 changes: 8 additions & 4 deletions log/logger.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,13 @@ type Logger interface {
// Enabled returns whether the Logger emits for the given context and
// param.
//
// The passed param is likely to be a partial record with only the
// bridge-relevant information being provided (e.g a param with only the
// Severity set). If a Logger needs more information than is provided, it
// This is useful for users that want to know if a [Record]
// will be processed or dropped before they perform complex operations to
// construct the [Record].
//
// The passed param is likely to be a partial record information being
// provided (e.g a param with only the Severity set).
// If a Logger needs more information than is provided, it
// is said to be in an indeterminate state (see below).
//
// The returned value will be true when the Logger will emit for the
Expand All @@ -46,7 +50,7 @@ type Logger interface {
// exist (e.g. performance, correctness).
//
// The param should not be held by the implementation. A copy should be
// made if the record needs to be held after the call returns.
// made if the param needs to be held after the call returns.
//
// Implementations of this method need to be safe for a user to call
// concurrently.
Expand Down
2 changes: 1 addition & 1 deletion sdk/log/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ at a single endpoint their origin is decipherable.
See [go.opentelemetry.io/otel/log] for more information about
the OpenTelemetry Logs Bridge API.

See [go.opentelemetry.io/otel/sdk/log/internal/x] for information about the
See [go.opentelemetry.io/otel/sdk/log/xlog] for information about the
experimental features.
*/
package log // import "go.opentelemetry.io/otel/sdk/log"
16 changes: 8 additions & 8 deletions sdk/log/example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
logapi "go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/log/global"
"go.opentelemetry.io/otel/sdk/log"
"go.opentelemetry.io/otel/sdk/log/xlog"
)

// Initialize OpenTelemetry Logs SDK and setup logging using a log bridge.
Expand Down Expand Up @@ -52,6 +53,7 @@ func Example() {
}

// Use a processor that filters out records based on the provided context.
// It also demonstrates the use of experimental [xlog.FilterProcessor].
func ExampleProcessor_filtering() {
// Existing processor that emits telemetry.
var processor log.Processor = log.NewBatchProcessor(nil)
Expand Down Expand Up @@ -84,14 +86,12 @@ type ContextFilterProcessor struct {
log.Processor

lazyFilter sync.Once
// Use the experimental FilterProcessor interface
// (go.opentelemetry.io/otel/sdk/log/internal/x).
filter filter
// Support the experimental FilterProcessor interface for the embedded processor.
filter xlog.FilterProcessor
}

type filter interface {
Enabled(ctx context.Context, param logapi.EnabledParameters) bool
}
// Compile time check.
var _ xlog.FilterProcessor = (*ContextFilterProcessor)(nil)

func (p *ContextFilterProcessor) OnEmit(ctx context.Context, record *log.Record) error {
if ignoreLogs(ctx) {
Expand All @@ -100,9 +100,9 @@ func (p *ContextFilterProcessor) OnEmit(ctx context.Context, record *log.Record)
return p.Processor.OnEmit(ctx, record)
}

func (p *ContextFilterProcessor) Enabled(ctx context.Context, param logapi.EnabledParameters) bool {
func (p *ContextFilterProcessor) Enabled(ctx context.Context, param xlog.EnabledParameters) bool {
p.lazyFilter.Do(func() {
if f, ok := p.Processor.(filter); ok {
if f, ok := p.Processor.(xlog.FilterProcessor); ok {
p.filter = f
}
})
Expand Down
88 changes: 88 additions & 0 deletions sdk/log/filter_processor.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package log // import "go.opentelemetry.io/otel/sdk/log"

import (
"context"
"reflect"

"go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/sdk/instrumentation"
"go.opentelemetry.io/otel/sdk/resource"
)

// filterProcessor uses reflect to support [go.opentelemetry.io/otel/sdk/log/xlog.FilterProcessor]
// via duck typing.
type filterProcessor struct {
enabledFn reflect.Value
paramType reflect.Type
}

func asFilterProccessor(p any) (filterProcessor, bool) {
fp := reflect.ValueOf(p)
m := fp.MethodByName("Enabled")
if m == (reflect.Value{}) {
// No Enabled method.
return filterProcessor{}, false
}
mty := m.Type()
if mty.NumOut() != 1 {
// Should return one output parameter.
return filterProcessor{}, false
}
if reflect.Bool != mty.Out(0).Kind() {
// Should return bool.
return filterProcessor{}, false
}
if mty.NumIn() != 2 {
// Should have two input parameters.
return filterProcessor{}, false
}
if mty.In(0) != reflect.TypeFor[context.Context]() {
// Should have context.Context as first input parameter.
return filterProcessor{}, false
}
// Duck typing of EnabledParameters
pt := mty.In(1)
if res, ok := pt.FieldByName("Resource"); !ok || res.Type != reflect.TypeFor[resource.Resource]() {
// The second parameter should have Resource resource.Resource field.
return filterProcessor{}, false
}
if res, ok := pt.FieldByName("InstrumentationScope"); !ok || res.Type != reflect.TypeFor[instrumentation.Scope]() {
// The second parameter should have InstrumentationScope instrumentation.Scope field.
return filterProcessor{}, false
}
if res, ok := pt.FieldByName("Severity"); !ok || res.Type != reflect.TypeFor[log.Severity]() {
// The second parameter should have Severity log.Severity field.
return filterProcessor{}, false
}

return filterProcessor{
enabledFn: m,
paramType: pt,
}, true
}

func (f filterProcessor) Enabled(ctx context.Context, param enabledParameters) bool {
p := reflect.New(f.paramType).Elem()
p.FieldByName("Resource").Set(reflect.ValueOf(param.Resource))
p.FieldByName("InstrumentationScope").Set(reflect.ValueOf(param.InstrumentationScope))
p.FieldByName("Severity").Set(reflect.ValueOf(param.Severity))

ctxV := reflect.ValueOf(ctx)
if ctxV == (reflect.Value{}) {
// In order to not get panic: reflect: Call using zero Value argument.
ctxV = reflect.Zero(reflect.TypeFor[context.Context]())
}

ret := f.enabledFn.Call([]reflect.Value{ctxV, p})
return ret[0].Bool()
}

// enabledParameters represents payload for Enabled method.
type enabledParameters struct {
Resource resource.Resource
InstrumentationScope instrumentation.Scope
Severity log.Severity
}
3 changes: 3 additions & 0 deletions sdk/log/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ require (
go.opentelemetry.io/otel v1.34.0
go.opentelemetry.io/otel/log v0.10.0
go.opentelemetry.io/otel/sdk v1.34.0
go.opentelemetry.io/otel/sdk/log/xlog v0.0.0-00010101000000-000000000000
Copy link
Member Author

@pellared pellared Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is interesting if the dependency is used only used in tests then it does not come up as an indirect dependency for modules using go.opentelemetry.io/otel/sdk/log 🎉

It looks like the Go module graph pruning has been improved at some point.

I think that it means that we could get rid of test modules 😉

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit strange. I've just tried that for otelhttp, and the SDK does come as an indirect dependency.

Copy link
Member Author

@pellared pellared Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I think we cannot rely on the Go modules pruning and I will get rid of this dependency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adressed 08ac146

Copy link
Member Author

@pellared pellared Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit strange. I've just tried that for otelhttp, and the SDK does come as an indirect dependency.

@dmathieu , I tried it once more and actually it does NOT come as an indirect dependency.

See: open-telemetry/opentelemetry-go-contrib#6763

I had to run make go-mod-tidy twice. Probably if we used crosslink tidylist then it would not be necessary

We can could consider revert 08ac146. But I am fine with the current approach as well. It looks cleaner. I propose to keep the current way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it looks like the difference is that you used a different package name. So that'd be the trick.

Copy link
Member Author

@pellared pellared Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it looks like the difference is that you used a different package name. So that'd be the trick.

I do not follow. The SDK is used in tests of both of otelhttp and otelhttp_test packages.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but when I tried it, I kept the otelhttp package name for the tests. I think that's why the dependency still showed up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tests still use it. Can you double check it? Make sure to run make go-mod-tidy more than once 😉

go.opentelemetry.io/otel/trace v1.34.0
)

Expand All @@ -29,6 +30,8 @@ replace go.opentelemetry.io/otel/trace => ../../trace

replace go.opentelemetry.io/otel/sdk => ../

replace go.opentelemetry.io/otel/sdk/log/xlog => ./xlog

replace go.opentelemetry.io/otel/log => ../../log

replace go.opentelemetry.io/otel => ../..
35 changes: 0 additions & 35 deletions sdk/log/internal/x/README.md

This file was deleted.

47 changes: 0 additions & 47 deletions sdk/log/internal/x/x.go

This file was deleted.

17 changes: 12 additions & 5 deletions sdk/log/logger.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ import (
"go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/log/embedded"
"go.opentelemetry.io/otel/sdk/instrumentation"
"go.opentelemetry.io/otel/sdk/log/internal/x"
"go.opentelemetry.io/otel/trace"
)

Expand Down Expand Up @@ -50,14 +49,22 @@ func (l *logger) Emit(ctx context.Context, r log.Record) {
// processed, true will be returned by default. A value of false will only be
// returned if it can be positively verified that no Processor will process.
func (l *logger) Enabled(ctx context.Context, param log.EnabledParameters) bool {
// If there are more Processors than FilterProcessors we cannot be sure
// that all Processors will drop the record. Therefore, return true.
p := enabledParameters{
Resource: *l.provider.resource,
InstrumentationScope: l.instrumentationScope,
Severity: param.Severity,
}

// If there are more Processors than FilterProcessors,
// which means not all Processors are FilterProcessors,
// we cannot be sure that all Processors will drop the record.
// Therefore, return true.
//
// If all Processors are FilterProcessors, check if any is enabled.
return len(l.provider.processors) > len(l.provider.fltrProcessors) || anyEnabled(ctx, param, l.provider.fltrProcessors)
return len(l.provider.processors) > len(l.provider.fltrProcessors) || anyEnabled(ctx, p, l.provider.fltrProcessors)
}

func anyEnabled(ctx context.Context, param log.EnabledParameters, fltrs []x.FilterProcessor) bool {
func anyEnabled(ctx context.Context, param enabledParameters, fltrs []filterProcessor) bool {
for _, f := range fltrs {
if f.Enabled(ctx, param) {
// At least one Processor will process the Record.
Expand Down
Loading