Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[reflect] sdk/log/xlog: Add FilterProcessor and EnabledParameters #6286

Closed
wants to merge 40 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
96a79ee
mv sdk/log/internal/x sdk/xlog
pellared Feb 5, 2025
6282e71
Introduce xlog module with FilterProcessor and EnabledParameters
pellared Feb 5, 2025
69ff66d
Update changelog
pellared Feb 5, 2025
bf61d44
Add trace context fields
pellared Feb 5, 2025
5b3c8ca
Test FilterProcessor functionality
pellared Feb 5, 2025
f24fc64
Update changelog
pellared Feb 5, 2025
fffc077
go mod tidy
pellared Feb 5, 2025
55adcc4
Merge branch 'main' into sdk/xlogs
pellared Feb 5, 2025
008cf4d
Merge branch 'main' into sdk/xlogs
pellared Feb 6, 2025
4df6669
Remove span context fields from EnabledParameters
pellared Feb 6, 2025
d31f329
Simplify doc comments
pellared Feb 6, 2025
fe5ae63
go mod tidy
pellared Feb 6, 2025
4e54827
Move sdk/xlog to sdk/log/xlog
pellared Feb 6, 2025
abe7833
Fix hyperlink
pellared Feb 6, 2025
d5817c6
Update README.md
pellared Feb 6, 2025
c23714b
Merge branch 'main' into sdk/xlogs
pellared Feb 6, 2025
cb91d27
Merge branch 'main' into sdk/xlogs
pellared Feb 7, 2025
71376d1
Update sdk/log/example_test.go
pellared Feb 7, 2025
e6b34ae
Update sdk/log/logger.go
pellared Feb 7, 2025
4ff41e1
Format comment
pellared Feb 7, 2025
628f916
Add missing period
pellared Feb 7, 2025
98d6811
Do not release xlog
pellared Feb 7, 2025
5cf1eaf
Improve Go doc comments
pellared Feb 7, 2025
9037771
sdk/log to use internal copy of interface instead of xlog
pellared Feb 7, 2025
3a07761
Cleanup go mod
pellared Feb 7, 2025
c56e9da
code snippet: TestReflectFilterProcessorPlayground
pellared Feb 7, 2025
91d84a9
Support xlog with duck typing using reflect
pellared Feb 7, 2025
685a417
Remove TestReflectFilterProcessorPlayground
pellared Feb 7, 2025
0547924
Fix typos
pellared Feb 7, 2025
7075931
Update CHANGELOG.md
pellared Feb 7, 2025
648e21f
Update CHANGELOG.md
pellared Feb 7, 2025
2941fb3
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 7, 2025
2a93c9f
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 10, 2025
9da340a
Apply suggestions from code review
pellared Feb 11, 2025
d55948e
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 11, 2025
08ac146
Remove usage of xlog in SDK
pellared Feb 11, 2025
9475ed8
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 12, 2025
25937e0
Add otel-experimental depguard rule
pellared Feb 12, 2025
e4d1ebf
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 12, 2025
2ac7f71
Merge branch 'main' into xlog-not-used-by-sdk
pellared Feb 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,12 @@ linters-settings:
deny:
- pkg: "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/internal"
desc: Do not use cross-module internal packages.
otel-experimental:
files:
- "!$test"
deny:
- pkg: "go.opentelemetry.io/otel/sdk/log/xlog"
desc: Do not use xlog in non-test code.
otel-internal:
files:
- "**/sdk/*.go"
Expand Down
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,13 @@ The next release will require at least [Go 1.23].
- Document the pitfalls of using `Resource` as a comparable type.
`Resource.Equal` and `Resource.Equivalent` should be used instead. (#6272)
- Support [Go 1.24]. (#6304)
- Make the initial release of `go.opentelemetry.io/otel/sdk/log/xlog`.
This new module contains experimental features of the OpenTelemetry Logs SDK.
It replaces `go.opentelemetry.io/otel/sdk/log/internal/x`.
This module is unstable and breaking changes may be introduced.
See our [versioning policy](VERSIONING.md) for more information about these stability guarantees. (#6286)
- Add `FilterProcessor` and `EnabledParameters` in `go.opentelemetry.io/otel/sdk/log/xlog` .
Compared to previous version it additionally gives the possibility to filter by resource and instrumentation scope. (#6286)

### Changed

Expand Down
12 changes: 8 additions & 4 deletions log/logger.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,13 @@ type Logger interface {
// Enabled returns whether the Logger emits for the given context and
// param.
//
// The passed param is likely to be a partial record with only the
// bridge-relevant information being provided (e.g a param with only the
// Severity set). If a Logger needs more information than is provided, it
// This is useful for users that want to know if a [Record]
// will be processed or dropped before they perform complex operations to
// construct the [Record].
//
// The passed param is likely to be a partial record information being
// provided (e.g a param with only the Severity set).
// If a Logger needs more information than is provided, it
// is said to be in an indeterminate state (see below).
//
// The returned value will be true when the Logger will emit for the
Expand All @@ -46,7 +50,7 @@ type Logger interface {
// exist (e.g. performance, correctness).
//
// The param should not be held by the implementation. A copy should be
// made if the record needs to be held after the call returns.
// made if the param needs to be held after the call returns.
//
// Implementations of this method need to be safe for a user to call
// concurrently.
Expand Down
2 changes: 1 addition & 1 deletion sdk/log/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ at a single endpoint their origin is decipherable.
See [go.opentelemetry.io/otel/log] for more information about
the OpenTelemetry Logs Bridge API.

See [go.opentelemetry.io/otel/sdk/log/internal/x] for information about the
See [go.opentelemetry.io/otel/sdk/log/xlog] for information about the
experimental features.
*/
package log // import "go.opentelemetry.io/otel/sdk/log"
64 changes: 0 additions & 64 deletions sdk/log/example_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import (
"context"
"fmt"
"strings"
"sync"

logapi "go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/log/global"
Expand Down Expand Up @@ -51,69 +50,6 @@ func Example() {
// slog.SetDefault(otelslog.NewLogger("my/pkg/name", otelslog.WithLoggerProvider(provider)))
}

// Use a processor that filters out records based on the provided context.
func ExampleProcessor_filtering() {
// Existing processor that emits telemetry.
var processor log.Processor = log.NewBatchProcessor(nil)

// Wrap the processor so that it ignores processing log records
// when a context deriving from WithIgnoreLogs is passed
// to the logging methods.
processor = &ContextFilterProcessor{Processor: processor}

// The created processor can then be registered with
// the OpenTelemetry Logs SDK using the WithProcessor option.
_ = log.NewLoggerProvider(
log.WithProcessor(processor),
)
}

type key struct{}

var ignoreLogsKey key

// WithIgnoreLogs returns a context which is used by [ContextFilterProcessor]
// to filter out log records.
func WithIgnoreLogs(ctx context.Context) context.Context {
return context.WithValue(ctx, ignoreLogsKey, true)
}

// ContextFilterProcessor filters out logs when a context deriving from
// [WithIgnoreLogs] is passed to its methods.
type ContextFilterProcessor struct {
log.Processor

lazyFilter sync.Once
// Use the experimental FilterProcessor interface
// (go.opentelemetry.io/otel/sdk/log/internal/x).
filter filter
}

type filter interface {
Enabled(ctx context.Context, param logapi.EnabledParameters) bool
}

func (p *ContextFilterProcessor) OnEmit(ctx context.Context, record *log.Record) error {
if ignoreLogs(ctx) {
return nil
}
return p.Processor.OnEmit(ctx, record)
}

func (p *ContextFilterProcessor) Enabled(ctx context.Context, param logapi.EnabledParameters) bool {
p.lazyFilter.Do(func() {
if f, ok := p.Processor.(filter); ok {
p.filter = f
}
})
return !ignoreLogs(ctx) && (p.filter == nil || p.filter.Enabled(ctx, param))
}

func ignoreLogs(ctx context.Context) bool {
_, ok := ctx.Value(ignoreLogsKey).(bool)
return ok
}

// Use a processor which redacts sensitive data from some attributes.
func ExampleProcessor_redact() {
// Existing processor that emits telemetry.
Expand Down
88 changes: 88 additions & 0 deletions sdk/log/filter_processor.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package log // import "go.opentelemetry.io/otel/sdk/log"

import (
"context"
"reflect"

"go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/sdk/instrumentation"
"go.opentelemetry.io/otel/sdk/resource"
)

// filterProcessor uses reflect to support [go.opentelemetry.io/otel/sdk/log/xlog.FilterProcessor]
// via duck typing.
type filterProcessor struct {
enabledFn reflect.Value
paramType reflect.Type
}

func asFilterProccessor(p any) (filterProcessor, bool) {
fp := reflect.ValueOf(p)
m := fp.MethodByName("Enabled")
if m == (reflect.Value{}) {
// No Enabled method.
return filterProcessor{}, false
}
mty := m.Type()
if mty.NumOut() != 1 {
// Should return one output parameter.
return filterProcessor{}, false
}
if reflect.Bool != mty.Out(0).Kind() {
// Should return bool.
return filterProcessor{}, false
}
if mty.NumIn() != 2 {
// Should have two input parameters.
return filterProcessor{}, false
}
if mty.In(0) != reflect.TypeFor[context.Context]() {
// Should have context.Context as first input parameter.
return filterProcessor{}, false
}
// Duck typing of EnabledParameters
pt := mty.In(1)
if res, ok := pt.FieldByName("Resource"); !ok || res.Type != reflect.TypeFor[resource.Resource]() {
// The second parameter should have Resource resource.Resource field.
return filterProcessor{}, false
}
if res, ok := pt.FieldByName("InstrumentationScope"); !ok || res.Type != reflect.TypeFor[instrumentation.Scope]() {
// The second parameter should have InstrumentationScope instrumentation.Scope field.
return filterProcessor{}, false
}
if res, ok := pt.FieldByName("Severity"); !ok || res.Type != reflect.TypeFor[log.Severity]() {
// The second parameter should have Severity log.Severity field.
return filterProcessor{}, false
}

return filterProcessor{
enabledFn: m,
paramType: pt,
}, true
}

func (f filterProcessor) Enabled(ctx context.Context, param enabledParameters) bool {
p := reflect.New(f.paramType).Elem()
p.FieldByName("Resource").Set(reflect.ValueOf(param.Resource))
p.FieldByName("InstrumentationScope").Set(reflect.ValueOf(param.InstrumentationScope))
p.FieldByName("Severity").Set(reflect.ValueOf(param.Severity))

ctxV := reflect.ValueOf(ctx)
if ctxV == (reflect.Value{}) {
// In order to not get panic: reflect: Call using zero Value argument.
ctxV = reflect.Zero(reflect.TypeFor[context.Context]())
}

ret := f.enabledFn.Call([]reflect.Value{ctxV, p})
return ret[0].Bool()
}

// enabledParameters represents payload for Enabled method.
type enabledParameters struct {
Resource resource.Resource
InstrumentationScope instrumentation.Scope
Severity log.Severity
}
35 changes: 0 additions & 35 deletions sdk/log/internal/x/README.md

This file was deleted.

47 changes: 0 additions & 47 deletions sdk/log/internal/x/x.go

This file was deleted.

17 changes: 12 additions & 5 deletions sdk/log/logger.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ import (
"go.opentelemetry.io/otel/log"
"go.opentelemetry.io/otel/log/embedded"
"go.opentelemetry.io/otel/sdk/instrumentation"
"go.opentelemetry.io/otel/sdk/log/internal/x"
"go.opentelemetry.io/otel/trace"
)

Expand Down Expand Up @@ -50,14 +49,22 @@ func (l *logger) Emit(ctx context.Context, r log.Record) {
// processed, true will be returned by default. A value of false will only be
// returned if it can be positively verified that no Processor will process.
func (l *logger) Enabled(ctx context.Context, param log.EnabledParameters) bool {
// If there are more Processors than FilterProcessors we cannot be sure
// that all Processors will drop the record. Therefore, return true.
p := enabledParameters{
Resource: *l.provider.resource,
InstrumentationScope: l.instrumentationScope,
Severity: param.Severity,
}

// If there are more Processors than FilterProcessors,
// which means not all Processors are FilterProcessors,
// we cannot be sure that all Processors will drop the record.
// Therefore, return true.
//
// If all Processors are FilterProcessors, check if any is enabled.
return len(l.provider.processors) > len(l.provider.fltrProcessors) || anyEnabled(ctx, param, l.provider.fltrProcessors)
return len(l.provider.processors) > len(l.provider.fltrProcessors) || anyEnabled(ctx, p, l.provider.fltrProcessors)
}

func anyEnabled(ctx context.Context, param log.EnabledParameters, fltrs []x.FilterProcessor) bool {
func anyEnabled(ctx context.Context, param enabledParameters, fltrs []filterProcessor) bool {
for _, f := range fltrs {
if f.Enabled(ctx, param) {
// At least one Processor will process the Record.
Expand Down
20 changes: 0 additions & 20 deletions sdk/log/logger_bench_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,23 +62,3 @@ func BenchmarkLoggerNewRecord(b *testing.B) {
})
})
}

func BenchmarkLoggerEnabled(b *testing.B) {
provider := NewLoggerProvider(
WithProcessor(newFltrProcessor("0", false)),
WithProcessor(newFltrProcessor("1", true)),
)
logger := provider.Logger(b.Name())
ctx := context.Background()
param := log.EnabledParameters{Severity: log.SeverityDebug}
var enabled bool

b.ReportAllocs()
b.ResetTimer()

for n := 0; n < b.N; n++ {
enabled = logger.Enabled(ctx, param)
}

_ = enabled
}
Loading