Releases: DataDog/dd-trace-go
v1.60.1
Summary
A few minor bug fixes in this release.
Changes
Fixes
- ddtrace/opentelemetry: add RWMutex to handle concurrent calls to setters by @darccio in #2521
- tracer: verify that hostname reporting is honored regardless of stats calculation by @katiehockman #2533
Full Changelog: v1.60.0...v1.60.1
v1.60.0
Summary
In this release, Application Performance Monitoring (APM) adds tracing instrumentation support for valyala/fasthttp. This release also fixes a bug with beta-level remote configuration of the tracer, ensuring that config deletion triggers reverting the configuration option to its original value. Startup logs will also be expanded to describe configured feature flags.
This release also adds Beta support for resource-based and tag-based sampling. This can be configured using the new trace SamplingRule
called TagResourceRule
or by setting "resource"
or "tags"
in the trace sampling rules JSON, e.g. DD_TRACE_SAMPLING_RULES=[{"service": "test.?", "resource": "ec2.*", "tags": {"aws.operation":"DescribeInstances"}, "sample_rate": 1.0}]
Application Security Management (ASM) adds support for Threat Monitoring on GraphQL operations, and trusted IPs can now be added to your ASM passlist.
The Profiler add the WithCustomProfilerLabelKeys
API, which enables using profiler labels
as attributes to filter flame graphs in the Continuous Profiler UI. The Profiler also improves execution trace coverage over time for apps which are deployed simultaneously across several instances, by randomizing when execution traces are collected. An execution trace is collected on average once every 15 minutes by default, but the time between execution traces is now randomized.
Changes
Application Performance Monitoring (APM)
- contrib/valyala/fasthttp.v1: add a fasthttp integration to ddtrace by @mtoffl01 in #2305
- contrib/google.golang.org/grpc: add error details to span tags by @eyasy1217 in #2228
- Send the
Datadog-Entity-ID
header, containing either the container-id or the cgroup inode if available by @AliDatadog in #2402 - Fix build on linux and do not send cgroup node inode if in host cgroup namespace by @AliDatadog in #2453
- ddtracer/tracer: fix flaky TestReportHealthMetrics on Windows by @darccio in #2439
- Configure parametric test to get the golang tracer in same way as system-tests by @robertomonteromiguel in #2462
- ddtrace/tracer: only listen on localhost in TestTransportResponse by @nsrip-dd in #2463
- ddtrace/tracer: clear global headers in remote config tests by @nsrip-dd in #2466
- ddtrace/tracer: adding sampling by resource and tags by @dianashevchenko in #2448
- tracer: fix example and add SamplingRule function tests by @katiehockman in #2493
- tracer/log: Add feature flags to startup logs by @ajgajg1134 in #2495
- tracer: handle rc deletion in dynamic config by @ahmed-mez in #2468
Application Security Management (ASM)
- appsec/graphql: add support for Threat Monitoring by @RomainMuller in #2309
- appsec: register
ASM_TRUSTED_IPS
capability to RC by @RomainMuller in #2460
Profiling
- profiler: randomize when execution traces are collected by @nsrip-dd in #2401
- profiler: deduplicate test profiler setup logic by @nsrip-dd in #2428
- Update gotraceui to v0.3.0. by @darccio in #2443
- profiler: support using custom profiler labels in our UI by @nsrip-dd in #2282
- profiler/internal/pprofutils: work around breaking pprof change by @nsrip-dd in #2515
General
- CI: Only mark PRs as stale by @ajgajg1134 in #2423
- readme: Add example for running just one integration container by @ajgajg1134 in #2429
- system-tests: add the uds-echo variant by @Julio-Guerra in #2438
- ci: setup smoke tests by @Julio-Guerra in #2441
- Revert "Update gotraceui to v0.3.0." by @darccio in #2457
- maitenance: new GitHub Triage Badger by @RomainMuller in #2451
- internal/datastreams: Improve performance by @piochelepiotr in #2455
- .github/workflows: adapting last v2 CI changes by @darccio in #2435
- ci: don't send issue body in triager by @nsrip-dd in #2472
- CODEOWNERS: Update for new gh team structure by @felixge in #2477
- Clean parametric tests pipeline by @robertomonteromiguel in #2485
- CODEOWNERS: Reduce go.mod/go.sum related pings by @felixge in #2484
- remoteconfig: fine grained locking by @Hellzy in #2458
- Don't ask for triage of issues from @DataDog/dd-trace-go-guild by @RomainMuller in #2489
- Use PAT to read team member list by @RomainMuller in #2491
- remoteconfig: remove mutex underscore prefixes by @Hellzy in #2497
- .github/workflows: generalize GH workflow for adding apm:ecosystems label by @katiehockman in #2496
- build(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 by @dependabot in #2445
- internal/telemetry: export Products field by @katiehockman in #2500
- internal/telemetry: remove DetectLibDl by @katiehockman in #2517
- internal/telemetry: do not send empty container and entity headers by @ahmed-mez in #2506
- go.mod: upgrade purego v0.5.0 => upgrade purego v0.5.2 by @eliottness in #2524
New Contributors
- @AliDatadog made their first contribution in #2402
- @robertomonteromiguel made their first contribution in #2462
- @eyasy1217 made their first contribution in #2228
Full Changelog: v1.59.0...v1.60.0
v1.59.1
Patch Release Notes
Application Security Management (ASM)
- Upgrade github.com/ebitengine/purego: v0.5.0 -> v0.5.2 to forward ebitengine/purego#189 fix to upstream
v1.59.1
:
dlopen: unhandled relocation for purego_dlopen (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
dlsym: unhandled relocation for purego_dlsym (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
dlerror: unhandled relocation for purego_dlerror (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
dlclose: unhandled relocation for purego_dlclose (type 46 (SDYNIMPORT) rtype 7 (R_CALL))
This bug can appear for all users of ASM (using the build tag appsec
) starting from v1.53.0
and for all users of dd-trace-go starting from v1.59.0
.
v1.59.0
Highlights
Important Information
dd-trace-go
requires libdl.so.2
1 and libm.so.6
to be present in the deployment environment (except Windows, where Application Security Management is not supported). This should be the case for the vast majority of environments (e.g: Redhat, Debian, Alpine, Amazon Linux, Ubuntu) but might not be the case in some minimal-footprint environments (e.g: BusyBox, docker images from scratch
). In situations when these requirements aren't met, you might see errors similar to (the exact error may be slightly different depending on the platform):
- If
ld.so
is missing (the environment is unable to load dynamic executables) or is not the expected flavor (the binary was built on one platform, and runs on another which uses a different interpreter):exec /path/to/binary: no such file or directory
- When one of the required shared libraries is missing (
libdl.so.2
in this example):/path/to/binary: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory
If this happens, please create a GitHub issue so we can reconsider this new default requirement. In the meantime, you can work around the problem by adding the datadog.no_waf
build tag (go build -tags='datadog.no_waf' ...
), which completely disables all Application Security Management features and removes these new requirements.
Application Security Management (ASM) Remote Activation
Application Security Management (ASM) can now be remotely activated by APM Tracing users via Datadog Remote Configuration, granted that the application was built either:
- with CGO enabled;
- with
CGO_ENABLED=0
and with theappsec
build tag (this setup already requiredlibdl.so.2
1 andlibm.so.6
to be present in the deployment environment on previous releases — this remains true).
Remote activation can be performed from different places in Datadog UI, such as ASM's Service Setup or APM's Service Catalog (hovering the ASM Status column).
ASM features can still be enabled locally using the DD_APPSEC_ENABLED=1
environment variable.
Setting the datadog.no_waf
build tag completely disables all ASM features, removing the ability for local as well as remote activation.
Application Performance Monitoring (APM)
This release includes a fix to several library integrations which could have previously caused data races related to start options. This was fixed in database/sql
, gin-gonic/gin
, go-chi/chi.v5
, go-chi/chi
, google.golang.org/grpc.v12
, google.golang.org/grpc
, gorilla/mux
, julienschmidt/httprouter
, k8s.io/client-go/kubernetes
, labstack/echo.v4
, labstack/echo
, net/http
, and urfave/negroni
. We recommend you update to this version if you are using any of these integrations.
Beta: In-app APM library configuration of trace sampling rate, HTTP header tags and custom tags.
This feature has a known bug: deleting the configuration entry in-app won't reset the configuration locally, this will be fixed in the next version of dd-trace-go (v1.60.0).
What's Changed
Application Security Management (ASM)
- appsec: remove the "appsec" build tag requirement by @RomainMuller in #2354
- go.mod: go-libddwaf v2.2.2 including major perf improvements and bug fixes by @eliottness in #2417
- appsec/api-security: http request schema collection and sensitive data scanning by @Hellzy in #2381
- appsec: support server.response.headers.no_cookies WAF address by @eliottness in #2347
Application Performance Monitoring (APM)
- contrib/google.golang.org/grpc: improve the memory efficiency of threats detection for grpc by @RomainMuller in #2338
- contrib: header_tags support on julienschmidt/httprouter by @mtoffl01 in #2331
- contrib/kafka: take env variable into account to enable DSM by @vandonr in #2353
- contrib/aws/{aws-sdk-go/aws, aws-sdk-go-v2/aws}: add context example by @mackjmr in #1504
- contrib/dimfeld/httptreemux.v5: parameterize redirects due to trailing slash by @laughingman-hass in #2332
- contrib/google.golang.org/grpc: add hostname tag by @rarguelloF in #2361
- contrib/database/sql: prevent DBM propagation full mode with incompatible dbs by @rarguelloF in #2328
- contrib: fix span start option races by @eliottness in #2418
- tracer: Fix race in spanContext.setSamplingPriority by @evanj in #2271
- tracer: report config-change telemetry in dynamic config by @ahmed-mez in #2350
- tracer: check for (service,env) matching in dynamic config by @ahmed-mez in #2365
- ddtrace/opentelemetry,opentracing: fixed the format of telemetry tags by @dianashevchenko in #2367
- tracer: report rc capabilities for dynamic config by @ahmed-mez in #2369
- tracer: configure global tags via remote-config by @ahmed-mez in #2378
- tracer: support dot notation for tags with array values by @katiehockman in #2253
- build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 by @dependabot in #2373
- tracer: improve debug message for propagating tag length limit reached by @katiehockman in #2405
General
- {telemetry,remoteconfig}: support fraction of second intervals by @ahmed-mez in #2364
- remoteconfig: add Subscribe function by @ahmed-mez in #2380
- remoteconfig: fix products reporting by @ahmed-mez in #2384
New Contributors
- @RomainMuller made their first contribution in #2338
- @vandonr made their first contribution in #2353
- @laughingman-hass made their first contribution in #2332
Full Changelog: v1.58.0...v.1.59.0
v1.58.1
Patch Release Notes
Application Performance Monitoring (APM)
- contrib/dimfeld/httptreemux.v5: parameterize redirects due to trailing slash by @laughingman-hass in #2332
- contrib: fix span start option races by @katiehockman in #2418
- tracer: Fix race in spanContext.setSamplingPriority by @evanj in #2271
- ddtrace/opentelemetry,opentracing: fixed the format of telemetry tags by @dianashevchenko in #2367
v1.59.0-rc.3
What's Changed
This release includes a fix to several library integrations which could have previously caused data races related to start options. This was fixed in database/sql
, gin-gonic/gin
, go-chi/chi.v5
, go-chi/chi
, google.golang.org/grpc.v12
, google.golang.org/grpc
, gorilla/mux
, julienschmidt/httprouter
, k8s.io/client-go/kubernetes
, labstack/echo.v4
, labstack/echo
, net/http
, and urfave/negroni
. We recommend you update to this version if you are using any of these integrations.
Application Security Management (ASM) now can be remotely activated by APM Tracing users with Datadog Remote Configuration, without requiring the appsec
build tag or DD_APPSEC_ENABLED
environment variable. This can be achieved from different places in our UI such as ASM's Service Setup or APM's Service Catalog (hovering the ASM Status column). Non-remote-configuration users still need to deploy their services with the DD_APPSEC_ENABLED
opt-in environment variable to enable ASM. Users of CGO_ENABLED=0
will still have to rely on the appsec
build tag for now, as it results into a dependency to libdl.so.2
to have on your deployment environment in this precise case, but not when CGO_ENABLED=1
.
This release also includes the latest ASM private beta feature, API Security (public documentation still in progress).
Beta: In-app APM library configuration of trace sampling rate, HTTP header tags and custom tags.
This feature has a known bug: deleting the configuration entry in-app won't reset the configuration locally, this will be fixed in the next version of dd-trace-go (v1.60.0).
Application Security Management (ASM)
- appsec: remove the "appsec" build tag requirement by @RomainMuller in #2354
- go.mod: go-libddwaf v2.2.2 including major perf improvements and bug fixes by @eliottness in #2417
- appsec/api-security: http request schema collection and sensitive data scanning by @Hellzy in #2381
- appsec: support server.response.headers.no_cookies WAF address by @eliottness in #2347
Application Performance Monitoring (APM)
- contrib/google.golang.org/grpc: improve the memory efficiency of threats detection for grpc by @RomainMuller in #2338
- contrib: header_tags support on julienschmidt/httprouter by @mtoffl01 in #2331
- contrib/kafka: take env variable into account to enable DSM by @vandonr in #2353
- contrib/aws/{aws-sdk-go/aws, aws-sdk-go-v2/aws}: add context example by @mackjmr in #1504
- contrib/dimfeld/httptreemux.v5: parameterize redirects due to trailing slash by @laughingman-hass in #2332
- contrib/google.golang.org/grpc: add hostname tag by @rarguelloF in #2361
- contrib/database/sql: prevent DBM propagation full mode with incompatible dbs by @rarguelloF in #2328
- contrib: fix span start option races by @eliottness in #2418
- tracer: Fix race in spanContext.setSamplingPriority by @evanj in #2271
- tracer: report config-change telemetry in dynamic config by @ahmed-mez in #2350
- tracer: check for (service,env) matching in dynamic config by @ahmed-mez in #2365
- ddtrace/opentelemetry,opentracing: fixed the format of telemetry tags by @dianashevchenko in #2367
- tracer: report rc capabilities for dynamic config by @ahmed-mez in #2369
- tracer: configure global tags via remote-config by @ahmed-mez in #2378
- tracer: support dot notation for tags with array values by @katiehockman in #2253
- build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 by @dependabot in #2373
- tracer: improve debug message for propagating tag length limit reached by @katiehockman in #2405
General
- {telemetry,remoteconfig}: support fraction of second intervals by @ahmed-mez in #2364
- remoteconfig: add Subscribe function by @ahmed-mez in #2380
- remoteconfig: fix products reporting by @ahmed-mez in #2384
New Contributors
- @RomainMuller made their first contribution in #2338
- @vandonr made their first contribution in #2353
- @laughingman-hass made their first contribution in #2332
Full Changelog: v1.58.0...v.1.59.0
v1.58.0
Summary
In this release, Application Performance Monitoring (APM) makes several changes that will improve the OpenTelemetry experience:
- The tracer will now generate 128-bit long trace IDs by default. To disable this, set
DD_TRACE_128_BIT_TRACEID_GENERATION_ENABLED
tofalse
. - For a distributed trace, the entire tracestate header will be propagated to avoid losing context data from other vendors. This should improve interoperability and migrations when using multiple trace context propagation styles across systems.
- There are a number of improvements to the OpenTelemetry API provided under
ddtrace/opentelemetry
:- BREAKING CHANGE: The operation name of the span will no longer be derived from the OTel span's name, and will now be crafted using OTel semantics. The OTel span's name will only be used as the resource name. This should provide a more appropriate span name and improve the UX. To explicitly set the span name when using the OTel API, you can set the
operation.name
attribute on the span. - The OTel API will now recognize reserved attributes, and map those to the corresponding properties of a Datadog span. The full list of attributes are here. This is to better match the OTLP behavior.
- We have embedded a no-op struct in the OTel API implementation so that the OTel API now supports v1.20+, v0.43.0+ of the upstream Go OpenTelemetry API library.
- BREAKING CHANGE: The operation name of the span will no longer be derived from the OTel span's name, and will now be crafted using OTel semantics. The OTel span's name will only be used as the resource name. This should provide a more appropriate span name and improve the UX. To explicitly set the span name when using the OTel API, you can set the
Changes
Application Performance Monitoring (APM)
- contrib/globalsign/mgo: err always nil. by @tttoad in #2269
- contrib/google.golang.org/grpc: isolate tests using an independent rig instance instead of a shared one by @darccio in #2278
- tracer: pre-size payload buffer to optimize memory allocations by @ahmed-mez in #2319
- ddtrace/opentelemetry: added support for special attributes mapping by @dianashevchenko in #2333
- ddtrace/opentelemetry: Introduced OTel name remapping by @dianashevchenko in #2337
- tracer: enable 128-bit TraceID generation by default by @katiehockman in #2335
- tracer: always propagate the tracestate header by @katiehockman in #2339
- ddtrace/opentelemetry: Fixed setting a wrong span field (span.kind) by @dianashevchenko in #2334
- Export SamplingPriority method on spanContext [SVLS-3934] by @purple4reina in #2291
- contrib/labstack/echo: Custom tags by @rafaeljusto in #2340
Application Security Management (ASM)
This release includes important optimizations of ASM Threats Detection by avoiding Go string copies, leading to major savings of memory allocations (note that this internal change is hidden inside the gRPC improvements below but apply to HTTP too which also relies on the same internals).
- contrib/google.golang.org/grpc: improve the memory efficiency of threats detection for gRPC by @RomainMuller in #2338
- appsec: upgrade security event rules to v1.8.0 by @eliottness in #2312
Datastreams Monitoring
- internal/datastreams: fix memory leak by @piochelepiotr in #2266
Profiler
New Contributors
- @tttoad made their first contribution in #2269
- @RomainMuller made their first contribution in #2338
Full Changelog: v1.57.0...v1.58.0
v1.57.0
Summary
In this release, Application Performance Monitoring (APM) will now correctly honor the trace context propagation style precedence, where previously W3C tracecontext
style would always take precedence even when a user specifies it later in the chain. This is a bug fix to align with expected behavior, but could potentially be a breaking change for customers relying on W3C trace context propagation.
Other APM changes include deprecation of the old Gorm libraries (gopkg.in/jinzhu/gorm.v1
and jinzhu/gorm
), and fixes for net/http
contrib to now honor WithHeaderTags
in WrapHandler
, the http.route
tag to be correctly collected in all relevant libraries.
Changes
Application Performance Monitoring (APM)
- contrib: implement http.route in missing contribs by @rarguelloF in #2234
- contrib/(github and gopkg)/gorm: Mark as deprecated by @ajgajg1134 in #2304
- go.mod: bump github.com/DataDog/datadog-agent/pkg/obfuscate to v0.48.0 by @ahmed-mez in #2256
- go.mod: bump otelhttp to v0.44.0 by @ahmed-mez in #2274
- [deps] update datadog-agent version to stable by @ufoot in #2308
- build(deps): bump google.golang.org/grpc from 1.57.0 to 1.57.1 by @dependabot in #2294
- tracer.startTestTracer: Call SetServiceName("") when stopping by @evanj in #2260
- tracer: call SetServiceName("") from tests to reset globals by @evanj in #2270
- ddtrace/tracer: encode span IDs in execution traces efficiently by @nsrip-dd in #2268
- Add
dd-trace-go
macrobenchmark to CI pipeline by @igoragoli in #2285 - remoteconfig: make rc client a singleton by @ahmed-mez in #2297
- contrib/net/http: honor WithHeaderTags in WrapHandler by @sudolibre in #2288
Application Security Management (ASM)
- appsec: update security event rules to 1.8.0 by @eliottness in #2292
Profiler
- profiler: remove PprofDiff by @nsrip-dd in #2267
- profiler: simplify startup logging by @nsrip-dd in #2283
- profiler: log cpuProfileRate when starting a trace by @felixge in #2030
- telemetry: fix profiler not passing config and align data model with spec AIT-7935 by @ahmed-mez in #2248
General
- ci: use older Windows for Go 1.19 tests by @nsrip-dd in #2250
- workflows: automatically label ecosystems issues and PRs by @katiehockman in #2254
- build(deps): bump golang.org/x/net from 0.14.0 to 0.17.0 by @dependabot in #2264
- source-code-integration: Remove credentials from git repository url. by @lraucy in #2296
Fixes
Application Performance Monitoring (APM)
- tracer: honor tracecontext propagation style precedence by @purple4reina in #2252
- tracer TestUserMonitoring: Wait for goroutines before test completes by @evanj in #2263
- tracer TestWithHeaderTags: Clear header tags to work with -count=2 by @evanj in #2262
- tracer: TestAgentIntegration: Clean up state for -count=2 by @evanj in #2261
- tracer: Fix TestMalformedTID so it passes on its own by @evanj in #2257
General
New Contributors
- @igoragoli made their first contribution in #2285
- @lraucy made their first contribution in #2296
- @sudolibre made their first contribution in #2288
Full Changelog: v1.56.1...v1.57.0
v1.56.1
Summary
A few minor bug fixes in this release: support for gofiber/fiber
up to v2.50.0, and a fix for the OpenTelemetry API.
Changes
Fixes
- [v1.56.1] opentelemetry: finish a span with an error if one is provided by @darccio in #2281
- [v1.56.1] contrib/gofiber/fiber: upgrade to v2.50.0 and fix breaking change by @darccio in #2280
Full Changelog: v1.56.0...v1.56.1
v1.56.0
Summary
In this release, Continuous Profiler enables execution tracing by default for Go 1.21+. The execution tracer is the data source powering our new timeline view for analyzing latency and identifying bottlenecks. The profiler additionally adds a new metric for number of active goroutines, and disables profiling in AWS Lambda.
Additionally, Data Stream Monitoring (DSM) has a new version tag to track deployments.
Finally, some contrib library improvements include support for translating custom errors in labstack/echo.v4
, and trace context propagation for libraries built on fasthttp
.
This release also includes several bug fixes and documentation improvements:
- Downgrade google.golang.org/protobuf and golang.org/x/exp dependencies.
- Clarified service defaulting.
- Clarified our policy for contrib dependency version upgrades.
Changes
Application Performance Monitoring (APM)
- contrib/labstack/echo.v4: add WithErrorTranslator Option by @mattscamp in #2169
Profiling
- Disable profiling when run in AWS Lambda. by @purple4reina in #2216
- profiler: add go_num_goroutine metric by @felixge in #2217
- profiler: enable execution traces by default for go1.21+ by @felixge in #2226
General
- internal/version: bump to v1.56.0-dev by @nsrip-dd in #2210
- all: downgrade google.golang.org/protobuf and golang.org/x/exp dependencies by @nsrip-dd in #2212
- datastreams: Add version tag by @piochelepiotr in #2211
- security: Add notes for contrib dependencies with instructions by @ajgajg1134 in #2208
- build(deps): bump github.com/gofiber/fiber/v2 from 2.48.0 to 2.49.2 by @dependabot in #2219
- contrib: add validation tests using test-agent by @rarguelloF in #2047
- ddtrace/mocktracer: use lock in String method by @phoenix2x in #2229
- fasthttptrace: Add trace context propagation support for libraries built on fasthttp by @mtoffl01 in #2218
- AIT-8312 tracer/option: document service defaulting by @ahmed-mez in #2245
New Contributors
- @mattscamp made their first contribution in #2169
- @mimfgg made their first contribution in #2221
Full Changelog: v1.55.0...v1.56.0