Skip to content

chore(debugging): in-product enablement #13386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

christophe-papazian
Copy link
Contributor

@christophe-papazian christophe-papazian commented May 12, 2025

We add in-product enablement support for Dynamic Instrumentation.## Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

Copy link
Contributor

CODEOWNERS have been resolved as:

ddtrace/debugging/_import.py                                            @DataDog/debugger-python
ddtrace/appsec/_ddwaf/ddwaf_types.py                                    @DataDog/asm-python
ddtrace/appsec/_processor.py                                            @DataDog/asm-python
ddtrace/bootstrap/sitecustomize.py                                      @DataDog/apm-core-python
ddtrace/debugging/_debugger.py                                          @DataDog/debugger-python
ddtrace/debugging/_products/dynamic_instrumentation.py                  @DataDog/debugger-python
ddtrace/internal/README.md                                              @DataDog/python-guild @DataDog/apm-core-python
ddtrace/internal/_unpatched.py                                          @DataDog/python-guild
ddtrace/internal/products.py                                            @DataDog/apm-core-python
tests/debugging/exploration/debugger.py                                 @DataDog/debugger-python
tests/debugging/mocking.py                                              @DataDog/debugger-python

Copy link
Contributor

github-actions bot commented May 12, 2025

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 270 ± 3 ms.

The average import time from base is: 236 ± 2 ms.

The import time difference between this PR and base is: 33.4 ± 0.1 ms.

Import time breakdown

The following import paths have appeared:

ddtrace.auto 6.831 ms (2.53%)
ddtrace 5.213 ms (1.93%)
ddtrace.internal._unpatched 5.213 ms (1.93%)
subprocess 5.213 ms (1.93%)
selectors 1.569 ms (0.58%)
math 0.199 ms (0.07%)
collections.abc 0.184 ms (0.07%)
locale 0.828 ms (0.31%)
_locale 0.076 ms (0.03%)
signal 0.675 ms (0.25%)
contextlib 0.576 ms (0.21%)
warnings 0.292 ms (0.11%)
fcntl 0.182 ms (0.07%)
select 0.170 ms (0.06%)
_posixsubprocess 0.152 ms (0.06%)
msvcrt 0.074 ms (0.03%)
errno 0.066 ms (0.02%)
ddtrace.bootstrap.sitecustomize 1.618 ms (0.60%)
ddtrace.bootstrap.preload 1.618 ms (0.60%)
ddtrace.debugging._import 1.618 ms (0.60%)
ddtrace.debugging._function.discovery 1.279 ms (0.47%)
ddtrace.debugging._function 0.259 ms (0.10%)

The following import paths have disappeared:

ddtrace.auto 5.544 ms (2.06%)
ddtrace 5.544 ms (2.06%)
ddtrace._logger 4.956 ms (1.84%)
ddtrace.internal.telemetry 3.251 ms (1.21%)
ddtrace.internal.telemetry.writer 1.587 ms (0.59%)
http.client 0.932 ms (0.35%)
email.parser 0.932 ms (0.35%)
email.feedparser 0.932 ms (0.35%)
email._policybase 0.932 ms (0.35%)
email.utils 0.932 ms (0.35%)
email._parseaddr 0.932 ms (0.35%)
calendar 0.932 ms (0.35%)
locale 0.932 ms (0.35%)
_locale 0.083 ms (0.03%)
ddtrace.internal.atexit 0.655 ms (0.24%)
signal 0.655 ms (0.24%)
ddtrace.settings._agent 1.397 ms (0.52%)
socket 1.397 ms (0.52%)
selectors 1.397 ms (0.52%)
select 0.188 ms (0.07%)
ddtrace.internal.utils.formats 0.266 ms (0.10%)
ddtrace.internal.compat 0.266 ms (0.10%)
pathlib 0.266 ms (0.10%)
urllib.parse 0.212 ms (0.08%)
math 0.212 ms (0.08%)
errno 0.055 ms (0.02%)
logging 1.705 ms (0.63%)
traceback 1.404 ms (0.52%)
contextlib 1.236 ms (0.46%)
collections.abc 0.167 ms (0.06%)
warnings 0.301 ms (0.11%)
ddtrace.settings._config 0.588 ms (0.22%)
ddtrace.internal.gitmetadata 0.588 ms (0.22%)
ddtrace.ext.ci 0.588 ms (0.22%)
ddtrace.ext.git 0.588 ms (0.22%)
subprocess 0.588 ms (0.22%)
fcntl 0.304 ms (0.11%)
_posixsubprocess 0.284 ms (0.11%)

The following import paths have grown:

ddtrace.auto 38.811 ms (14.39%)
ddtrace.bootstrap.sitecustomize 34.867 ms (12.93%)
ddtrace.bootstrap.preload 31.643 ms (11.73%)
ddtrace.internal.core.crashtracking 29.682 ms (11.00%)
ddtrace.settings.crashtracker 0.758 ms (0.28%)
ddtrace.internal.remoteconfig.client 0.621 ms (0.23%)
ddtrace.internal.remoteconfig.constants 0.621 ms (0.23%)
ddtrace.internal.products 0.307 ms (0.11%)
importlib.metadata 0.307 ms (0.11%)
importlib.abc 0.150 ms (0.06%)
importlib.resources 0.072 ms (0.03%)
csv 0.091 ms (0.03%)
zipfile 0.065 ms (0.02%)
zipfile._path 0.065 ms (0.02%)
ddtrace.settings.profiling 0.130 ms (0.05%)
ddtrace.internal.datadog.profiling.ddup 0.068 ms (0.03%)
ddtrace.internal.datadog.profiling.ddup._ddup 0.068 ms (0.03%)
ddtrace.vendor.psutil 0.063 ms (0.02%)
ddtrace.vendor.psutil._pslinux 0.063 ms (0.02%)
glob 0.063 ms (0.02%)
multiprocessing 0.082 ms (0.03%)
multiprocessing.context 0.082 ms (0.03%)
multiprocessing.process 0.082 ms (0.03%)
ddtrace.settings.dynamic_instrumentation 0.063 ms (0.02%)
ddtrace._trace.trace_handlers 2.031 ms (0.75%)
ddtrace.contrib.trace_utils 0.893 ms (0.33%)
ddtrace.contrib.internal.trace_utils 0.400 ms (0.15%)
ddtrace.contrib.internal.trace_utils_base 0.197 ms (0.07%)
ddtrace.ext.user 0.096 ms (0.04%)
ddtrace.internal.utils.wrappers 0.095 ms (0.04%)
ddtrace.contrib.internal.redis_utils 0.294 ms (0.11%)
ddtrace.contrib.internal 0.100 ms (0.04%)
ddtrace.ext.redis 0.095 ms (0.04%)
ddtrace.contrib.internal.trace_utils_async 0.097 ms (0.04%)
ddtrace._trace._inferred_proxy 0.599 ms (0.22%)
ddtrace.propagation.http 0.509 ms (0.19%)
ddtrace.internal._tagset 0.106 ms (0.04%)
ddtrace.propagation 0.106 ms (0.04%)
ddtrace.propagation._utils 0.097 ms (0.04%)
ddtrace.contrib.internal.botocore.constants 0.214 ms (0.08%)
ddtrace.contrib.internal.botocore 0.109 ms (0.04%)
ddtrace.ext.db 0.097 ms (0.04%)
ddtrace._trace.utils 0.035 ms (0.01%)
ddtrace.appsec._common_module_patches 0.563 ms (0.21%)
ddtrace.appsec._asm_request_context 0.289 ms (0.11%)
ddtrace.appsec._utils 0.096 ms (0.04%)
ddtrace.appsec._metrics 0.181 ms (0.07%)
ddtrace.appsec._deduplications 0.094 ms (0.03%)
sitecustomize 0.118 ms (0.04%)
ddtrace.contrib.internal.subprocess.constants 0.104 ms (0.04%)
ddtrace 3.944 ms (1.46%)
ddtrace._logger 2.464 ms (0.91%)
ddtrace.internal.telemetry 2.402 ms (0.89%)
ddtrace.internal.telemetry.writer 1.507 ms (0.56%)
ddtrace.internal.utils.version 0.988 ms (0.37%)
ddtrace.vendor.packaging.version 0.988 ms (0.37%)
ddtrace.vendor.packaging 0.988 ms (0.37%)
ddtrace.vendor 0.988 ms (0.37%)
ddtrace.internal.module 0.988 ms (0.37%)
ddtrace.internal.wrapping.context 0.988 ms (0.37%)
ddtrace.internal.wrapping 0.988 ms (0.37%)
bytecode 0.664 ms (0.25%)
bytecode.bytecode 0.664 ms (0.25%)
ddtrace.internal.assembly 0.324 ms (0.12%)
http.client 0.205 ms (0.08%)
email.parser 0.121 ms (0.04%)
email.feedparser 0.121 ms (0.04%)
email._policybase 0.121 ms (0.04%)
email.utils 0.065 ms (0.02%)
email._parseaddr 0.065 ms (0.02%)
calendar 0.065 ms (0.02%)
email.header 0.056 ms (0.02%)
ddtrace.internal.forksafe 0.104 ms (0.04%)
wrapt 0.104 ms (0.04%)
wrapt.__wrapt__ 0.104 ms (0.04%)
wrapt.wrappers 0.104 ms (0.04%)
ddtrace.internal.telemetry.data 0.073 ms (0.03%)
ddtrace.internal.runtime.container 0.073 ms (0.03%)
ddtrace.internal.encoding 0.069 ms (0.03%)
ddtrace.internal._encoding 0.069 ms (0.03%)
ddtrace.internal.runtime 0.068 ms (0.03%)
uuid 0.068 ms (0.03%)
platform 0.068 ms (0.03%)
ddtrace.settings._agent 0.721 ms (0.27%)
ddtrace.settings 0.567 ms (0.21%)
ddtrace.settings.http 0.567 ms (0.21%)
ddtrace.internal.utils.http 0.567 ms (0.21%)
email.encoders 0.567 ms (0.21%)
base64 0.567 ms (0.21%)
socket 0.068 ms (0.03%)
ddtrace.internal.utils.formats 0.174 ms (0.06%)
ddtrace.internal.compat 0.174 ms (0.06%)
pathlib 0.051 ms (0.02%)
logging 0.062 ms (0.02%)
weakref 0.062 ms (0.02%)
ddtrace._monkey 0.720 ms (0.27%)
ddtrace.appsec._listeners 0.720 ms (0.27%)
ddtrace.internal.core 0.720 ms (0.27%)
ddtrace.vendor.debtcollector 0.720 ms (0.27%)
ddtrace.vendor.debtcollector.removals 0.720 ms (0.27%)
ddtrace.trace 0.519 ms (0.19%)
ddtrace._trace.filters 0.345 ms (0.13%)
ddtrace._trace.processor 0.345 ms (0.13%)
ddtrace._trace.sampler 0.175 ms (0.06%)
ddtrace._trace.span 0.175 ms (0.06%)
ddtrace._trace._span_pointer 0.101 ms (0.04%)
ddtrace._trace.telemetry 0.101 ms (0.04%)
ddtrace.internal.dogstatsd 0.081 ms (0.03%)
ddtrace.vendor.dogstatsd 0.081 ms (0.03%)
ddtrace.vendor.dogstatsd.base 0.081 ms (0.03%)
ddtrace._trace.tracer 0.100 ms (0.04%)
ddtrace._trace.context 0.074 ms (0.03%)
ddtrace._trace._span_link 0.074 ms (0.03%)
ddtrace.settings._config 0.240 ms (0.09%)
ddtrace.internal.gitmetadata 0.161 ms (0.06%)
ddtrace.ext.ci 0.161 ms (0.06%)
ddtrace.ext.git 0.161 ms (0.06%)
shutil 0.060 ms (0.02%)
ddtrace.internal._file_queue 0.079 ms (0.03%)
secrets 0.079 ms (0.03%)
hmac 0.079 ms (0.03%)
_hashlib 0.079 ms (0.03%)

The following import paths have shrunk:

ddtrace.auto 6.577 ms (2.44%)
ddtrace 4.012 ms (1.49%)
ddtrace._logger 1.788 ms (0.66%)
ddtrace.internal.telemetry 1.719 ms (0.64%)
ddtrace.internal.telemetry.writer 1.421 ms (0.53%)
ddtrace.internal.utils.version 0.965 ms (0.36%)
ddtrace.vendor.packaging.version 0.965 ms (0.36%)
ddtrace.vendor.packaging 0.965 ms (0.36%)
ddtrace.vendor 0.694 ms (0.26%)
ddtrace.internal.module 0.694 ms (0.26%)
http.client 0.243 ms (0.09%)
email.parser 0.158 ms (0.06%)
email.feedparser 0.158 ms (0.06%)
email._policybase 0.089 ms (0.03%)
email.utils 0.089 ms (0.03%)
random 0.089 ms (0.03%)
bisect 0.014 ms (0.01%)
email.errors 0.069 ms (0.03%)
http 0.085 ms (0.03%)
ddtrace.internal.forksafe 0.075 ms (0.03%)
wrapt 0.075 ms (0.03%)
wrapt.decorators 0.075 ms (0.03%)
ddtrace.internal.runtime 0.074 ms (0.03%)
uuid 0.074 ms (0.03%)
ddtrace.internal.telemetry.data 0.063 ms (0.02%)
ddtrace.internal.packages 0.063 ms (0.02%)
_sysconfigdata__linux_x86_64-linux-gnu 0.063 ms (0.02%)
ddtrace.settings._agent 0.207 ms (0.08%)
ddtrace.settings 0.126 ms (0.05%)
ddtrace.settings.http 0.126 ms (0.05%)
ddtrace.internal.utils.cache 0.126 ms (0.05%)
inspect 0.126 ms (0.05%)
ast 0.126 ms (0.05%)
_ast 0.126 ms (0.05%)
ddtrace.settings._core 0.081 ms (0.03%)
envier 0.081 ms (0.03%)
envier.env 0.081 ms (0.03%)
ddtrace.internal.utils.formats 0.091 ms (0.03%)
ddtrace.internal.compat 0.091 ms (0.03%)
pathlib 0.091 ms (0.03%)
urllib.parse 0.091 ms (0.03%)
logging 0.070 ms (0.03%)
string 0.070 ms (0.03%)
ddtrace.settings._config 0.976 ms (0.36%)
ddtrace.internal.gitmetadata 0.175 ms (0.06%)
ddtrace.ext.ci 0.175 ms (0.06%)
ddtrace.ext.git 0.071 ms (0.03%)
shutil 0.071 ms (0.03%)
lzma 0.071 ms (0.03%)
ddtrace.internal._file_queue 0.093 ms (0.03%)
secrets 0.093 ms (0.03%)
hmac 0.093 ms (0.03%)
ddtrace.trace 0.482 ms (0.18%)
ddtrace._trace.filters 0.370 ms (0.14%)
ddtrace._trace.processor 0.370 ms (0.14%)
ddtrace._trace.sampler 0.175 ms (0.06%)
ddtrace._trace.span 0.074 ms (0.03%)
ddtrace.internal.sampling 0.074 ms (0.03%)
ddtrace._trace.sampling_rule 0.074 ms (0.03%)
ddtrace.internal.glob_matching 0.074 ms (0.03%)
ddtrace.internal.dogstatsd 0.102 ms (0.04%)
ddtrace.vendor.dogstatsd 0.102 ms (0.04%)
ddtrace.vendor.dogstatsd.base 0.102 ms (0.04%)
queue 0.102 ms (0.04%)
heapq 0.102 ms (0.04%)
_heapq 0.102 ms (0.04%)
ddtrace.internal.writer 0.093 ms (0.03%)
ddtrace.internal.writer.writer 0.093 ms (0.03%)
gzip 0.093 ms (0.03%)
ddtrace._trace.tracer 0.112 ms (0.04%)
ddtrace.internal.debug 0.112 ms (0.04%)
ddtrace._monkey 0.094 ms (0.03%)
ddtrace.appsec._listeners 0.094 ms (0.03%)
ddtrace.internal.core 0.094 ms (0.03%)
ddtrace.bootstrap.sitecustomize 2.565 ms (0.95%)
ddtrace.bootstrap.preload 2.565 ms (0.95%)
ddtrace.internal.remoteconfig.client 1.278 ms (0.47%)
ddtrace.internal.runtime.runtime_metrics 0.767 ms (0.28%)
ddtrace.internal.runtime.metric_collectors 0.767 ms (0.28%)
ddtrace.internal.products 0.312 ms (0.12%)
importlib.metadata 0.312 ms (0.12%)
importlib.metadata._meta 0.074 ms (0.03%)
importlib.abc 0.064 ms (0.02%)
importlib.resources 0.064 ms (0.02%)
importlib.resources._common 0.064 ms (0.02%)
zipfile 0.063 ms (0.02%)
ddtrace.settings.profiling 0.135 ms (0.05%)
ddtrace.internal.datadog.profiling.ddup 0.074 ms (0.03%)
ddtrace.vendor.psutil 0.061 ms (0.02%)
pwd 0.061 ms (0.02%)
multiprocessing 0.072 ms (0.03%)
multiprocessing.context 0.072 ms (0.03%)

@pr-commenter
Copy link

pr-commenter bot commented May 12, 2025

Benchmarks

Benchmark execution time: 2025-05-14 14:41:54

Comparing candidate commit 8a8fbe9 in PR branch christophe-papazian/fix_waf_init_with_gevent with baseline commit 0a92748 in branch main.

Found 2 performance improvements and 8 performance regressions! Performance is the same for 497 metrics, 5 unstable metrics.

scenario:ddtracerun-auto_appsec

  • 🟥 execution_time [+55.513ms; +57.945ms] or [+7.983%; +8.333%]

scenario:ddtracerun-auto_baseline

  • 🟥 execution_time [+44.986ms; +46.221ms] or [+7.110%; +7.306%]

scenario:ddtracerun-auto_runtimemetrics

  • 🟥 execution_time [+49.710ms; +52.870ms] or [+7.769%; +8.263%]

scenario:ddtracerun-auto_telemetry

  • 🟥 execution_time [+52.097ms; +55.647ms] or [+8.232%; +8.793%]

scenario:iast_aspects-format_map_aspect

  • 🟥 execution_time [+430.476ns; +569.984ns] or [+13.011%; +17.227%]

scenario:startup-import_ddtrace_auto

  • 🟥 execution_time [+46.325ms; +49.031ms] or [+12.123%; +12.831%]

scenario:startup-import_ddtrace_auto_django

  • 🟥 execution_time [+69.178ms; +71.724ms] or [+14.099%; +14.618%]

scenario:startup-import_ddtrace_auto_flask

  • 🟥 execution_time [+66.751ms; +71.686ms] or [+12.316%; +13.226%]

scenario:telemetryaddmetric-flush-100-metrics

  • 🟩 execution_time [-18.290µs; -15.932µs] or [-9.059%; -7.892%]

scenario:telemetryaddmetric-flush-1000-metrics

  • 🟩 execution_time [-227.939µs; -211.545µs] or [-9.475%; -8.794%]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants