Skip to content
This repository was archived by the owner on Mar 4, 2025. It is now read-only.

Commit

Permalink
Upstream sync part2 (#442)
Browse files Browse the repository at this point in the history
* Upstream sync bfc8b54..4d54189

* Make all integration tests utilize Native Loader instead of loading `Datadog.Trace.ClrProfiler.Native` directly (#2358)

* Make all integration tests utilize the Native Loader instead of loading Datadog.Trace.ClrProfiler.Native.dll directly

* Fix NUKE dependency graph to ensure we build Native Loader *before* running integration tests
Rename "BuildMonitoringHome" to "BuildNativeLoader" in CI scripts as well

* Fix Native Loader crashing on Linux due to missing -lstdc++fs compiler flag

* Fix Native Loader crashing on Debian when loader.conf contains empty lines

* Utilize the Native Loader on ARM64

* Adjust all launchSettings.json files to new folder structure

* Change tracer home folder to lower-case.

* Enable testing both with and without the Native Loader in CI.

* Integration tests only utilize NativeLoader when 'use_native_loader' is true.

* fix build serveless

* fix integration tests on windows

* [Cleanup] Move shared classes/files to shared and ensure Profiler, Tracer and Native Loader use them (#2493)

* Move spdlog, fmt libraries/include files from Tracer lib folder to shared
* Move coreclr include files from Tracer lib folder to shared
* Delete lib folder from Tracer
* Delete dotnet-runtime-coreclr from Profiler
* Make Tracer use the WSTRING from shared instead of its own string.
* Make sure Profiler, Tracer and Native Loader rely on coreclr, spdlog and fmt from shared.
* Make Profiler, Tracer and Native Loader use the the same native underlying mechanism (logger_impl.h)
* Delete duplicated files

* [Profiler] Bump profiler version to 2.5.0.0 (#2520)

Co-authored-by: Gregory LEOCADIE <[email protected]>

* Fix warnings (#2518)

* fix native unit tests on windows

Co-authored-by: Omer Raviv <[email protected]>
Co-authored-by: Gregory LEOCADIE <[email protected]>
Co-authored-by: chrisnas <[email protected]>
  • Loading branch information
4 people authored Mar 28, 2022
1 parent cf04d52 commit 306c102
Show file tree
Hide file tree
Showing 1,132 changed files with 2,923 additions and 247,281 deletions.
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ packages/
**/obj/

!tracer/src/bin/
!shared/bin/monitoring-home/tracer/
!tracer/test/test-applications/integrations/Samples.AWS.Lambda/bin/
!tracer/test/test-applications/aspnet/Samples.AspNet472.LoaderOptimizationRegKey/bin/
!tracer/src/WindowsInstaller/bin/
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,12 @@ on:
env:
buildConfiguration: Release
dotnetSdkVersion: 6.0.200
relativeTracerHome: /tracer/src/bin/windows-tracer-home
relativeTracerHome: /shared/bin/monitoring-home/tracer
relativeArtifacts: /tracer/src/bin/artifacts
binDir: ${{ github.workspace }}/tracer/src/bin
ddTracerHome: ${{ github.workspace }}/tracer/src/bin/dd-tracer-home
tracerHome: ${{ github.workspace }}/tracer/src/bin/windows-tracer-home
tracerHome: ${{ github.workspace }}/shared/bin/monitoring-home/tracer
profilerSrcDirectory: ${{ github.workspace }}/../dd-continuous-profiler-dotnet
artifacts: ${{ github.workspace }}/tracer/src/bin/artifacts
isMainBranch: $[eq(github.ref, 'refs/heads/main')]
NugetPackageDirectory: ${{ github.workspace }}/packages
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-draft.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
docker run --rm \
--mount type=bind,source="${GITHUB_WORKSPACE}",target=/project \
--env NugetPackageDirectory=/project/packages \
--env tracerHome=/project/tracer/bin/tracer-home \
--env tracerHome=/shared/bin/monitoring-home/tracer \
--env artifacts=/project/tracer/src/bin/artifacts \
dd-trace-dotnet/${baseImage}-builder \
dotnet /build/bin/Debug/_build.dll Clean BuildTracerHome ZipTracerHome
Expand Down
2 changes: 1 addition & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ linux-build:
- |
docker run \
--env NugetPackageDirectory=/project/packages \
--env tracerHome=/project/tracer/bin/tracer-home \
--env tracerHome=/shared/bin/monitoring-home/tracer \
--env artifacts=/project/tracer/src/bin/artifacts \
--name release \
dd-trace-dotnet/${baseImage}-builder \
Expand Down
6 changes: 3 additions & 3 deletions dev/envvars.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,15 @@ CURDIR=$(current_dir)
# Enable .NET Framework Profiling API
export COR_ENABLE_PROFILING="1"
export COR_PROFILER="{B4C89B0F-9908-4F73-9F59-0D77C5A06874}"
export COR_PROFILER_PATH="${CURDIR}/tracer/bin/tracer-home/${OPT_DIR}SignalFx.Tracing.ClrProfiler.Native.${SUFIX}"
export COR_PROFILER_PATH="${CURDIR}/shared/bin/monitoring-home/tracer${OPT_DIR}SignalFx.Tracing.ClrProfiler.Native.${SUFIX}"

# Enable .NET Core Profiling API
export CORECLR_ENABLE_PROFILING="1"
export CORECLR_PROFILER="{B4C89B0F-9908-4F73-9F59-0D77C5A06874}"
export CORECLR_PROFILER_PATH="${CURDIR}/tracer/bin/tracer-home/${OPT_DIR}SignalFx.Tracing.ClrProfiler.Native.${SUFIX}"
export CORECLR_PROFILER_PATH="${CURDIR}/shared/bin/monitoring-home/tracer/${OPT_DIR}SignalFx.Tracing.ClrProfiler.Native.${SUFIX}"

# Configure SFx .NET Tracer
export SIGNALFX_DOTNET_TRACER_HOME="${CURDIR}/tracer/bin/tracer-home"
export SIGNALFX_DOTNET_TRACER_HOME="${CURDIR}/shared/bin/monitoring-home/tracer"
export SIGNALFX_VERSION="1.0.0"
export SIGNALFX_TRACE_DEBUG="1"
export SIGNALFX_DUMP_ILREWRITE_ENABLED="0"
Expand Down
6 changes: 3 additions & 3 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,7 @@ services:
- ./:/project
environment:
- NugetPackageDirectory=/project/${relativeNugetPackageDirectory:-packages}
- tracerHome=/project/${relativeTracerHome:-tracer/src/bin/windows-tracer-home}
- tracerHome=/project/${relativeTracerHome:-shared/bin/monitoring-home/tracer}
- artifacts=/project/${relativeArtifacts:-tracer/src/bin/artifacts}
- framework=${framework:-netcoreapp3.1}
- baseImage=${baseImage:-default}
Expand Down Expand Up @@ -510,7 +510,7 @@ services:
- ./:/project
environment:
- NugetPackageDirectory=/project/${relativeNugetPackageDirectory:-packages}
- tracerHome=/project/${relativeTracerHome:-tracer/src/bin/windows-tracer-home}
- tracerHome=/project/${relativeTracerHome:-shared/bin/monitoring-home/tracer}
- artifacts=/project/${relativeArtifacts:-tracer/src/bin/artifacts}
- framework=${framework:-netcoreapp3.1}
- explorationTestUseCase=${explorationTestUseCase:-debugger}
Expand Down Expand Up @@ -563,7 +563,7 @@ services:
- ./:/project
environment:
- NugetPackageDirectory=/project/${relativeNugetPackageDirectory:-packages}
- tracerHome=/project/${relativeTracerHome:-tracer/src/bin/windows-tracer-home}
- tracerHome=/project/${relativeTracerHome:-shared/bin/monitoring-home/tracer}
- artifacts=/project/${relativeArtifacts:-tracer/src/bin/artifacts}
- framework=${framework:-netcoreapp3.1}
- baseImage=${baseImage:-debian}
Expand Down
5 changes: 0 additions & 5 deletions docs/internal/internal-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,6 @@ These settings should be never used by the users.
|-|-|-|
| `SIGNALFX_AGENT_HOST` | The Agent host where the tracer can send traces. | |
| `SIGNALFX_APM_RECEIVER_PORT` | The port for Trace Agent binding. | `8126` |
| `SIGNALFX_APPSEC_ENABLED` | Enables the AppSec. | `false` |
| `SIGNALFX_APPSEC_EXTRA_HEADERS` | Optional custom headers the user wants to send. | |
| `SIGNALFX_APPSEC_IPHEADER` | Optional name of the custom header to take into account for the ip address. | |
| `SIGNALFX_APPSEC_KEEP_TRACES` | Specifies if the AppSec traces should be explicitly kept or droped. | `true` |
| `SIGNALFX_APPSEC_RULES` | Overrides the default rules file provided. Must be a path to a valid JSON rules file. | |
| `SIGNALFX_CIVISIBILITY_AGENTLESS_ENABLED` | Enable to activate Agentless in CI Visibility. | `false` |
| `SIGNALFX_CIVISIBILITY_ENABLED` | Enable to activate CI Visibility. | `false` |
| `SIGNALFX_DOGSTATSD_ARGS` | Comma-separated list of arguments to be passed to the DogStatsD process. | |
Expand Down
4 changes: 2 additions & 2 deletions profiler/Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -188,9 +188,9 @@


<PropertyGroup Condition=" '$(MSBuildProjectExtension)' == '.vcxproj' ">
<CORECLR-PATH>$(EnlistmentRoot)/profiler/src/dotnet-runtime-coreclr/</CORECLR-PATH>
<DOTNET-TRACER-REPO-ROOT-PATH>$([System.IO.Path]::GetFullPath( $(EnlistmentRoot)/ ))</DOTNET-TRACER-REPO-ROOT-PATH>
<SHARED-LIB-PATH>$(DOTNET-TRACER-REPO-ROOT-PATH)shared/src/native-lib/</SHARED-LIB-PATH>
<SHARED-LIB-PATH>$(DOTNET-TRACER-REPO-ROOT-PATH)shared\src\native-lib\</SHARED-LIB-PATH>
<CORECLR-PATH>$(SHARED-LIB-PATH)coreclr\src</CORECLR-PATH>
<SHARED-LIB-PLATFORM Condition="'$(Platform)'=='x64'">x64</SHARED-LIB-PLATFORM>
<SHARED-LIB-PLATFORM Condition="'$(Platform)'=='Win32' OR '$(Platform)'=='x86'">x86</SHARED-LIB-PLATFORM>
<SHARED-LIB-INCLUDES>$(SHARED-LIB-PATH)fmt_$(SHARED-LIB-PLATFORM)-windows-static\include;$(SHARED-LIB-PATH)spdlog\include</SHARED-LIB-INCLUDES>
Expand Down
4 changes: 2 additions & 2 deletions profiler/cppcheck-suppressions.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
*:*/dotnet-runtime-coreclr/*
*:*/coreclr/*
*:*/test/*
*:*/_deps/*
// This error is an issue with cppcheck. So do not report this one specifically
preprocessorErrorDirective:*/logging.h
preprocessorErrorDirective:*/fmt/core.h
preprocessorErrorDirective:*/fmt/bundled/core.h
// in the loader, we compute the size of the assembly and the pdb file (l.1791 and l.1794)
// this triggers the rule below. So skip for the whole file for now, and add inline suppression
// later.
Expand Down
2 changes: 1 addition & 1 deletion profiler/src/Demos/WindowsService01/ReadMe.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Wait until the shutdown is complete.
Start the service again.
Wait until starting is complete (pausing/resuming will not be enough).

6. Profiler should attach and start working. Logs should be found in the default (or specified) log location. The default is `C:\ProgramData\Datadog-APM\logs\DotNet\`.
6. Profiler should attach and start working. Logs should be found in the default (or specified) log location. The default is `C:\ProgramData\SignalFx .NET Tracing\logs\DotNet\`.

For some additional info, see https://docs.datadoghq.com/tracing/setup_overview/setup/dotnet-framework/?tab=environmentvariables#installation-and-getting-started

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ private static bool IsWindowsFileSystem()

private static class DefaultDirectories
{
public const string WindowsAppDataRoot = @"Datadog-APM\"; // relative to Environment.SpecialFolder.CommonApplicationData
public const string WindowsAppDataRoot = @"SignalFx .NET Tracing\"; // relative to Environment.SpecialFolder.CommonApplicationData
public const string LinuxAppDataRoot = @"/var/log/signalfx"; // global path

public const string WindowsLogsDir = @"logs"; // relative to AppDataRoot
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ public static IProductConfiguration ApplyDevDefaults(this IProductConfiguration

// --> to preprod
mutableConfig.ProfilesIngestionEndpoint_Url = "https://intake.profile.datadoghq.com/v1/input";
mutableConfig.ProfilesIngestionEndpoint_DatadogApiKey = "";
mutableConfig.ProfilesIngestionEndpoint_DatadogApiKey = string.Empty;

string ddService = ConfigurationProviderUtils.GetDdServiceFallback();
mutableConfig.DDDataTags_Host = ConfigurationProviderUtils.GetMachineName();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// <copyright file="GlobalSuppressions.cs" company="Datadog">
// Unless explicitly stated otherwise all files in this repository are licensed under the Apache 2 License.
// This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2022 Datadog, Inc.
// </copyright>

// This file is used by Code Analysis to maintain SuppressMessage
// attributes that are applied to this project.
// Project-level suppressions either have no target or are given
// a specific target and scoped to a namespace, type, member, etc.

using System.Diagnostics.CodeAnalysis;

[assembly: SuppressMessage("StyleCop.CSharp.LayoutRules", "SA1518:Use line endings correctly at end of file", Justification = "Add no value")]
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
// <copyright file="PProfSampleValueType.cs" company="Datadog">
// <copyright file="PProfSampleValueType.cs" company="Datadog">
// Unless explicitly stated otherwise all files in this repository are licensed under the Apache 2 License.
// This product includes software developed at Datadog (https://www.datadoghq.com/). Copyright 2022 Datadog, Inc.
// </copyright>

using System;

namespace Datadog.PProf.Export
{
internal struct PProfSampleValueType
Expand All @@ -19,4 +17,4 @@ public PProfSampleValueType(string type, string unit)

public string Unit { get; }
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ add_library(${PROFILER_STATIC_LIB_NAME} STATIC
${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-src/miniutf.cpp
${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-src/il_rewriter.cpp
${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-src/il_rewriter_wrapper.cpp
../../dotnet-runtime-coreclr/pal/prebuilt/idl/corprof_i.cpp
${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-lib/coreclr/src/pal/prebuilt/idl/corprof_i.cpp
${GENERATED_OBJ_FILES}
)

Expand All @@ -154,10 +154,10 @@ set_target_properties(${PROFILER_STATIC_LIB_NAME} PROPERTIES PREFIX "")
# Define directories includes
target_include_directories(${PROFILER_STATIC_LIB_NAME}
PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/../Datadog.Profiler.Native
PUBLIC ../../dotnet-runtime-coreclr/pal/inc/rt
PUBLIC ../../dotnet-runtime-coreclr/pal/prebuilt/inc
PUBLIC ../../dotnet-runtime-coreclr/pal/inc
PUBLIC ../../dotnet-runtime-coreclr/inc
PUBLIC ${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-lib/coreclr/src/pal/inc/rt
PUBLIC ${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-lib/coreclr/src/pal/prebuilt/inc
PUBLIC ${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-lib/coreclr/src/pal/inc
PUBLIC ${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-lib/coreclr/src/inc
PUBLIC ${DOTNET_TRACER_REPO_ROOT_PATH}/shared/src/native-lib/spdlog/include
PUBLIC ${OUTPUT_DEPS_DIR}/fmt/include
PUBLIC ${OUTPUT_DEPS_DIR}/libunwind/include
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,12 @@ NETCOREAPP20_Datadog_AutoInstrumentation_ManagedLoader_pdb SYMBOLS "net
// ------- version info -------------------------------------------------------
// Alpha 1.0.2
// Beta 1.1.1
// GA 1.2.0
// Beta 2.?.?.x once integrated into Tracer repository (where x = beta version such as 1 or 2)
// GA ???
//
VS_VERSION_INFO VERSIONINFO
FILEVERSION 1, 1, 1, 1
PRODUCTVERSION 1, 1, 1, 1
FILEVERSION 0, 2, 2, 0
PRODUCTVERSION 0, 2, 2, 0
FILEFLAGSMASK VS_FF_PRERELEASE
FILEOS VOS_NT
FILETYPE VFT_DLL
Expand All @@ -105,12 +106,12 @@ BEGIN
BEGIN
VALUE "CompanyName", "Datadog"
VALUE "FileDescription", "Continuous Profiler for .NET Applications"
VALUE "FileVersion", "1.1.1.1"
VALUE "FileVersion", "0.2.2.0"
VALUE "InternalName", "Native Profiler Engine"
VALUE "LegalCopyright", "(c) Datadog 2020-2022"
VALUE "OriginalFilename", "Datadog.Profiler.Native.dll"
VALUE "ProductName", "Continuous Profiler for .NET Applications"
VALUE "ProductVersion", "1.1.1.1"
VALUE "ProductVersion", "0.2.2.0"
END
END
BLOCK "VarFileInfo"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
#include "shared/src/native-src/loader.h"
#include "shared/src/native-src/pal.h"
#include "shared/src/native-src/string.h"
#include "shared/src/native-src/environment_variables.h"

// The following macros are used to construct the profiler file:
#ifdef _WINDOWS
Expand Down
19 changes: 9 additions & 10 deletions profiler/src/ProfilerEngine/Datadog.Profiler.Native/Log.h
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,19 @@

#include <string>

#include "shared/src/native-src/logging.h"
#include "shared/src/native-src/logger_impl.h"
#include "shared/src/native-src/string.h"

class Log final
{
private:
struct ProfilerLoggerPolicy
{
inline static const shared::WSTRING filename = WStr("dotNet-profiler-native");
inline static const std::string file_name = WStr("dotNet-profiler-native");
#ifdef _WIN32
inline static const shared::WSTRING folder_path = WStr(R"(SignalFx .NET Tracing\logs)");
#endif
inline static const shared::WSTRING folder_path = WStr(R"(SignalFx .NET Tracing\logDotNets)");
inline static const std::string pattern = "[%Y-%m-%d %H:%M:%S.%e | %l | PId: %P | TId: %t] %v";
struct environment
struct logging_environment
{
inline static const shared::WSTRING log_path = EnvironmentVariables::LogPath;
inline static const shared::WSTRING log_directory = EnvironmentVariables::LogDirectory;
Expand All @@ -30,29 +29,29 @@ class Log final
public:
static bool IsDebugEnabled()
{
return shared::Logger<ProfilerLoggerPolicy>::IsDebugEnabled();
return shared::LoggerImpl<ProfilerLoggerPolicy>::Instance()->IsDebugEnabled();
}

static void EnableDebug()
{
shared::Logger<ProfilerLoggerPolicy>::EnableDebug();
shared::LoggerImpl<ProfilerLoggerPolicy>::Instance()->EnableDebug();
}

template <typename... Args>
static inline void Debug(const Args... args)
{
shared::Logger<ProfilerLoggerPolicy>::Debug<Args...>(args...);
shared::LoggerImpl<ProfilerLoggerPolicy>::Instance()->Debug<Args...>(args...);
}

template <typename... Args>
static void Info(const Args... args)
{
shared::Logger<ProfilerLoggerPolicy>::Info<Args...>(args...);
shared::LoggerImpl<ProfilerLoggerPolicy>::Instance()->Info<Args...>(args...);
}

template <typename... Args>
static void Error(const Args... args)
{
shared::Logger<ProfilerLoggerPolicy>::Error<Args...>(args...);
shared::LoggerImpl<ProfilerLoggerPolicy>::Instance()->Error<Args...>(args...);
}
};
16 changes: 9 additions & 7 deletions profiler/src/ProfilerEngine/ProductVersion.props
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,18 @@

<!-- * * * * * * * * * * * INPUTS. Update this section EVERY time the component is shipped/released! * * * * * * * * * * * -->
<PropertyGroup>
<ProductVersionMajor>1</ProductVersionMajor>
<ProductVersionMinor>1</ProductVersionMinor>
<ProductVersionBuild>1</ProductVersionBuild>
<ProductVersionMajor>0</ProductVersionMajor>
<ProductVersionMinor>2</ProductVersionMinor>
<ProductVersionBuild>2</ProductVersionBuild>
<BetaVersion>0</BetaVersion>
<!-- Alpha 1.0.2 -->
<!-- Beta 1.1.1 -->
<!-- GA 1.2.0 -->
<!-- Beta 2.?.?.x once in Tracer repository (where x is the beta number ex: 1 or 2 when ready) -->
<!-- GA ?.?.? -->
<ProductVersion>$(ProductVersionMajor).$(ProductVersionMinor).$(ProductVersionBuild)</ProductVersion>

<!-- ProductVersionPrerelease format examples: alpha.1, alpha.2, beta.1, beta.2; EMPTY for stable releases. -->
<ProductVersionPrerelease>beta.$(ProductVersionBuild)</ProductVersionPrerelease>
<ProductVersionPrerelease>beta.$(BetaVersion)</ProductVersionPrerelease>
</PropertyGroup>
<!-- * * * * * * * * * * * END OF INPUTS. * * * * * * * * * * * -->

Expand All @@ -32,8 +34,8 @@
</PropertyGroup>

<PropertyGroup>
<AssemblyVersion>$(ProductVersion).1</AssemblyVersion>
<FileVersion>$(ProductVersion).1</FileVersion>
<AssemblyVersion>$(ProductVersion).$(BetaVersion)</AssemblyVersion>
<FileVersion>$(ProductVersion).$(BetaVersion)</FileVersion>
<InformationalVersion>$(ProductVersion)$(ProductSuffix)</InformationalVersion>

<!-- May need to add 'PackageVersion' here if and when we build NuGet packages. -->
Expand Down
20 changes: 0 additions & 20 deletions profiler/src/dotnet-runtime-coreclr/README.md

This file was deleted.

Loading

0 comments on commit 306c102

Please sign in to comment.