Skip to content

Commit

Permalink
Update toolset to latest gcc and conan (#108) (#110)
Browse files Browse the repository at this point in the history 
* Udpdate native build toolset (#108)
* updated gcc to 14.2.0
* updated conan to 2.x
* simplified building for unsupported targets
* Updated development guide
  • Loading branch information
@intuibase
intuibase authored Oct 21, 2024
1 parent 43b3d3e commit 81c856d
Show file tree
Hide file tree
Showing 52 changed files with 794 additions and 2,258 deletions.
79 changes: 60 additions & 19 deletions DEVELOPMENT.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The best method for building is to use a set of Bash scripts that we utilize in
All scripts are located in the `tools/build` folder, but they should be called from the root folder of the repository. To ensure everything works correctly on your system, you need to have Docker installed.
Each of the scripts mentioned below has a help page; to display it, simply provide the `--help` argument.

### Building the native library
### Building the native library like on CI

```bash
cd elastic-otel-php
Expand All @@ -30,6 +30,45 @@ linuxmusl-arm64

If you want to enable debug logging in tested classes, you need to export environment variable `ELASTIC_OTEL_DEBUG_LOG_TESTS=1` before run.

### Building the native library for other platforms

You can always try to compile the native part for an unsupported architecture or platform. To facilitate this, we have made it possible to remove hard dependencies on Docker images, the compiler, and build profiles.

To make everything work on your system, you will need the gcc compiler (at the time of writing, version 12.0+), cmake (v3.26+), and python 3.x.

Since our system uses Conan as the repository for required dependencies, you need to install them first. The following script will install everything necessary in the `~/.conan2` folder. If you haven't used Conan before, provide the argument `--detect_conan_profile` to create a default profile – if you have used Conan before, you can skip this. If you are not using python-venv and have Conan installed directly on your system, you can pass the argument `--skip_venv_conan`, which will cause the script to skip creating a venv and installing Conan.

```bash
./prod/native/building/install_dependencies.sh --build_output_path ./prod/native/_build/custom-release --build_type Release --detect_conan_profile
```

The script will install dependencies and generate the files necessary to configure the project in the next step (prod/native/_build/custom-release):

```bash
cmake -S ./prod/native/ -B ./prod/native/_build/custom-release/ -DCMAKE_PREFIX_PATH=./prod/native/_build/custom-release/build/Release/generators/ -DSKIP_CONAN_INSTALL=1 -DCMAKE_BUILD_TYPE=Release
```

Building:
```bash
cmake --build ./prod/native/_build/custom-release/
```

If the build is successful, you can find the built libraries using the following command:
```bash
find prod/native/_build/custom-release -name elastic*.so
```

As a result you should see:
```bash
prod/native/_build/custom-release/loader/code/elastic_otel_php_loader.so
prod/native/_build/custom-release/extension/code/elastic_otel_php_81.so
prod/native/_build/custom-release/extension/code/elastic_otel_php_80.so
prod/native/_build/custom-release/extension/code/elastic_otel_php_82.so
prod/native/_build/custom-release/extension/code/elastic_otel_php_83.so
```



### Testing the native library

The following script will run the phpt tests for the native library, which should be built in the previous step - make sure to use the same architecture. You can run tests for multiple PHP versions simultaneously by providing several versions separated by a space to the `--php_versions` parameter.
Expand Down Expand Up @@ -106,8 +145,8 @@ docker compose build
```
It will build images for all supported architectures. As a result you should get summary like this:
```bash
Successfully tagged elasticobservability/apm-agent-php-dev:native-build-gcc-12.2.0-linux-x86-64-0.0.2
Successfully tagged elasticobservability/apm-agent-php-dev:native-build-gcc-12.2.0-linuxmusl-x86-64-0.0.2
Successfully tagged elasticobservability/apm-agent-php-dev:native-build-gcc-14.2.0-linux-x86-64-0.0.1
Successfully tagged elasticobservability/apm-agent-php-dev:native-build-gcc-14.2.0-linuxmusl-x86-64-0.0.1
```

Be aware that if you want to build images for ARM64 you must run it on ARM64 hardware or inside emulator. The same applies to x86-64.
Expand All @@ -117,7 +156,7 @@ To test freshly built images, you need to udate image version in ```./tools/buil
\
If everything works as you expected, you just need to push new image to dockerhub by calling:
```bash
docker push elasticobservability/apm-agent-php-dev:native-build-gcc-12.2.0-linux-x86-64-0.0.2
docker push elasticobservability/apm-agent-php-dev:native-build-gcc-14.2.0-linux-x86-64-0.0.1
```

## Building and publishing conan artifacts
Expand All @@ -128,7 +167,7 @@ The following are instructions for building and uploading artifacts for the linu

Execution of container. All you need to do here is to use latest container image revision and replace path to your local repository.
```bash
docker run -ti -v /your/forked/repository/path/elastic-otel-php:/source -w /source/agent/native elasticobservability/apm-agent-php-dev:native-build-gcc-12.2.0-linux-x86-64-0.0.2 bash
docker run -ti -v /your/forked/repository/path/elastic-otel-php:/source -w /source/agent/native elasticobservability/apm-agent-php-dev:native-build-gcc-14.2.0-linux-x86-64-0.0.1 bash
```

In container environment we need to configure project - it will setup build environment, conan environment and build all required conan dependencies
Expand All @@ -138,36 +177,38 @@ cmake --preset linux-x86-64-release

Now we need to load python virtual environment created in previous step. This will enable path to conan tool.
```bash
source _build/linux-x86-64-release/python_venv/bin/activate
source _build/linux-x86-64-release/venv/bin/activate
```

You can list all local conan packages simply by calling:
```bash
conan search
conan list -c "*"
```

it should output listing similar to this:
```bash
recipes:
php-headers-72/1.0@elastic/local
php-headers-73/1.0@elastic/local
php-headers-74/1.0@elastic/local
php-headers-80/1.0@elastic/local
php-headers-81/1.0@elastic/local
php-headers-82/1.0@elastic/local

Local Cache
...
php-headers-80
php-headers-80/2.0
php-headers-81
php-headers-81/2.0
php-headers-82
php-headers-82/2.0
php-headers-83
php-headers-83/2.0
...
```

Now you need to login into conan as elastic user. Package upload is allowed only for mainteiners.
Now you need to login into conan as elastic user. Package upload is allowed only for mainteiners. You need to generate token from UI and use is instead of password.
```bash
conan user -r ElasticConan [email protected]
conan remote login ElasticConan [email protected]
```

Now you can upload package to conan artifactory.

`--all` option will upload all revisions of `php-headers-72` you have stored in your .conan/data folder (keep it in mind if you're sharing conan cache folder between containers). You can remove it, then conan will ask before uploading each version.
```bash
conan upload php-headers-72 --all -r=ElasticConan
conan upload -r=ElasticConan php-headers-80
```

Now you can check conan artifactory for new packages here:
Expand Down
2 changes: 2 additions & 0 deletions elastic-otel-php.properties
View file
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
version=0.2.0
supported_php_versions=(80 81 82 83)
php_headers_version=2.0
100 changes: 31 additions & 69 deletions prod/native/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
cmake_minimum_required(VERSION 3.26.0)


#TODO paplo tutaj jakos zgrac te sciezki ze skryptami i tyle

# set path for our local includes
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_SOURCE_DIR}/building/cmake")
list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/building/cmake")
list(APPEND CMAKE_PREFIX_PATH "${CMAKE_BINARY_DIR}/build/${CMAKE_BUILD_TYPE}/generators")

foreach(path ${CMAKE_PREFIX_PATH})
message(STATUS "Prefix Path: ${path}")
endforeach()

foreach(path ${CMAKE_MODULE_PATH})
message(STATUS "Module Path: ${path}")
endforeach()


include(elastic_read_properties)
Expand Down Expand Up @@ -36,6 +48,17 @@ else()
message(FATAL_ERROR "Build type not supported: ${CMAKE_BUILD_TYPE}")
endif()


if (SKIP_CONAN_INSTALL)
message(STATUS "Skipping conan dependencies installation")
else()
message(STATUS "${CMAKE_SOURCE_DIR}/building/install_dependencies.sh --build_output_path ${CMAKE_BINARY_DIR} --build_preset ${CMAKE_PRESET_NAME} --build_type ${CMAKE_BUILD_TYPE}")
execute_process(
COMMAND ${CMAKE_SOURCE_DIR}/building/install_dependencies.sh --build_output_path ${CMAKE_BINARY_DIR} --build_preset ${CMAKE_PRESET_NAME} --build_type ${CMAKE_BUILD_TYPE}
COMMAND_ERROR_IS_FATAL ANY
)
endif()

# Detect Alpine/MUSLC build
if(EXISTS /etc/alpine-release)
set(MUSL_BUILD true)
Expand All @@ -49,82 +72,21 @@ include(elastic_set_default_build_options)

message(STATUS "Detected ${CMAKE_CXX_COMPILER_ID} compiler version: ${CMAKE_CXX_COMPILER_VERSION}")

include(elastic_conan_installer)
include(elastic_conan_debugsymbols)

# Install project dependencies

set(_supported_php_versions 80 81 82 83)
elastic_array_to_list(${_PROJECT_PROPERTIES_SUPPORTED_PHP_VERSIONS} _supported_php_versions)

function(get_php_api_from_release php_version ret_val)
block(SCOPE_FOR VARIABLES)
set(_php_release_80 20200930)
set(_php_release_81 20210902)
set(_php_release_82 20220829)
set(_php_release_83 20230831)
message(STATUS "Supported PHP versions ${_supported_php_versions}")

set(${ret_val} ${_php_release_${php_version}})
return(PROPAGATE ${ret_val})
endblock()
endfunction()


message(STATUS "Creating dependencies from local directories")

# build pkgconf from local reciepe because default one rely on unstable sources host, we're fetching from github tag
elastic_conan_create(
PATH ${CMAKE_SOURCE_DIR}/building/dependencies/pkgconf
REFERENCE pkgconf/1.9.3@elastic/local
PROFILE ${_CONAN_PROFILE}
)

elastic_conan_alias(
REFERENCE pkgconf/1.9.3
TARGET pkgconf/1.9.3@elastic/local
)

foreach(_php_version ${_supported_php_versions})
elastic_conan_create(
PATH ${CMAKE_SOURCE_DIR}/building/dependencies/php${_php_version}
REFERENCE php-headers-${_php_version}/1.0@elastic/local
PROFILE ${_CONAN_PROFILE}
)
endforeach()



# TODO implement multiarray with mapping of library->version

set(dependencies
"zlib/1.3"
"libcurl/8.0.1"
"libunwind/1.6.2"
"boost/1.82.0"
"gtest/1.13.0"
)

# cmocka has issues on arm64
if (NOT ${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64")
list(APPEND dependencies "cmocka/1.1.5@")
endif()
find_package(CURL 8.10.1 REQUIRED)
find_package(libunwind 1.8.1 REQUIRED)
find_package(Boost 1.86 REQUIRED CONFIG)
find_package(GTest 1.15.0 REQUIRED)

foreach(_php_version ${_supported_php_versions})
list(APPEND dependencies "php-headers-${_php_version}/1.0@elastic/local")
find_package(php-headers-${_php_version} ${_PROJECT_PROPERTIES_PHP_HEADERS_VERSION} REQUIRED)
endforeach()

conan_cmake_run(REQUIRES ${dependencies}
OPTIONS Pkg/*:shared=False
boost:header_only=True
libcurl:shared=True
libcurl:with_libssh2=True
BUILD missing
PROFILE ${_CONAN_PROFILE}
PROFILE_BUILD ${_CONAN_PROFILE}
DEBUG_PROFILE ${_CONAN_PROFILE}
RELEASE_PROFILE ${_CONAN_PROFILE}
BASIC_SETUP CMAKE_TARGETS UPDATE NO_OUTPUT_DIRS
)

enable_testing()

add_subdirectory(libcommon)
Expand Down
13 changes: 8 additions & 5 deletions prod/native/CMakePresets.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,23 @@
"binaryDir": "${sourceDir}/_build/${presetName}",
"environment": {
"PATH": "$env{HOME}/ninja/bin:$penv{PATH}"
},
"cacheVariables": {
"CMAKE_PRESET_NAME": "${presetName}"
}
},
{
"name": "binutils",
"hidden": true,
"environment": {
"PATH": "/opt/binutils-2.40/bin:$penv{PATH}"
"PATH": "/opt/binutils-2.43/bin:$penv{PATH}"
}
},
{
"name": "gcc",
"hidden": true,
"environment": {
"COMPILER_HOME_PATH": "/opt/gcc-12.2.0"
"COMPILER_HOME_PATH": "/opt/gcc-14.2.0"
},
"cacheVariables": {
"CMAKE_C_COMPILER": {
Expand Down Expand Up @@ -60,7 +63,7 @@
"value": "Release"
}
}
},
},
{
"name": "os-type-unix",
"hidden": true,
Expand Down Expand Up @@ -133,7 +136,7 @@
"debug",
"binutils"
]
}
}
],
"buildPresets": [
{
Expand Down Expand Up @@ -279,6 +282,6 @@
"inherits": [
"release"
]
}
}
]
}
Loading

0 comments on commit 81c856d

Please sign in to comment.