Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Conan 2 #92

Merged
merged 18 commits into from
Feb 19, 2024
Merged

Upgrade to Conan 2 #92

merged 18 commits into from
Feb 19, 2024

Conversation

kyllingstad
Copy link
Member

Building on @eidekrist's earlier work, here I've completed the transition to Conan 2. The main changes are:

  • Updated conanfile.py to the new format.
  • Updated conan invocations in the GH Actions workflow to use the new command-line options.
  • Switched to using version ranges for dependencies, for greater flexibility.
  • Removed most of the custom find_package scripts, relying instead on package configuration files provided by Conan (in Conan builds) or by upstream (in non-Conan builds).
  • Removed the ability to run Conan from within CMake.

Regarding the last point: I really tried to make this work, but the upstream cmake-conan project has not released a version with full support for all features yet. Notably, their current development version does not support the CMakeToolchain generator. I also tried to modify the Build workflow to use Conan 1.61, but unfortunately, that version does not have sufficient support for Conan 2.0 features. I suggest we reintroduce this as soon as upstream releases a more feature complete cmake-conan version.

@kyllingstad kyllingstad added the enhancement New feature or request label Nov 7, 2023
@kyllingstad kyllingstad self-assigned this Nov 7, 2023
@markaren
Copy link
Collaborator

Removed the ability to run Conan from within CMake.

Removing this feature also for the future is totally legit and also a requirement at this stage

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we skipping test step by removing build.yml?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, you're right. I've restored the deleted workflow and cleaned it up a bit.

@@ -75,6 +76,6 @@ jobs:
SHORT_REFNAME="${REFNAME:0:40}"
CHANNEL="testing-${SHORT_REFNAME//\//_}"
fi
conan create -s build_type=${{ matrix.build_type }} -b missing . osp/${CHANNEL}
conan create -s build_type=${{ matrix.build_type }} -b missing . --user=osp --channel=${CHANNEL}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the build process for conan2 has been changed, it would be good to have an updated README.md that explains how to build the source locally for development and release purposes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current README simply refers users to the CI workflow setup for instructions. This is good in the sense that the instructions will always be up-to-date and known to work, but it is unfortunate in the sense that it requires users to have at least some understanding of the GitHub workflow definition files.

Do I understand you correctly that you wish me to add the instructions to the README? (If not, I still have to update the link to the workflow file, since I changed its name.)

@kyllingstad
Copy link
Member Author

The "Build and test" workflow fails in the "Build" step for reasons I do not understand and don't really know how to debug, because the exact same commands work perfectly on my computer. It's complaining that <thrift/Thrift.h> cannot be found, even though the find_package(Thrift) command apparently succeeds in the "Generate build system" step. (And it also builds just fine when the whole process is driven by Conan in the "Upload Conan packages" workflow.)

@markaren, @davidhjp01, all ideas are welcome. And it would be great if you too could test the same commands locally if you have the chance.

Copy link
Contributor

@davidhjp01 davidhjp01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build and test workflow now succeeds.

There are still issues remain (need discussion):

  1. conan2 does not copy output files to bin directory during build. This causes ctest to fail (see the comments). We need to either set CMAKE_RUNTIME_OUTPUT_DIRECTORY_XXX or install files to build/bin directory.
  2. Some of the conan2 libraries uploaded conan.io does not have Debug builds that triggers rebuilding of large libraries like boost, openssl. This increases the overall build time quite drastically (~30mins). Need to upload the build library to jfrog or cache them during build.

Comment on lines 23 to 27
run: conan install --build=missing --output-folder=build -s build_type=${{ matrix.build_type }} .
- name: Generate build system
run: cmake --preset conan-$(echo ${{ matrix.build_type }} | tr '[:upper:]' '[:lower:]') -DPROXYFMU_BUILD_EXAMPLES=ON -DPROXYFMU_BUILD_TESTS=ON
- name: Build
run: cmake --build --preset conan-$(echo ${{ matrix.build_type }} | tr '[:upper:]' '[:lower:]')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using preset for cmake configure and build

Comment on lines 54 to 59
- name: Install dependencies
run: conan install --build=missing --output-folder=build -s build_type=${{ matrix.build_type }} .
- name: Generate build system
run: cmake --preset conan-default -DPROXYFMU_BUILD_EXAMPLES=ON -DPROXYFMU_BUILD_TESTS=ON
- name: Build
run: cmake --build --preset "conan-${{ matrix.build_type }}".ToLower()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using cmake preset to configure and build (notice conan-default in case for Windows when configuring).

if: matrix.build_type == 'Release'
with:
name: proxyfmu
path: build/bin/proxyfmu*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

outputs are not copied to bin directory. This is because we do not call conan_output_dirs_setup in conan2.

We need to either copy the following (from conanv1) to our cmake or --target install all the artifacts to bin before upload. I have not fixed this issue yet.

macro(conan_output_dirs_setup)
    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/bin)
    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELWITHDEBINFO ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_MINSIZEREL ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})
    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_RUNTIME_OUTPUT_DIRECTORY})

    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib)
    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY_RELEASE ${CMAKE_ARCHIVE_OUTPUT_DIRECTORY})
    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY_RELWITHDEBINFO ${CMAKE_ARCHIVE_OUTPUT_DIRECTORY})
    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY_MINSIZEREL ${CMAKE_ARCHIVE_OUTPUT_DIRECTORY})
    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY_DEBUG ${CMAKE_ARCHIVE_OUTPUT_DIRECTORY})

    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/lib)
    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_RELWITHDEBINFO ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_MINSIZEREL ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_DEBUG ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
endmacro()

Comment on lines +14 to +21

set_tests_properties(
test_identity
test_controlled_temp
test_mass-spring-damper
PROPERTIES
ENVIRONMENT "PATH=$<SHELL_PATH:$<TARGET_FILE_DIR:proxyfmu-client>\;>$ENV{PATH}"
)
Copy link
Contributor

@davidhjp01 davidhjp01 Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ctest failed because conan2 did not copy output files to bin (On windows, .exe can load .dlls in the same dir).

gcc9 builds are not available from conancenter. Built libraries are uploaded to osp jfrog.
@davidhjp01 davidhjp01 merged commit 748059f into master Feb 19, 2024
8 checks passed
@davidhjp01 davidhjp01 deleted the feature/conan-2 branch February 19, 2024 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants