cpython: avoid invalid configurations #25500

valgur · 2024-10-03T08:58:30Z

Summary

Changes to recipe: cpython/[*]

Motivation

The cpython recipe is currently marked as invalid on

macOS x86_64 due to tk and ncurses not supporting cross-building.
tk/8.6.10: Invalid: The tk conan recipe does not currently support Macos cross-builds. A contribution to add this functionality would be welcome.
ncurses/6.4: Invalid: Cross building to/from arm is (currently) not supported
cpython/*:shared=False on MSVC for v3.10 and greater due to build errors.
cpython/3.12.7: Invalid ID: Static msvc build disabled (>=3.10) due to "AttributeError: module 'sys' has no attribute 'winver'"

Both of these are avoidable.

This affects the build of these configurations for nanobind, for example: #20297 (comment)

/cc @Ahajha

Details

Read the contributing guidelines
Checked that this PR is not a duplicate: list of PRs by recipe
Tested locally with at least one configuration using a recent version of Conan

Ahajha · 2024-10-04T05:12:50Z

I like the goal here. There's another invalid config, relating to mpdecimal on Windows/shared. The cxx option is enabled by default, but is incompatible with the shared option on Windows, so the Windows/all shared build is invalid. So probably want to also just disable the cxx option, at least on Windows.

Fixing the issues that come up from these might not be trivial, but I can take a look if I have some spare time. It might not be a bad idea to split this into a few PRs if it gets difficult.

recipes/cpython/all/conanfile.py

Ahajha · 2024-10-14T04:07:15Z

@AbrilRBS To fix the Mac cross-compile errors, we would need an additional build-time dependency on cpython itself. Can CCI support this?

conan-center-bot · 2024-10-14T06:47:35Z

Conan v1 pipeline ❌

Warning

Conan Center will stop receiving updates for Conan 1.x packages soon - please see announcement.

Failure in build 4 (6d8d46c6c3abe10494f4fa56db94c34e3544df2a):

cpython/3.12.7:
CI failed to create some packages (All logs)

Logs for packageID 4a14cda3e7ca148a6b1faadb3dd157d4515b117c:

[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=7
os=Linux
[options]
cpython:shared=False

[...]
conaninfo.txt
conanrun.sh
conanrunenv-release-x86_64.sh
deactivate_conanbuild.sh
deactivate_conanrun.sh
deactivate_conanrunenv-release-x86_64.sh
gdbm.db
graph_info.json
lib
cpython/3.12.7 (test package): Result of test was expected.

----Running------
> . "/home/conan/workspace/prod-v1/bsr/cci-45ad3f50/recipes/cpython/all/test_v1_package/build/a1e56edeec75e2e77a868017f5a8d20d8d2bc14d/conanbuild.sh" && bin/test_package
-----------------
Today is Mon Oct 14 06:38:06 2024
Using Python_ADDITIONAL_VERSIONS: 3.12.7;3.12;3;3.12.7;3.12;3
Python3_EXECUTABLE: /home/conan/workspace/prod-v1/bsr/110064/ecabc/.conan/data/cpython/3.12.7/_/_/package/4a14cda3e7ca148a6b1faadb3dd157d4515b117c/bin/python3.12
Python3_INTERPRETER_ID: Python
Python3_VERSION: 3.12.7
Python3_INCLUDE_DIRS: /home/conan/workspace/prod-v1/bsr/110064/ecabc/.conan/data/cpython/3.12.7/_/_/package/4a14cda3e7ca148a6b1faadb3dd157d4515b117c/include/python3.12
Python3_LIBRARIES: /home/conan/workspace/prod-v1/bsr/110064/ecabc/.conan/data/cpython/3.12.7/_/_/package/4a14cda3e7ca148a6b1faadb3dd157d4515b117c/lib/libpython3.12.a
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_EXPORT_NO_PACKAGE_REGISTRY
    CMAKE_INSTALL_BINDIR
    CMAKE_INSTALL_DATAROOTDIR
    CMAKE_INSTALL_INCLUDEDIR
    CMAKE_INSTALL_LIBDIR
    CMAKE_INSTALL_LIBEXECDIR
    CMAKE_INSTALL_OLDINCLUDEDIR
    CMAKE_INSTALL_SBINDIR


WARN: **************************************************
WARN: *** Conan 1 is legacy and on a deprecation path **
WARN: *********** Please upgrade to Conan 2 ************
WARN: **************************************************
WARN: openssl/3.3.2: requirement zlib/[>=1.2.11 <2] overridden by cpython/3.12.7 to zlib/1.3.1 
WARN: tcl/8.6.10: requirement zlib/[>=1.2.11 <2] overridden by tk/8.6.10 to zlib/1.3.1 
WARN: fontconfig/2.13.93: requirement expat/[>=2.6.2 <3] overridden by tk/8.6.10 to expat/2.6.3 
WARN: freetype/2.13.2: requirement zlib/[>=1.2.10 <2] overridden by fontconfig/2.13.93 to zlib/1.3.1 
WARN: libpng/1.6.44: requirement zlib/[>=1.2.11 <2] overridden by freetype/2.13.2 to zlib/1.3.1 
cpython/3.12.7 (test package): WARN: 
     ************************************************
     The 'cmake' generator is deprecated.
     Please update your code and remove it.
     *************************************************

cpython/3.12.7 (test package): WARN: **** The 'from conans import CMake' helper is deprecated. Please update your code and remove it. ****

cpython/3.11.9:
Didn't run or was cancelled before finishing
cpython/3.12.2:
Didn't run or was cancelled before finishing
cpython/3.8.19:
Didn't run or was cancelled before finishing
cpython/3.10.14:
Didn't run or was cancelled before finishing
cpython/3.9.19:
Didn't run or was cancelled before finishing

^{Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.}

Conan v2 pipeline ❌

Note: Conan v2 builds are now mandatory. Please read our discussion about it.

The v2 pipeline failed. Please, review the errors and note this is required for pull requests to be merged. In case this recipe is still not ported to Conan 2.x, please, ping @conan-io/barbarians on the PR and we will help you.

Failure in build 4 (6d8d46c6c3abe10494f4fa56db94c34e3544df2a):

cpython/3.12.7:
Didn't run or was cancelled before finishing

cpython/3.10.14:
CI failed to create some packages (All logs)

Logs for packageID 832cf6f7ab6b91ef09d21447b2d2dfe1d8692894:

[settings]
arch=x86_64
build_type=Release
compiler=apple-clang
compiler.cppstd=17
compiler.libcxx=libc++
compiler.version=13
os=Macos
[options]
*/*:shared=False

[...]
cpython/3.10.14: package(): Packaged 1 '.fish' file: activate.fish
cpython/3.10.14: package(): Packaged 1 '.csh' file: activate.csh
cpython/3.10.14: package(): Packaged 1 '.ps1' file: Activate.ps1
cpython/3.10.14: package(): Packaged 2 '.pickle' files: Grammar3.10.14.final.0.pickle, PatternGrammar3.10.14.final.0.pickle
cpython/3.10.14: package(): Packaged 1 '.html' file: help.html
cpython/3.10.14: package(): Packaged 1 '.pyw' file: idle.pyw
cpython/3.10.14: package(): Packaged 4 '.def' files: config-main.def, config-keys.def, config-extensions.def, config-highlight.def
cpython/3.10.14: package(): Packaged 9 '.gif' files
cpython/3.10.14: package(): Packaged 4 '.png' files: idle_48.png, idle_16.png, idle_32.png, idle_256.png
cpython/3.10.14: package(): Packaged 1 '.ico' file: idle.ico
cpython/3.10.14: package(): Packaged 1 '.css' file: _pydoc.css
cpython/3.10.14: package(): Packaged 1 '.rst' file: architecture.rst
cpython/3.10.14: Created package revision 04474de4e7bd44058e6377e9ecf20554
cpython/3.10.14: Package '3ebb45ad952837444686011331625ac965bae6ad' created
cpython/3.10.14: Full package reference: cpython/3.10.14#ba4425226637e1d7a7e6f50b91222fa2:3ebb45ad952837444686011331625ac965bae6ad#04474de4e7bd44058e6377e9ecf20554
cpython/3.10.14: Package folder /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth0ecdc34d7f76c/p
cpython/3.10.14: Appending PATH environment variable: /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth0ecdc34d7f76c/p/bin
cpython/3.10.14: Appending PYTHON environment variable: /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth0ecdc34d7f76c/p/bin/python3.10
cpython/3.10.14: Setting PYTHON_ROOT environment variable: /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth0ecdc34d7f76c/p

-------- Installing package cpython/3.10.14 (25 of 25) --------
cpython/3.10.14: Building from source
cpython/3.10.14: Package cpython/3.10.14:832cf6f7ab6b91ef09d21447b2d2dfe1d8692894
cpython/3.10.14: Copying sources to build folder
cpython/3.10.14: Building your package in /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth1c3bbd7425499/b
cpython/3.10.14: Calling generate()
cpython/3.10.14: Generators folder: /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth1c3bbd7425499/b/build-release/conan
cpython/3.10.14: Generating aggregated env files
cpython/3.10.14: Generated aggregated env files: ['conanbuild.sh']
cpython/3.10.14: Calling build()
cpython/3.10.14: Apply patch (bugfix): Fix ARM/ARM64 mismatch in project file
cpython/3.10.14: Apply patch (portability): Remove duplicate libffi symbols and support shared libffi
cpython/3.10.14: Apply patch (conan): Allow package to be relocatable
cpython/3.10.14: Apply patch (conan): Remove section of solution file forcing projects to be built that might not be used for this recipe
cpython/3.10.14: RUN: "/Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth1c3bbd7425499/b/src/configure" --enable-shared --prefix=/Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth1c3bbd7425499/p '--bindir=${prefix}/bin' '--sbindir=${prefix}/bin' '--libdir=${prefix}/lib' '--includedir=${prefix}/include' '--oldincludedir=${prefix}/include' --host=x86_64-apple-darwin --build=aarch64-apple-darwin --with-doc-strings=yes --with-pymalloc=yes --with-system-expat --enable-optimizations=no --with-lto=no --with-pydebug=no --with-system-libmpdec --with-openssl=/Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/opensfed589a3aefa8/p --with-system-ffi --disable-test-modules --enable-loadable-sqlite-extensions=yes --with-build-python=/Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth0ecdc34d7f76c/p/bin/python --enable-ipv6 ac_cv_file__dev_ptmx=yes ac_cv_file__dev_ptc=no 
configure: WARNING: unrecognized options: --with-build-python
checking build system type... aarch64-apple-darwin
checking host system type... x86_64-apple-darwin
checking for python3.10... python3.10
checking for python interpreter for cross build... python3.10
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... configure: error: cross build not supported for x86_64-apple-darwin

cpython/3.10.14: ERROR: 
Package '832cf6f7ab6b91ef09d21447b2d2dfe1d8692894' build failed
cpython/3.10.14: WARN: Build folder /Users/jenkins/workspace/prod-v2/bsr/98254/afbee/p/b/cpyth1c3bbd7425499/b/build-release
ERROR: cpython/3.10.14: Error in build() method, line 562
	autotools.configure()
	ConanException: Error 1 while executing

cpython/3.8.19:
Didn't run or was cancelled before finishing
cpython/3.12.2:
Didn't run or was cancelled before finishing
cpython/3.11.9:
Didn't run or was cancelled before finishing
cpython/3.9.19:
Didn't run or was cancelled before finishing

^{Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.}

Ahajha · 2024-10-19T03:47:37Z

@valgur Seems that cpython doesn't support Mac cross compiles, but it does support universal builds (see python/cpython#90905). That might go into a rabbithole of needing to update dependencies to also support universal builds. Also see https://docs.conan.io/2/reference/tools/cmake/cmaketoolchain.html#support-for-universal-binaries-in-macos, somewhat relevant.

valgur added 2 commits October 3, 2024 11:49

cpython: enable macOS x86_64 by disabling broken deps

06aeb06

cpython: mark package_type as static-library for MSVC

1e83747

conan-center-bot added the Failed label Oct 3, 2024

This comment has been minimized.

Sign in to view

Ahajha reviewed Oct 4, 2024

View reviewed changes

recipes/cpython/all/conanfile.py Outdated Show resolved Hide resolved

cpython: fix package type

783662d

This comment has been minimized.

Sign in to view

Act as if package is in shared mode if self.options.shared doesn't exist

4d58275

This comment has been minimized.

Sign in to view

Ahajha mentioned this pull request Oct 13, 2024

cpython: add 3.13.1 #25536

Open

cpython: add cross-building support

6d8d46c

This was referenced Nov 3, 2024

nanobind: add a new recipe #20297

Open

CPython: Fix MSVC/all shared #25890

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cpython: avoid invalid configurations #25500

cpython: avoid invalid configurations #25500

valgur commented Oct 3, 2024

This comment has been minimized.

Ahajha commented Oct 4, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

Ahajha commented Oct 14, 2024

conan-center-bot commented Oct 14, 2024

Ahajha commented Oct 19, 2024 •

edited

Loading

cpython: avoid invalid configurations #25500

Are you sure you want to change the base?

cpython: avoid invalid configurations #25500

Conversation

valgur commented Oct 3, 2024

Summary

Motivation

Details

This comment has been minimized.

Ahajha commented Oct 4, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

Ahajha commented Oct 14, 2024

conan-center-bot commented Oct 14, 2024

Conan v1 pipeline ❌

Conan v2 pipeline ❌

Ahajha commented Oct 19, 2024 • edited Loading

Ahajha commented Oct 4, 2024 •

edited

Loading

Ahajha commented Oct 19, 2024 •

edited

Loading