Compile-isssue when installing via pip

[Ultrahero]

I just wanted to install the package on a new system, other modules like pyscipopt were already installed via pip, when building gcg, i got the following traceback: (I use a conda env, but installed my other packages with pip in there, because some packages were not registered on anaconda, only on pypi)

Collecting pygcgopt
  Using cached PyGCGOpt-0.3.1.tar.gz (363 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: wheel in /home/sg114224/miniconda3/envs/dtadvenv/lib/python3.11/site-packages (from pygcgopt) (0.41.2)
Collecting pyscipopt>=4.4.0 (from pygcgopt)
  Using cached PySCIPOpt-5.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.8 kB)
Using cached PySCIPOpt-5.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.9 MB)
Building wheels for collected packages: pygcgopt
  Building wheel for pygcgopt (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [30 lines of output]
      Using include path </w0/tmp/sg114224/login18-3_174311/pip-install-f3dc1dvp/pygcgopt_c1f585a186bf4137a729c6c0cb6f27b2/src, /usr/include/gcg>.
      Using SCIP library <scip> at </w0/tmp/sg114224/login18-3_174311/pip-install-f3dc1dvp/pygcgopt_c1f585a186bf4137a729c6c0cb6f27b2/lib>.
      Using GCG library <gcg> at </w0/tmp/sg114224/login18-3_174311/pip-install-f3dc1dvp/pygcgopt_c1f585a186bf4137a729c6c0cb6f27b2/lib>.
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/__init__.py -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/util.py -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/gcg.pyx -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/gcg.pxd -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/decomposition.pxi -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/detector.pxi -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/detprobdata.pxi -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/partition.pxi -> build/lib.linux-x86_64-cpython-311/pygcgopt
      copying src/pygcgopt/pricing_solver.pxi -> build/lib.linux-x86_64-cpython-311/pygcgopt
      running build_ext
      building 'pygcgopt.gcg' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/src
      creating build/temp.linux-x86_64-cpython-311/src/pygcgopt
      icc -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/sg114224/miniconda3/envs/dtadvenv/include -fPIC -O2 -isystem /home/sg114224/miniconda3/envs/dtadvenv/include -fPIC -I/w0/tmp/sg114224/login18-3_174311/pip-install-f3dc1dvp/pygcgopt_c1f585a186bf4137a729c6c0cb6f27b2/src -I/usr/include/gcg -I/home/sg114224/miniconda3/envs/dtadvenv/include/python3.11 -c src/pygcgopt/gcg.cpp -o build/temp.linux-x86_64-cpython-311/src/pygcgopt/gcg.o
      src/pygcgopt/gcg.cpp(1244): catastrophic error: cannot open source file "scip/type_retcode.h"
        #include "scip/type_retcode.h"
                                      ^
      
      compilation aborted for src/pygcgopt/gcg.cpp (code 4)
      error: command '/cvmfs/software.hpc.rwth.de/Linux/RH8/x86_64/intel/skylake_avx512/software/intel-compilers/2022.1.0/compiler/2022.1.0/linux/bin/intel64/icc' failed with exit code 4
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pygcgopt
  Running setup.py clean for pygcgopt
Failed to build pygcgopt
ERROR: Could not build wheels for pygcgopt, which is required to install pyproject.toml-based projects

This happened with both pyscipopt installed and not installed.
The installation did work via conda, which updated some standard libraries:

The following packages will be UPDATED:

  libgcc-ng          pkgs/main::libgcc-ng-11.2.0-h1234567_1 --> conda-forge::libgcc-ng-13.2.0-h807b86a_5 
  libgomp              pkgs/main::libgomp-11.2.0-h1234567_1 --> conda-forge::libgomp-13.2.0-h807b86a_5 
  libstdcxx-ng       pkgs/main::libstdcxx-ng-11.2.0-h12345~ --> conda-forge::libstdcxx-ng-13.2.0-h7e041cc_5 
  openssl              pkgs/main::openssl-3.0.13-h7f8727e_0 --> conda-forge::openssl-3.2.1-hd590300_1 
  zlib                    pkgs/main::zlib-1.2.13-h5eee18b_0 --> conda-forge::zlib-1.2.13-hd590300_5 

The following packages will be SUPERSEDED by a higher-priority channel:

  _libgcc_mutex           pkgs/main::_libgcc_mutex-0.1-main --> conda-forge::_libgcc_mutex-0.1-conda_forge 
  _openmp_mutex          pkgs/main::_openmp_mutex-5.1-1_gnu --> conda-forge::_openmp_mutex-4.5-2_gnu

It also did install other stuff but I assume, this was already there, just from pip and used only by pip so conda installed their own. For completion here is that list as well:

The following NEW packages will be INSTALLED:

  ampl-mp            conda-forge/linux-64::ampl-mp-3.1.0-h2cc385e_1006 
  cliquer            conda-forge/linux-64::cliquer-1.22-h36c2ea0_0 
  cppad              conda-forge/linux-64::cppad-20240000.2-h59595ed_0 
  gcg                conda-forge/linux-64::gcg-3.5.5-hd47b8d6_7 
  gmp                conda-forge/linux-64::gmp-6.3.0-h59595ed_1 
  icu                conda-forge/linux-64::icu-73.2-h59595ed_0 
  ipopt              conda-forge/linux-64::ipopt-3.14.14-h04b96a2_1 
  libblas            conda-forge/linux-64::libblas-3.9.0-21_linux64_openblas 
  libcblas           conda-forge/linux-64::libcblas-3.9.0-21_linux64_openblas 
  libedit            conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2 
  libgfortran-ng     conda-forge/linux-64::libgfortran-ng-13.2.0-h69a702a_5 
  libgfortran5       conda-forge/linux-64::libgfortran5-13.2.0-ha4646dd_5 
  libhwloc           conda-forge/linux-64::libhwloc-2.9.3-default_h554bfaf_1009 
  libiconv           conda-forge/linux-64::libiconv-1.17-hd590300_2 
  liblapack          conda-forge/linux-64::liblapack-3.9.0-21_linux64_openblas 
  libopenblas        conda-forge/linux-64::libopenblas-0.3.26-pthreads_h413a1c8_0 
  libscotch          conda-forge/linux-64::libscotch-7.0.4-h91e35bf_1 
  libspral           conda-forge/linux-64::libspral-2023.09.07-h6aa6db2_2 
  libxml2            conda-forge/linux-64::libxml2-2.12.6-h232c23b_1 
  libzlib            conda-forge/linux-64::libzlib-1.2.13-hd590300_5 
  metis              conda-forge/linux-64::metis-5.1.0-h59595ed_1007 
  mumps-include      conda-forge/linux-64::mumps-include-5.6.2-ha770c72_4 
  mumps-seq          conda-forge/linux-64::mumps-seq-5.6.2-hfef103a_4 
  pygcgopt           conda-forge/linux-64::pygcgopt-0.3.1-py311hb755f60_1 
  pyscipopt          conda-forge/linux-64::pyscipopt-4.4.0-py311hb755f60_1 
  python_abi         conda-forge/linux-64::python_abi-3.11-2_cp311 
  scip               conda-forge/linux-64::scip-8.1.0-h41ab4b8_7 
  scotch             conda-forge/linux-64::scotch-7.0.4-h23d43cc_1 
  tbb                conda-forge/linux-64::tbb-2021.11.0-h00ab1b0_1 
  unixodbc           conda-forge/linux-64::unixodbc-2.3.12-h661eb56_0 

Now this may be just a problem of not up-to-date libraries and conda uses their own, while pip uses the system's libraries, but maybe you want to include some check or similar.

[jurgen-lentz]

We are aware of this. pyscipopt builds now directly scip in pip since version 5.0.0. Unfortunately, you need to stick with pyscipopt 4.4.0 until we fix it. Maybe fixing pyscipopt to 4.4.0 on conda will be possible. PR are welcomed https://github.com/conda-forge/pygcgopt-feedstock !