[R-package] installation via install.packages fails if /bin/sh is not bash (bashisms in configure)

[smoser]

Description

An attempt to install lightgbm in an environment that has dash or some other posix compatible shell as /bin/sh will fail.

$ R -e "install.packages('lightgbm', version = '4.5.0', repos = 'https://cran.rstudio.com/')"
* installing *source* package 'lightgbm' ...
** package 'lightgbm' successfully unpacked and MD5 sums checked
** using staged installation
checking location of R... /usr/lib/R
checking whether MM_PREFETCH works... yes
./configure: line 1792: LGB_CPPFLAGS+= -DMM_PREFETCH=1: not found
checking whether MM_MALLOC works... yes
./configure: line 1827: LGB_CPPFLAGS+= -DMM_MALLOC=1: not found
configure: creating ./config.status                                                                    
config.status: creating src/Makevars     
** libs
using C++ compiler: 'g++ (Wolfi 14.2.0-r5) 14.2.0'                                 15:48:45 [1827/7522]
using C++17                                        
g++ -std=gnu++17 -I"/usr/include/R" -DNDEBUG -I./include -DEIGEN_MPL2_ONLY -DEIGEN_DONT_PARALLELIZE -DU
SE_SOCKET -DLGB_R_BUILD  -I/usr/local/include   -fopenmp -pthread -fpic  -g -O2   -c boosting/boosting.
cpp -o boosting/boosting.o                                                                             
In file included from ./include/LightGBM/config.h:21,                                                  
                 from ./include/LightGBM/boosting.h:8,                                                 
                 from boosting/boosting.cpp:5:
./include/LightGBM/utils/common.h:53:26: error: 'void* malloc(size_t)' was declared 'extern' and later 
'static' [-fpermissive]
   53 | #define _mm_malloc(a, b) malloc(a)
      |                          ^~~~~~
In file included from /usr/include/c++/14/cstdlib:79,
                 from /usr/include/c++/14/ext/string_conversions.h:43,
                 from /usr/include/c++/14/bits/basic_string.h:4154,
                 from /usr/include/c++/14/string:54,
                 from ./include/LightGBM/utils/json11.h:59,
                 from ./include/LightGBM/utils/common.h:8:
/usr/include/stdlib.h:672:14: note: previous declaration of 'void* malloc(size_t)'
  672 | extern void *malloc (size_t __size) __THROW __attribute_malloc__
      |              ^~~~~~
In file included from /usr/lib/gcc/x86_64-pc-linux-gnu/14/include/xmmintrin.h:34,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/14/include/emmintrin.h:31,
                 from /usr/lib/gcc/x86_64-pc-linux-gnu/14/include/pmmintrin.h:31,
                 from /usr/include/c++/14/x86_64-pc-linux-gnu/bits/opt_random.h:34,
                 from /usr/include/c++/14/random:49,
                 from ./include/LightGBM/utils/random.h:9,
                 from ./include/LightGBM/feature_group.h:11,
                 from ./include/LightGBM/dataset.h:10,
                 from ./include/LightGBM/objective_function.h:9,
                 from boosting/gbdt.h:9,
                 from boosting/dart.hpp:16,
                 from boosting/boosting.cpp:7:
/usr/lib/gcc/x86_64-pc-linux-gnu/14/include/mm_malloc.h: In function 'void* malloc(size_t)':
/usr/lib/gcc/x86_64-pc-linux-gnu/14/include/mm_malloc.h:41:7: error: '__alignment' was not declared in 
this scope
   41 |   if (__alignment == 1)
      |       ^~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/14/include/mm_malloc.h:43:7: error: '__alignment' was not declared in 
this scope
   43 |   if (__alignment == 2 || (sizeof (void *) == 8 && __alignment == 4))
      |       ^~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/14/include/mm_malloc.h:45:31: error: '__alignment' was not declared in
 this scope
   45 |   if (posix_memalign (&__ptr, __alignment, __size) == 0)
      |                               ^~~~~~~~~~~
/usr/lib/gcc/x86_64-pc-linux-gnu/14/include/mm_malloc.h: At global scope:
./include/LightGBM/utils/common.h:54:21: error: 'void free(void*)' was declared 'extern' and later 'sta
tic' [-fpermissive]
   54 | #define _mm_free(a) free(a)
      |                     ^~~~
/usr/include/stdlib.h:687:13: note: previous declaration of 'void free(void*)'
  687 | extern void free (void *__ptr) __THROW;
      |             ^~~~

make: *** [/usr/lib/R/etc/Makeconf:204: boosting/boosting.o] Error 1
ERROR: compilation failed for package 'lightgbm'

Reproducible example

Environment info

Reproduced in wolfi environment, but will reproduce anywhere where /bin/sh is not bash.

LightGBM version or commit hash: 4.5.0

Command(s) you used to install LightGBM

R -e "install.packages('lightgbm', version = '4.5.0', repos = 'https://cran.rstudio.com/')"

Additional Comments

Assignment with '+=' is not posix shell.
From the configure script:

#! /bin/sh
...
# line 1826
if test "${ac_mm_malloc}" = yes; then
   LGB_CPPFLAGS+=" -DMM_MALLOC=1"
fi

[jameslamb]

Thanks very much for the report!

I'm very surprised that CRAN has not complained about this before... I know it checks scripts for POSIX compliance with checkbashisms.

ref: https://github.com/wch/r-source/blob/872770ededfc26429adca09fab0803e1b78b6eac/doc/manual/R-ints.texi#L4377-L4385

Would you like to submit a pull request to fix this?

[smoser]

Would you like to submit a pull request to fix this?

#6746

[bastistician]

Great, thanks for finding this! My Alpine Linux system showed the same malloc compilation error when trying to install.packages("lightgbm"). I had found this error in another issue so had only left a comment there (but I now marked as off-topic). In the end, the installation failure on Alpine Linux was simply caused by this configure problem meaning that -DMM_PREFETCH=1 -DMM_MALLOC=1 were not set for compilation. Now I can successfully install the current development version on Alpine Linux. It would be great if this fix could be released on CRAN. Thanks!

[jameslamb]

Thanks for taking the time to write that and to go back and mark that other comment off-topic.

We will not be able to get a release up to CRAN before Monday (saw your comment at fabsig/GPBoost#151 (comment) that they'll close for 2 weeks after that), but can try for one in January. There are many other fixes (many unrelated to the R package) that have piled up unreleased here in the last few months, so I was hoping to do that anyway.