Add pkg-config --libs tbb

[koheiw]

No description provided.

[kbenoit]

Thanks. But no, I still get

/Users/kbenoit/Dropbox (Personal)/GitHub/quanteda/quanteda.textstats/src/RcppExports.cpp updated.
/Users/kbenoit/Dropbox (Personal)/GitHub/quanteda/quanteda.textstats/R/RcppExports.R updated.
ℹ Re-compiling quanteda.textstats (debug build)
── R CMD INSTALL ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─  installing *source* package ‘quanteda.textstats’ ...
   ** using staged installation
   ** libs
   using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.3.9.4)’
   using SDK: ‘MacOSX14.4.sdk’
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I'/Users/kbenoit/Library/R/arm64/4.4/library/RcppArmadillo/include' -I'/Users/kbenoit/Library/R/arm64/4.4/library/quanteda/include' -I/opt/R/arm64/include   -DARMA_64BIT_WORD=1 `/Library/Frameworks/R.framework/Resources/bin/Rscript ../inst/deftbb.R` -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c RcppExports.cpp -o RcppExports.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I'/Users/kbenoit/Library/R/arm64/4.4/library/RcppArmadillo/include' -I'/Users/kbenoit/Library/R/arm64/4.4/library/quanteda/include' -I/opt/R/arm64/include   -DARMA_64BIT_WORD=1 `/Library/Frameworks/R.framework/Resources/bin/Rscript ../inst/deftbb.R` -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c collocations.cpp -o collocations.o
   In file included from collocations.cpp:1:
   In file included from /Users/kbenoit/Library/R/arm64/4.4/library/quanteda/include/lib.h:17:
   In file included from /usr/local/include/tbb/tbb.h:17:
   /usr/local/include/tbb/../oneapi/tbb.h:28:10: fatal error: 'oneapi/tbb/blocked_range.h' file not found
   #include "oneapi/tbb/blocked_range.h"
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
   1 error generated.
   make: *** [collocations.o] Error 1
   ERROR: compilation failed for package ‘quanteda.textstats’
─ 

but later this morning I will try it on my new office M2 mac and see if that works.

[kbenoit]

It's not just me however. CRAN accepted the new submission yesterday, and it's erroring on the mac builds:
https://cran.r-project.org/web/checks/check_results_quanteda.textstats.html

[koheiw]

I wonder how quanteda is compiled on your system. There must be -L option used to tell the linker where the library is.

[kbenoit]

quanteda compiles fiine.

── R CMD INSTALL ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─  installing *source* package ‘quanteda’ ...
   ** using staged installation
   checking whether the C++ compiler works... yes
   checking for C++ compiler default output file name... a.out
   checking for suffix of executables... 
   checking whether we are cross compiling... no
   checking for suffix of object files... o
   checking whether the compiler supports GNU C++... yes
   checking whether clang++ -arch arm64 -std=gnu++17 accepts -g... yes
   checking for clang++ -arch arm64 -std=gnu++17 option to enable C++11 features... none needed
   checking how to run the C++ preprocessor... clang++ -arch arm64 -std=gnu++17 -E
   checking for gcc... clang -arch arm64
   checking whether the compiler supports GNU C... yes
   checking whether clang -arch arm64 accepts -g... yes
   checking for clang -arch arm64 option to enable C11 features... none needed
   checking whether the compiler supports GNU C++... (cached) yes
   checking whether clang++ -arch arm64 -std=gnu++17 accepts -g... (cached) yes
   checking for clang++ -arch arm64 -std=gnu++17 option to enable C++11 features... (cached) none needed
   checking Intel TBB existence:... yes
   checking tbb available for compiling and linking:... yes
   configure: Package CPP flags: -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include
   configure: Package LIBS: -L/opt/homebrew/Cellar/tbb/2021.13.0/lib -ltbb
   configure: creating ./config.status
   config.status: creating src/Makevars
   ** libs
   using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.3.9.4)’
   using SDK: ‘MacOSX14.4.sdk’
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c RcppExports.cpp -o RcppExports.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c fcm.cpp -o fcm.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c index.cpp -o index.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c kwic.cpp -o kwic.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c pattern2fixed.cpp -o pattern2fixed.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c serialize.cpp -o serialize.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_chunk.cpp -o tokens_chunk.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_combine.cpp -o tokens_combine.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_compound.cpp -o tokens_compound.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_group.cpp -o tokens_group.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_lookup.cpp -o tokens_lookup.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_ngrams.cpp -o tokens_ngrams.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_replace.cpp -o tokens_replace.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_restore.cpp -o tokens_restore.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_segment.cpp -o tokens_segment.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_select.cpp -o tokens_select.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c tokens_xptr.cpp -o tokens_xptr.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Users/kbenoit/Library/R/arm64/4.4/library/Rcpp/include' -I/opt/R/arm64/include   -I../inst/include -DTBB -I/opt/homebrew/Cellar/tbb/2021.13.0/include -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c utility.cpp -o utility.o
   clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o quanteda.so RcppExports.o fcm.o index.o kwic.o pattern2fixed.o serialize.o tokens_chunk.o tokens_combine.o tokens_compound.o tokens_group.o tokens_lookup.o tokens_ngrams.o tokens_replace.o tokens_restore.o tokens_segment.o tokens_select.o tokens_xptr.o utility.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/gfortran/lib/gcc/aarch64-apple-darwin20.0/12.2.0 -L/opt/gfortran/lib -lgfortran -lemutls_w -lquadmath -L/opt/homebrew/Cellar/tbb/2021.13.0/lib -ltbb -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
   installing to /private/var/folders/rc/y2gv0_dn0z19ygq271w7f9dr0000gq/T/RtmpphqtMq/devtools_install_258742e04da5/00LOCK-quanteda/00new/quanteda/libs
   ** checking absolute paths in shared objects and dynamic libraries

[koheiw]

I see -L/opt/homebrew/Cellar/tbb/2021.13.0/lib -ltbb in your install log for quanteda. We could just copy configure.ac and Makevars.in to textstats, but good to know what is happening.

[kbenoit]

1d113c8 solves the build problem on Mac!

[koheiw]

I think TBB is simply disabled because there is no -ltbb. Makevars.in dose not work without the configure script.

[kbenoit]

Ah of course. The configure script ensures that TBB is correctly located on the Mac. So even though TBB is working for quanteda, the lack of the Makevars from quanteda.textstats means that the textstat_*() functions are all single-threaded? Does Makevars.win at least set multi-threading for Windows? What about Linux?

Do you think the solution is to use the same configuration scripts (pkgconfig) as we do for quanteda?

[koheiw]

If the package builds with -ltbb, multi-threading is enabled. On my Windows and Linux, multi-threading is enabled. We probably need to use configure and configure.ac here too for Mac.

[koheiw]

Is the configure script working on your Mac?

[kbenoit]

Yes, it works with TBB installed.

When I uninstall TBB, however, I get this:

  checking Intel TBB existence:... no
   configure: WARNING: Intel TBB not installed; install TBB devel package for parallel processing or update PKG_CONFIG_PATH environment variable
   configure: Package CPP flags: 
   configure: Package LIBS: 
   configure: creating ./config.status
   config.status: creating src/Makevars
   ** libs
   using C++ compiler: ‘Apple clang version 15.0.0 (clang-1500.3.9.4)’
   using SDK: ‘MacOSX14.4.sdk’
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppArmadillo/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/quanteda/include' -I/opt/R/arm64/include   -I../inst/include  -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c RcppExports.cpp -o RcppExports.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppArmadillo/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/quanteda/include' -I/opt/R/arm64/include   -I../inst/include  -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c collocations.cpp -o collocations.o
   clang++ -arch arm64 -std=gnu++17 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/Rcpp/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/RcppArmadillo/include' -I'/Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/library/quanteda/include' -I/opt/R/arm64/include   -I../inst/include  -fPIC  -falign-functions=64 -Wall -g -O2  -UNDEBUG -Wall -pedantic -g -O0 -fdiagnostics-color=always  -c keyness.cpp -o keyness.o
   clang++ -arch arm64 -std=gnu++17 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o quanteda.textstats.so RcppExports.o collocations.o keyness.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/gfortran/lib/gcc/aarch64-apple-darwin20.0/12.2.0 -L/opt/gfortran/lib -lgfortran -lemutls_w -lquadmath -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
   installing to /private/var/folders/ng/bv7v9vzd1p75vwz6r9x3fh280000gp/T/Rtmpi3VHf1/devtools_install_314a21d03ef/00LOCK-quanteda.textstats/00new/quanteda.textstats/libs
   ** checking absolute paths in shared objects and dynamic libraries
─  DONE (quanteda.textstats)
Error in load_imports(path) : 
  The package "quanteda" (>= 4.0.0) is required.
Calls: suppressPackageStartupMessages ... load_imports -> deps_check_installed -> check_installed
Execution halted

Exited with status 1.

Seems that it is still looking for something in the quanteda namespace that it's not finding.

[koheiw]

TBB is disabled properly as there is no -ltbb flag.

The package "quanteda" (>= 4.0.0) is required.

Which version of quanteda do you have installed 😕

[kbenoit]

4.1.0 of course! I thought it was a false positive message because it was not finding something it expected to find from the quanteda namespace. Could this be related to the removal of RcppArmadillo?

[koheiw]

It is unrelated to C++. You may have multiple locations to install R packages.

[kbenoit]

It is unrelated to C++. You may have multiple locations to install R packages.

It was the annoying dynLoad thing that needed resetting using devtools::document(). It's all working now, with and without TBB!

[kbenoit]

Building the package leaves config.log, config.status, and src/Makevars in my local project root. Isn't cleanup supposed to remove these? I made sure that the file is executable. This is not a problem with quanteda, which uses a similar (same?) setup.

[koheiw]

I copied the cleanup script too from quanteda.