tidyr install errors OS/X Catalina

[mik3hall]

Can you reproduce with the latest development build?

Yes

• If you explicitly used different CRAN mirror than the default, please include its URL.
  Also tried CRAN mirror (tidyr 1.1.3) besides MRAN default (tidyr 1.1.2).

  • Use $GRAALVM_HOME/bin/R --vm.version and include the full output.
    openjdk version "11.0.11" 2021-04-20
    OpenJDK Runtime Environment GraalVM CE 21.2.0-dev (build 11.0.11+8-jvmci-21.2-b02)
    OpenJDK 64-Bit Server VM GraalVM CE 21.2.0-dev (build 11.0.11+8-jvmci-21.2-b02, mixed mode, sharing)

• Output of R built-in function sessionInfo().

sessionInfo()
FastR version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin19.6.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS: /Library/Java/JavaVirtualMachines/graalvm-ce-java11-21.2.0-dev/Contents/Home/languages/R/lib/libRblas.dylib
LAPACK: /Library/Java/JavaVirtualMachines/graalvm-ce-java11-21.2.0-dev/Contents/Home/languages/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached):
[1] tools_4.0.3

• OS name and version.
  As shown in sessionInfo()

Optionally: try to reduce the error

Package installation consists of several steps. Run the installation with the
following options to turn all those steps off and then try removing the
--no-{XYZ} options one by one to determine, which step causes the issue.

install.packages('Rcpp', INSTALL_opts='--no-R --no-libs --no-help --no-data --no-demo --no-exec --no-test-load')

Eliminating just -no-test-load is enough to fail...

install.packages('tidyr', INSTALL_opts='--no-R --no-libs --no-help --no-data --no-demo --no-exec',verbose=TRUE,keep_output=TRUE)
Installing package into ‘/Users/mjh/Documents/R/fastr’
(as ‘lib’ is unspecified)
system (cmd0): /Library/Java/JavaVirtualMachines/graalvm-ce-java11-21.2.0-dev/Contents/Home/languages/R/bin/R CMD INSTALL --no-R --no-libs --no-help --no-data --no-demo --no-exec
Content type 'application/octet-stream' length 881544 bytes (860 KB)
foundpkgs: tidyr, /var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/RtmpF0U18f/downloaded_packages/tidyr_1.1.2.tar.gz
files: /var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/RtmpF0U18f/downloaded_packages/tidyr_1.1.2.tar.gz

• installing source package ‘tidyr’ ...
  ** package ‘tidyr’ successfully unpacked and MD5 sums checked
  ** using staged installation
  ** inst
  ** building package indices
  ** installing vignettes
  ** testing if installed package can be loaded from temporary location
  Warning: S3 methods ‘complete.data.frame’, ‘complete.default’, ‘complete_.data.frame’, ‘drop_na.data.frame’, ‘drop_na.default’, ‘drop_na_.data.frame’, ‘expand.data.frame’, ‘expand.default’, ‘expand.grouped_df’, ‘expand_.data.frame’, ‘extract.data.frame’, ‘extract.default’, ‘extract_.data.frame’, ‘fill.data.frame’, ‘fill.default’, ‘fill_.data.frame’, ‘full_seq.Date’, ‘full_seq.POSIXct’, ‘full_seq.numeric’, ‘gather.data.frame’, ‘gather.default’, ‘gather_.data.frame’, ‘nest.data.frame’, ‘nest.grouped_df’, ‘nest.tbl_df’, ‘nest_legacy.data.frame’, ‘nest_legacy.tbl_df’, ‘pivot_longer.data.frame’, ‘pivot_wider.data.frame’, ‘replace_na.data.frame’, ‘replace_na.default’, ‘separate.data.frame’, ‘separate.default’, ‘separate_.data.frame’, ‘separate_rows.data.frame’, ‘separate_rows.default’, ‘separate_rows_.data.frame’, ‘spread.data.frame’, ‘spread.default’, ‘spread_.data.frame’, ‘unite.data.frame’, ‘unite.default’, ‘unite_.data.frame’, ‘unnest.data.frame’, ‘unnest.rowwise_df’, ‘unnest_legacy.data.frame’ were declared in NAMESPACE but not found
  Error: package or namespace load failed for ‘tidyr’ in library.dynam(lib, package, package.lib):
  shared object ‘tidyr.so’ not found
  Error: loading failed
  ERROR: loading failed
• removing ‘/Users/mjh/Documents/R/fastr/tidyr’
• restoring previous ‘/Users/mjh/Documents/R/fastr/tidyr’

The downloaded source packages are in
‘/private/var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/RtmpF0U18f/downloaded_packages’
Warning message:
In install.packages("tidyr", INSTALL_opts = "--no-R --no-libs --no-help --no-data --no-demo --no-exec", :
installation of package ‘tidyr’ had non-zero exit status

But this misses messages in normal verbose installation...

• installing source package ‘tidyr’ ...
  ** package ‘tidyr’ successfully unpacked and MD5 sums checked
  ** using staged installation
  ** libs
  "/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang++" -std=gnu++11 -I"/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/include" -DNDEBUG -I'/Users/mjh/Documents/R/fastr/cpp11/include' -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c cpp11.cpp -o cpp11.o
  "/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang++" -std=gnu++11 -I"/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/include" -DNDEBUG -I'/Users/mjh/Documents/R/fastr/cpp11/include' -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c fill.cpp -o fill.o
  "/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang++" -std=gnu++11 -I"/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/include" -DNDEBUG -I'/Users/mjh/Documents/R/fastr/cpp11/include' -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c melt.cpp -o melt.o
  "/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang++" -std=gnu++11 -I"/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/include" -DNDEBUG -I'/Users/mjh/Documents/R/fastr/cpp11/include' -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c simplifyPieces.cpp -o simplifyPieces.o
  /Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang++ -std=gnu++11 -Wl,-rpath,/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/lib/,-rpath,/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/../llvm/native/lib,-undefined,dynamic_lookup -L/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/lib -undefined dynamic_lookup -L/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/../llvm/native/lib -lf2c -dynamiclib -Wl,-headerpad_max_install_names -single_module -multiply_defined suppress -lR -Wl,-rpath,/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/lib/,-rpath,/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/../llvm/native/lib,-undefined,dynamic_lookup -L/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/lib -L/Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/../llvm/native/lib -dynamiclib -lf2c -o tidyr.so cpp11.o fill.o melt.o simplifyPieces.o
  installing to /Users/mjh/Documents/R/fastr/00LOCK-tidyr/00new/tidyr/libs
  ** R
  ** data
  *** moving datasets to lazyload DB
  ** inst
  ** byte-compile and prepare package for lazy loading
  Error in get(paste0(generic, ".", class), envir = get_method_env()) :
  object 'vec_ptype2.tbl_df.tbl_df' not found
  Error in get(paste0(generic, ".", class), envir = get_method_env()) :
  object 'vec_restore.grouped_df' not found
  ** help
  *** installing help indices
  *** copying figures
  ** building package indices
  ** installing vignettes
  ** testing if installed package can be loaded from temporary location
  Error in get(paste0(generic, ".", class), envir = get_method_env()) :
  object 'vec_ptype2.tbl_df.tbl_df' not found
  Error in get(paste0(generic, ".", class), envir = get_method_env()) :
  object 'vec_restore.grouped_df' not found
  Error in polyglot evaluation : dlopen(/Users/mjh/Documents/R/fastr/00LOCK-tidyr/00new/tidyr/libs/tidyr.so, 6): Symbol not found: _SET_GROWABLE_BIT
  Referenced from: /Users/mjh/Documents/R/fastr/00LOCK-tidyr/00new/tidyr/libs/tidyr.so
  Expected in: flat namespace
  in /Users/mjh/Documents/R/fastr/00LOCK-tidyr/00new/tidyr/libs/tidyr.so
  ERROR: loading failed
• removing ‘/Users/mjh/Documents/R/fastr/tidyr’
• restoring previous ‘/Users/mjh/Documents/R/fastr/tidyr’

This was run from my application but R command gets same results. 'tidyr' was required as a dependency for the 'caret' package. I also had to name change for some caret dependency as mentioned in
#184
I already made that change to my JavaVirtualMachines installed GraalVM but had to make the changes again for my application embedded JRE. I also had some dependency indicate it couldn't find gfortran even though it should be in PATH...
which gfortran
/usr/local/bin/gfortran
So I made that path the FC value in Makeconf

[steve-s]

Thank you for the report. It looks like we are missing the SET_GROWABLE_BIT C API function.

[mik3hall]

There do seem to be differences.
For me this...

/* Growable vector support */
#define GROWABLE_MASK ((unsigned short)(1<<5))
#define GROWABLE_BIT_SET(x) ((x)->sxpinfo.gp & GROWABLE_MASK)
#define SET_GROWABLE_BIT(x) (((x)->sxpinfo.gp) |= GROWABLE_MASK)
#define IS_GROWABLE(x) (GROWABLE_BIT_SET(x) && XLENGTH(x) < XTRUELENGTH(x))

in /Library/Frameworks/R.framework/Versions/4.0/PrivateHeaders/Rinternals.h
is not in ~/Documents/GitHub/fastr/com.oracle.truffle.r.native/gnur/patch/src/include/Rinternals.h

it seems to be referenced in
https://github.com/r-lib/cpp11/blob/master/inst/include/cpp11/r_vector.hpp
but not in
~/Library/R/4.0/library/cpp11/include/cpp11/r_vector.hpp

[steve-s]

Tricky thing about Rinternals.h is that half of this is effective or not effective depending on whether USE_RINTERNALS was defined. Those macros:

/* Growable vector support */
#define GROWABLE_MASK ((unsigned short)(1<<5))
#define GROWABLE_BIT_SET(x) ((x)->sxpinfo.gp & GROWABLE_MASK)
#define SET_GROWABLE_BIT(x) (((x)->sxpinfo.gp) |= GROWABLE_MASK)
#define IS_GROWABLE(x) (GROWABLE_BIT_SET(x) && XLENGTH(x) < XTRUELENGTH(x))

are defined only if USE_RINTERNALS is defined and packages should not do that (it is meant only for internal usage). If USE_RINTERNALS is not defined, then we have:

/* Growable vector support */      
int (IS_GROWABLE)(SEXP x);
(SET_GROWABLE_BIT)(SEXP x);

which are prototypes of regular C functions, which we however do not implement in FastR yet.

[mik3hall]

As near as I can tell in the Gnu R source these come from memory.c which fastr doesn't appear to include.

/* Growable vector support */
int (IS_GROWABLE)(SEXP x) { return IS_GROWABLE(CHK(x)); }
void (SET_GROWABLE_BIT)(SEXP x) { SET_GROWABLE_BIT(CHK(x)); }

tidyr doesn't seem itself to do anything with Rinternals maybe through some other package like cpp11? I tried reinstall that no difference.

I was curious about the other errors as well...
Error in get(paste0(generic, ".", class), envir = get_method_env()) :
object 'vec_ptype2.tbl_df.tbl_df' not found

These seemed to come from the vctrs package. I tried reinstalling that which generates more exceptions on the tidyr install. I restarted my app and was back to the initial errors so that seems to only have had something to do with re-installing vctrs on the fly. The exceptions follow if of interest.
But, no tidyr workaround seems readily available. So no caret. I was hoping to do some benchmarking of R machine learning algorithms. For now I may have to use something simpler like randomForest which I think I did use successfully.

---

tidyr install following vctrs re-install

---

R> install.packages("vctrs")
Installing package into ‘/Users/mjh/Documents/R/fastr’
(as ‘lib’ is unspecified)
Content type 'application/octet-stream' length 778016 bytes (759 KB)

The downloaded source packages are in
‘/private/var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/RtmpeSylyy/downloaded_packages’
R> install.packages("tidyr",verbose=TRUE,keep_output=TRUE)
Installing package into ‘/Users/mjh/Documents/R/fastr’
(as ‘lib’ is unspecified)
system (cmd0): /Users/mjh/HalfPipe/HalfPipe_jpkg/HPGraal11/HalfPipeGraal11.app/Contents/runtime/Contents/Home/languages/R/bin/R CMD INSTALL
Content type 'application/octet-stream' length 881544 bytes (860 KB)
foundpkgs: tidyr, /var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/RtmpeSylyy/downloaded_packages/tidyr_1.1.2.tar.gz
files: /var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/RtmpeSylyy/downloaded_packages/tidyr_1.1.2.tar.gz

• installing source package ‘tidyr’ ...
  ** package ‘tidyr’ successfully unpacked and MD5 sums checkedAn internal error occurred.
  Please report an issue at https://github.com/oracle/fastr including the commands. You can rerun FastR with --R.PrintErrorStacktracesToFile=true to turn on internal errors logging. Please attach the log file to the issue if possible.
  org.graalvm.polyglot.PolyglotException: com.oracle.truffle.r.runtime.RInternalError: java.lang.NullPointerException
  at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:342)
  at writeLines(Unknown)
  at install.packages(unknown.r:1:0-29)
  at (:1:0-54)
  at null(:1:0-54)
  at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:353)
  at us.hall.fastr.SimpleTest.main(SimpleTest.java:48)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  at org.cmdline.psuedoGestalt.Runner.invoke(Runner.java:209)
  at org.cmdline.psuedoGestalt.Runner.runStatic(Runner.java:236)
  at org.cmdline.psuedoGestalt.Runner.runMain(Runner.java:228)
  at org.cmdline.psuedoGestalt.Runner.run(Runner.java:146)
  Original Internal Error:
  com.oracle.truffle.r.runtime.RInternalError: java.lang.NullPointerException
  at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:342)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:613)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:584)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:534)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:518)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:481)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
  at com.oracle.truffle.r.nodes.function.call.CallRFunctionNode.execute(CallRFunctionNode.java:63)
  at com.oracle.truffle.r.nodes.function.RCallNode$DispatchedCallNode.execute(RCallNode.java:1233)
  at com.oracle.truffle.r.nodes.function.RCallNode$FunctionDispatch.dispatch(RCallNode.java:920)
  at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.execute(RCallNodeGen.java:692)
  at com.oracle.truffle.r.nodes.function.RCallNode.call(RCallNode.java:295)
  at com.oracle.truffle.r.nodes.function.RCallNodeGen.execute(RCallNodeGen.java:168)
  at com.oracle.truffle.r.runtime.nodes.RNode.voidExecute(RNode.java:66)
  at com.oracle.truffle.r.nodes.control.IfNode.voidExecute(IfNode.java:69)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:79)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:35)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedBlockNode.executeVoid(OptimizedBlockNode.java:120)
  at com.oracle.truffle.r.nodes.control.RBlockNode.voidExecute(RBlockNode.java:57)
  at com.oracle.truffle.r.nodes.control.ForNode$AbstractIndexRepeatingNode.executeRepeating(ForNode.java:258)
  at org.graalvm.truffle/com.oracle.truffle.api.nodes.RepeatingNode.executeRepeatingWithValue(RepeatingNode.java:107)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedOSRLoopNode.profilingLoop(OptimizedOSRLoopNode.java:172)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedOSRLoopNode.execute(OptimizedOSRLoopNode.java:124)
  at com.oracle.truffle.r.nodes.control.ForNode.iterate(ForNode.java:94)
  at com.oracle.truffle.r.nodes.control.ForNodeGen.execute(ForNodeGen.java:60)
  at com.oracle.truffle.r.runtime.nodes.RNode.voidExecute(RNode.java:66)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:79)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:35)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedBlockNode.executeVoid(OptimizedBlockNode.java:120)
  at com.oracle.truffle.r.nodes.control.RBlockNode.voidExecute(RBlockNode.java:57)
  at com.oracle.truffle.r.nodes.control.IfNode.voidExecute(IfNode.java:72)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:79)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:35)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedBlockNode.executeVoid(OptimizedBlockNode.java:120)
  at com.oracle.truffle.r.nodes.control.RBlockNode.voidExecute(RBlockNode.java:57)
  at com.oracle.truffle.r.nodes.control.IfNode.voidExecute(IfNode.java:69)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:79)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeVoid(RBlockNode.java:35)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedBlockNode.executeGeneric(OptimizedBlockNode.java:79)
  at com.oracle.truffle.r.nodes.control.RBlockNode.visibleExecute(RBlockNode.java:62)
  at com.oracle.truffle.r.nodes.function.FunctionBodyNode.visibleExecute(FunctionBodyNode.java:67)
  at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:292)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:613)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:584)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:534)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:518)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:481)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
  at com.oracle.truffle.r.nodes.function.call.CallRFunctionNode.execute(CallRFunctionNode.java:63)
  at com.oracle.truffle.r.nodes.function.RCallNode$DispatchedCallNode.execute(RCallNode.java:1233)
  at com.oracle.truffle.r.nodes.function.RCallNode$FunctionDispatch.dispatch(RCallNode.java:920)
  at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.executeAndSpecialize(RCallNodeGen.java:738)
  at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.execute(RCallNodeGen.java:702)
  at com.oracle.truffle.r.nodes.function.RCallNode.call(RCallNode.java:295)
  at com.oracle.truffle.r.nodes.function.RCallNodeGen.executeAndSpecialize(RCallNodeGen.java:235)
  at com.oracle.truffle.r.nodes.function.RCallNodeGen.execute(RCallNodeGen.java:210)
  at com.oracle.truffle.r.runtime.nodes.RNode.visibleExecute(RNode.java:74)
  at com.oracle.truffle.r.engine.REngine$AnonymousBodyNode.visibleExecute(REngine.java:741)
  at com.oracle.truffle.r.engine.REngine$AnonymousRootNode.execute(REngine.java:668)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:613)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:584)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:534)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:518)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:481)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
  at com.oracle.truffle.r.engine.EngineRootNode$EngineBodyNode.execute(EngineRootNode.java:150)
  at com.oracle.truffle.r.engine.EngineRootNode.execute(EngineRootNode.java:88)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:613)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:584)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:534)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:518)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callIndirect(OptimizedCallTarget.java:463)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.call(OptimizedCallTarget.java:444)
  at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.eval(PolyglotContextImpl.java:1050)
  at org.graalvm.sdk/org.graalvm.polyglot.Context.eval(Context.java:353)
  at us.hall.fastr.SimpleTest.main(SimpleTest.java:48)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  at org.cmdline.psuedoGestalt.Runner.invoke(Runner.java:209)
  at org.cmdline.psuedoGestalt.Runner.runStatic(Runner.java:236)
  at org.cmdline.psuedoGestalt.Runner.runMain(Runner.java:228)
  at org.cmdline.psuedoGestalt.Runner.run(Runner.java:146)
  Caused by: java.lang.NullPointerException
  at java.base/java.util.StringTokenizer.(StringTokenizer.java:199)
  at java.base/java.util.StringTokenizer.(StringTokenizer.java:221)
  at org.cmdline.cmdIO.CmdOutThread.appendText(CmdOutThread.java:37)
  at org.cmdline.cmdIO.TextInterfaceOut.write(TextInterfaceOut.java:166)
  at org.cmdline.cmdIO.CmdOut.write(CmdOut.java:306)
  at org.graalvm.truffle/com.oracle.truffle.api.impl.DispatchOutputStream.write(DispatchOutputStream.java:132)
  at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233)
  at java.base/sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:312)
  at java.base/sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:316)
  at java.base/sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:153)
  at java.base/java.io.OutputStreamWriter.flush(OutputStreamWriter.java:251)
  at com.oracle.truffle.r.runtime.context.RContext$ConsoleIO.print(RContext.java:1099)
  at com.oracle.truffle.r.runtime.conn.StdConnections$StdoutConnection.writeString(StdConnections.java:319)
  at com.oracle.truffle.r.runtime.conn.StdConnections$StdoutConnection.writeLines(StdConnections.java:299)
  at com.oracle.truffle.r.nodes.builtin.base.ConnectionFunctions$WriteLines.writeLines(ConnectionFunctions.java:692)
  at com.oracle.truffle.r.nodes.builtin.base.ConnectionFunctionsFactory$WriteLinesNodeGen.execute(ConnectionFunctionsFactory.java:1344)
  at com.oracle.truffle.r.nodes.builtin.RBuiltinNode$Arg4.call(RBuiltinNode.java:227)
  at com.oracle.truffle.r.nodes.builtin.InternalNode$InternalCallNode.execute(InternalNode.java:260)
  at com.oracle.truffle.r.runtime.nodes.RNode.visibleExecute(RNode.java:74)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeGeneric(RBlockNode.java:87)
  at com.oracle.truffle.r.nodes.control.RBlockNode.executeGeneric(RBlockNode.java:35)
  at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedBlockNode.executeGeneric(OptimizedBlockNode.java:81)
  at com.oracle.truffle.r.nodes.control.RBlockNode.visibleExecute(RBlockNode.java:62)
  at com.oracle.truffle.r.nodes.function.FunctionBodyNode.visibleExecute(FunctionBodyNode.java:67)
  at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:292)
  ... 83 more
  Caused by: Attached Guest Language Frames (4)
  R>

[steve-s]

There are some known issues with vctrs and tibble that we are in process of fixing. We hope to get the fixes into the upcoming release.

But, no tidyr workaround seems readily available. So no caret. I was hoping to do some benchmarking of R machine learning algorithms. For now I may have to use something simpler like randomForest which I think I did use successfully.

A word of caution here. We understand the importance of caret and tried installing/running it on FastR several times in the past with various results. In general caret with all its (transitive) dependencies is such a large library that I must admit it is out of our reach at this point and in foreseeable future. Maybe some parts of it can work, but as far as I understand its purpose, you really want to be able to run multiple models/methods on your data with caret and the breadth is just too large for FastR at this point.

Additionally, most of the models are, I assume, going to be implemented in C/C++ and FastR cannot provide any performance advantage over that. We just execute the same C/C++ code as GNU-R. Even worse, the transitions between C/C++ execution and JVM are more costly than in GNU-R, so I do not expect great results from such comparison. What is FastR better at is pure R code execution, for example, if there is some model that is implemented in pure R there I am sure we'd shine.

[mik3hall]

I was looking for a few good R classifiers to use in benchmarking. caret's random forest was mentioned as a good implementation of that.
tidyr and the whole tidyverse is of course very popular and commonly used in a lot of R that I have seen. I saw past FastR issues where it seemed to have at least built so something must of broke since then. tidyr working itself might be more generally important than caret.
Maybe some of this isn't ready for benchmarking yet, or I can keep looking for implementations that work. Some might not support multiple commonly used implementations like Random Forest does though.
I have looked a little bit at a couple java to R interfaces and for one thing it seemed to me that having to move data from one to the other was a bit of a bottleneck where FastR I thought might be expected to perform much better. How it compares to Gnu R might also be interesting to see.

[steve-s]

We have experimental option --BackEnd=llvm, which makes FastR use the GraalVM LLVM runtime [0] to execute the C/C++ code. This avoids any data transformation to native memory and allows the Truffle compiler to optimize the code across the R and C boundary. This can be interesting and has the potential to be even better than GNU-R. However, it is highly experimental option and at this point we do not have the resources to push this project further.

[0] https://www.graalvm.org/reference-manual/llvm/

[mik3hall]

I'm not quite sure how if this doesn't provide the missing symbol in the original error it will correct it?
I did try it...
fR --BackEnd=llvm
...
Warning: Loading of '/Users/mjh/Documents/R/fastr/rlang/libs/rlang.so' in LLVM mode failed. You may load this package via the native mode by adding it to /Library/Java/JavaVirtualMachines/graalvm-ce-java11-21.2.0-dev/Contents/Home/languages/R/etc/native-packages or by running FastR with option --R.BackEndNative=packageName.
An internal error occurred.
Please report an issue at https://github.com/oracle/fastr including the commands. You can rerun FastR with --R.PrintErrorStacktracesToFile=true to turn on internal errors logging. Please attach the log file to the issue if possible.

[mik3hall]

fR --BackEnd=llvm --R.BackEndNative=rlang

This seems to get back to the original error.

[steve-s]

I'm not quite sure how if this doesn't provide the missing symbol in the original error it will correct it?

No, sorry, I did not mean that. I was referring to benchmarking of caret and similar packages on FastR in general (provided they even install). With these options you can expect better performance from native R extensions execution after sufficient warmup.

[mik3hall]

Ah, ok. I was thinking about maybe waiting on that until some of the related packages were working. I was considering including the Weka application in the benchmark that mainly uses the mlr package to run R classifiers. fastr had issues with that #189. I would hope to make the comparisons in as similar a way as possible and it is seeming that could be difficult.