Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[jvm-packages] NoClassDefError for javax/tools/ToolProvider #4000

Closed
Vivraan opened this issue Dec 14, 2018 · 8 comments
Closed

[jvm-packages] NoClassDefError for javax/tools/ToolProvider #4000

Vivraan opened this issue Dec 14, 2018 · 8 comments

Comments

@Vivraan
Copy link

Vivraan commented Dec 14, 2018

System
Windows 10.0.17134
xgboost.dll successfully compiled with MSVC Compiler.

Log

building Java wrapper
cd ..
mkdir -p build
cd build
cmake .. -DUSE_OPENMP:BOOL=ON -DUSE_HDFS:BOOL=OFF -DUSE_AZURE:BOOL=OFF -DUSE_S3:BOOL=OFF -DPLUGIN_UPDATER_GPU:BOOL=OFF -DJVM_BINDINGS:BOOL=ON -G"Visual Studio 15 2017 Win64"
cmake --build . --config Release
cd demo\regression
C:\Users\shiva\AppData\Local\Programs\Python\Python36-32\python.exe mapfeat.py
C:\Users\shiva\AppData\Local\Programs\Python\Python36-32\python.exe mknfold.py machine.txt 1
copying native library
mkdir -p xgboost4j\src\main\resources\lib
cp ..\lib\xgboost4j.dll xgboost4j\src\main\resources\lib
copying pure-Python tracker
cp ..\dmlc-core\tracker\dmlc_tracker\tracker.py xgboost4j\src\main\resources
copying train/test files
mkdir -p xgboost4j-spark\src\test\resources
cd ..\demo\regression
C:\Users\shiva\AppData\Local\Programs\Python\Python36-32\python.exe mapfeat.py
C:\Users\shiva\AppData\Local\Programs\Python\Python36-32\python.exe mknfold.py machine.txt 1
cp ..\demo\regression\machine.txt.test xgboost4j-spark\src\test\resources
cp ..\demo\regression\machine.txt.train xgboost4j-spark\src\test\resources
cp ..\demo\data\agaricus.txt.test xgboost4j-spark\src\test\resources
cp ..\demo\data\agaricus.txt.train xgboost4j-spark\src\test\resources
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ xgboost4j ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost4j ---
[WARNING]  Expected all dependencies to require Scala version: 2.11.12
[WARNING]  com.typesafe.akka:akka-actor_2.11:2.3.11 requires scala version: 2.11.5
[WARNING] Multiple versions of scala libraries detected!
[INFO] C:\Program Files\xgboost\jvm-packages\xgboost4j\src\main\java:-1: info: compiling
[INFO] C:\Program Files\xgboost\jvm-packages\xgboost4j\src\main\scala:-1: info: compiling
[INFO] Compiling 24 source files to C:\Program Files\xgboost\jvm-packages\xgboost4j\target\classes at 1544818940529
Downloading from central_maven: https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.7/scala-library-2.11.7.jar
Downloaded from central_maven: https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.7/scala-library-2.11.7.jar (5.7 MB at 593 kB/s)
[ERROR] error: java.lang.NoClassDefFoundError: javax/tools/ToolProvider
[INFO]  at scala.reflect.io.JavaToolsPlatformArchive.iterator(ZipArchive.scala:301)
[INFO]  at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
[INFO]  at scala.reflect.io.AbstractFile.foreach(AbstractFile.scala:92)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.traverse(ClassPath.scala:277)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.x$15$lzycompute(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.x$15(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.packages$lzycompute(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:264)
[INFO]  at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358)
[INFO]  at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358)
[INFO]  at scala.collection.Iterator$class.foreach(Iterator.scala:891)
[INFO]  at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
[INFO]  at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
[INFO]  at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
[INFO]  at scala.tools.nsc.util.MergedClassPath.packages$lzycompute(ClassPath.scala:358)
[INFO]  at scala.tools.nsc.util.MergedClassPath.packages(ClassPath.scala:353)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply$mcV$sp(SymbolLoaders.scala:269)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260)
[INFO]  at scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:235)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader.doComplete(SymbolLoaders.scala:260)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:211)
[INFO]  at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1535)
[INFO]  at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:256)
[INFO]  at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73)
[INFO]  at scala.tools.nsc.Global.rootMirror(Global.scala:71)
[INFO]  at scala.tools.nsc.Global.rootMirror(Global.scala:39)
[INFO]  at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257)
[INFO]  at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257)
[INFO]  at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
[INFO]  at scala.tools.nsc.Global$Run.<init>(Global.scala:1242)
[INFO]  at scala.tools.nsc.Driver.doCompile(Driver.scala:31)
[INFO]  at scala.tools.nsc.MainClass.doCompile(Main.scala:23)
[INFO]  at scala.tools.nsc.Driver.process(Driver.scala:51)
[INFO]  at scala.tools.nsc.Driver.main(Driver.scala:64)
[INFO]  at scala.tools.nsc.Main.main(Main.scala)
[INFO]  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[INFO]  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[INFO]  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[INFO]  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[INFO]  at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
[INFO]  at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[INFO] java.lang.reflect.InvocationTargetException
[INFO]  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[INFO]  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[INFO]  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[INFO]  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[INFO]  at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
[INFO]  at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[ERROR] Caused by: java.lang.NoClassDefFoundError: javax/tools/ToolProvider
[INFO]  at scala.reflect.io.JavaToolsPlatformArchive.iterator(ZipArchive.scala:301)
[INFO]  at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
[INFO]  at scala.reflect.io.AbstractFile.foreach(AbstractFile.scala:92)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.traverse(ClassPath.scala:277)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.x$15$lzycompute(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.x$15(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.packages$lzycompute(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:299)
[INFO]  at scala.tools.nsc.util.DirectoryClassPath.packages(ClassPath.scala:264)
[INFO]  at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358)
[INFO]  at scala.tools.nsc.util.MergedClassPath$$anonfun$packages$1.apply(ClassPath.scala:358)
[INFO]  at scala.collection.Iterator$class.foreach(Iterator.scala:891)
[INFO]  at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
[INFO]  at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
[INFO]  at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
[INFO]  at scala.tools.nsc.util.MergedClassPath.packages$lzycompute(ClassPath.scala:358)
[INFO]  at scala.tools.nsc.util.MergedClassPath.packages(ClassPath.scala:353)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply$mcV$sp(SymbolLoaders.scala:269)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader$$anonfun$doComplete$1.apply(SymbolLoaders.scala:260)
[INFO]  at scala.reflect.internal.SymbolTable.enteringPhase(SymbolTable.scala:235)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$PackageLoader.doComplete(SymbolLoaders.scala:260)
[INFO]  at scala.tools.nsc.symtab.SymbolLoaders$SymbolLoader.complete(SymbolLoaders.scala:211)
[INFO]  at scala.reflect.internal.Symbols$Symbol.info(Symbols.scala:1535)
[INFO]  at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:256)
[INFO]  at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73)
[INFO]  at scala.tools.nsc.Global.rootMirror(Global.scala:71)
[INFO]  at scala.tools.nsc.Global.rootMirror(Global.scala:39)
[INFO]  at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257)
[INFO]  at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257)
[INFO]  at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
[INFO]  at scala.tools.nsc.Global$Run.<init>(Global.scala:1242)
[INFO]  at scala.tools.nsc.Driver.doCompile(Driver.scala:31)
[INFO]  at scala.tools.nsc.MainClass.doCompile(Main.scala:23)
[INFO]  at scala.tools.nsc.Driver.process(Driver.scala:51)
[INFO]  at scala.tools.nsc.Driver.main(Driver.scala:64)
[INFO]  at scala.tools.nsc.Main.main(Main.scala)
[INFO]  ... 6 more
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for XGBoost JVM Package 0.82-SNAPSHOT:
[INFO]
[INFO] XGBoost JVM Package ................................ SUCCESS [ 38.418 s]
[INFO] xgboost4j .......................................... FAILURE [21:57 min]
[INFO] xgboost4j-spark .................................... SKIPPED
[INFO] xgboost4j-flink .................................... SKIPPED
[INFO] xgboost4j-example .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  22:36 min
[INFO] Finished at: 2018-12-15T01:52:43+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (default) on project xgboost4j: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: -10000 (Exit value: -10000) -> [Help 1]

Reproduce with

xgboost\build> cmake .. -G"Visual Studio 15 2017 Win64" -DUSE_CUDA=ON -DUSE_OPENMP=ON; cd ..\jvm-packages\
xgboost\jvm-packages> mvn -DskipTests clean package install

The default maybe_generator in create_jni.py (line 83 for me) has been changed to the required version, -G"Visual Studio 15 2017 Win64" before proceeding with the maven install command. Successful completion before "Building Java Wrapper" has been omitted.

@Vivraan Vivraan changed the title [jvm-package] NoClassDefError for javax/tools/ToolsProvider [jvm-package] NoClassDefError for javax/tools/ToolProvider Dec 15, 2018
@Vivraan Vivraan changed the title [jvm-package] NoClassDefError for javax/tools/ToolProvider [jvm-packages] NoClassDefError for javax/tools/ToolProvider Dec 15, 2018
@Craigacp
Copy link
Contributor

What JVM are you using to compile it? It's at least Java 9, but is it 9, 10 or 11?

@Vivraan
Copy link
Author

Vivraan commented Dec 19, 2018

Ah, I completely forgot... should've used a template. ('._.)
It's 11.0.1.

Found a lot of issues online that suggested adding -nc and -nobootcp so I tried adding those options to scala-maven-plugin, but without much effect. Where exactly should I put these arguments?

@Craigacp
Copy link
Contributor

Craigacp commented Dec 19, 2018

Adding

                <configuration>
                  <args>
                    <arg>-nobootcp</arg>
                  </args>
                </configuration>

to the bottom of the scala-maven-plugin config fixed it for me on Linux. I don't have easy access to a configured Windows box to check it on that platform, but I suspect this will be sufficient.

@Vivraan
Copy link
Author

Vivraan commented Dec 20, 2018

Apparently previously I had added this detail well nested under maven-site-plugin, having mistaken its <reportPlugins> version of scala-maven-plugin.

Seems like the build has started. I'll close this for now.

@Vivraan Vivraan closed this as completed Dec 20, 2018
@Vivraan Vivraan reopened this Dec 20, 2018
@Vivraan
Copy link
Author

Vivraan commented Dec 20, 2018

Haha, doesn't seem to be leaving me (without of course using one of the pre-built JARs) 😛

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
        at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
Caused by: java.lang.RuntimeException: Class java/util/UUID could not be instrumented.
        at org.jacoco.agent.rt.internal_8ff85ea.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:140)
        at org.jacoco.agent.rt.internal_8ff85ea.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:101)
        at org.jacoco.agent.rt.internal_8ff85ea.PreMain.createRuntime(PreMain.java:55)
        at org.jacoco.agent.rt.internal_8ff85ea.PreMain.premain(PreMain.java:47)
        ... 6 more
Caused by: java.lang.NoSuchFieldException: $jacocoAccess
        at java.base/java.lang.Class.getField(Class.java:2000)
        at org.jacoco.agent.rt.internal_8ff85ea.core.runtime.ModifiedSystemClassRuntime.createFor(ModifiedSystemClassRuntime.java:138)
        ... 9 more
FATAL ERROR in native method: processing of -javaagent failed

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for XGBoost JVM Package 0.82-SNAPSHOT:
[INFO]
[INFO] XGBoost JVM Package ................................ SUCCESS [ 15.672 s]
[INFO] xgboost4j .......................................... FAILURE [01:22 min]
[INFO] xgboost4j-spark .................................... SKIPPED
[INFO] xgboost4j-flink .................................... SKIPPED
[INFO] xgboost4j-example .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:38 min
[INFO] Finished at: 2018-12-20T12:28:47+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project xgboost4j: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was cmd.exe /X /C ""C:\Program Files\Java\jdk-11.0.1\bin\java" "-javaagent:C:\\Users\\shiva\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.7.9\\org.jacoco.agent-0.7.9-runtime.jar=destfile=C:\\Program Files\\xgboost\\jvm-packages\\xgboost4j\\target\\jacoco.exec" org.apache.maven.surefire.booter.ForkedBooter "C:\Program Files\xgboost\jvm-packages\xgboost4j\target\surefire\surefire13689783208094078316tmp" "C:\Program Files\xgboost\jvm-packages\xgboost4j\target\surefire\surefire_017480945474061054799tmp""
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :xgboost4j

@Craigacp
Copy link
Contributor

Bump the jacoco-maven-plugin version to 0.8.2. That has support for Java 11 class files.

Then you get an error out of scalatest which I think requires bumping everything in the stack up beyond Java 9, and I'm not sure how to do that (see plasma-umass/doppio#497). I tried to bump the class file version to Java 9, but that caused a different error in ScalaTest (because it insists on picking up a Java 8 JDK from somewhere on my path - though you might not get this issue).

I'd just skip the tests, it'll probably be fine.

@Vivraan
Copy link
Author

Vivraan commented Dec 20, 2018

... But I did skip the tests 😱

@CodingCat
Copy link
Member

I don't think we have gone so far to support anything beyond JDK8

@lock lock bot locked as resolved and limited conversation to collaborators Apr 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants