java.lang.NoSuchFieldError: EMPTY_BYTE_ARRAY

[haksunlinm]

Hello all,

I think there may be a potential bug in kotlin-jupyter-kernel.

I installed a jupyter lab with kotlin-jupyter-kernel and I added a 3rd party jar to call the functions in the jar.

It works fine on kotlin-jupyter-kernel version 0.11.0.45.

The output is as follows.

However, if I use any version above that, e.g., 0.11.0.61, 0.11.0.140, 0.11.0.208, 0.11.0.385, it does not work.
So, I suspect that there might be something changed/broken after version 0.11.0.45.

The error is as follows.

EMPTY_BYTE_ARRAY
java.lang.NoSuchFieldError: EMPTY_BYTE_ARRAY
at org.apache.logging.log4j.core.config.ConfigurationSource.(ConfigurationSource.java:56)
at org.apache.logging.log4j.core.config.NullConfiguration.(NullConfiguration.java:32)
at org.apache.logging.log4j.core.LoggerContext.(LoggerContext.java:85)
at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.createContext(ClassLoaderContextSelector.java:254)
at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.locateContext(ClassLoaderContextSelector.java:218)
at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:136)
at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:123)
at org.apache.logging.log4j.core.selector.ClassLoaderContextSelector.getContext(ClassLoaderContextSelector.java:117)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:150)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:47)
at org.apache.logging.log4j.LogManager.getContext(LogManager.java:194)
at org.apache.logging.log4j.LogManager.getLogger(LogManager.java:581)
at org.jzy3d.chart.factories.ChartFactory.(ChartFactory.java:37)
at Line_23.chart(Line_23.jupyter-kts:14)
at Line_23.(Line_23.jupyter-kts:45)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at kotlin.script.experimental.jvm.BasicJvmScriptEvaluator.evalWithConfigAndOtherScriptsResults(BasicJvmScriptEvaluator.kt:105)
at kotlin.script.experimental.jvm.BasicJvmScriptEvaluator.invoke$suspendImpl(BasicJvmScriptEvaluator.kt:47)
at kotlin.script.experimental.jvm.BasicJvmScriptEvaluator.invoke(BasicJvmScriptEvaluator.kt)
at kotlin.script.experimental.jvm.BasicJvmReplEvaluator.eval(BasicJvmReplEvaluator.kt:49)
at org.jetbrains.kotlinx.jupyter.repl.impl.InternalEvaluatorImpl$eval$resultWithDiagnostics$1.invokeSuspend(InternalEvaluatorImpl.kt:103)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
at org.jetbrains.kotlinx.jupyter.repl.impl.InternalEvaluatorImpl.eval(InternalEvaluatorImpl.kt:103)
at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$result$1.invoke(CellExecutorImpl.kt:71)
at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$result$1.invoke(CellExecutorImpl.kt:69)
at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withHost(repl.kt:635)
at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:69)
at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:15)
at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:444)
at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$evalEx$1.invoke(repl.kt:433)
at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:397)
at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.evalEx(repl.kt:433)
at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:485)
at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$2$res$1.invoke(protocol.kt:321)
at org.jetbrains.kotlinx.jupyter.messaging.ProtocolKt$shellMessagesHandler$2$res$1.invoke(protocol.kt:320)
at org.jetbrains.kotlinx.jupyter.JupyterExecutorImpl$runExecution$execThread$1.invoke(execution.kt:33)
at org.jetbrains.kotlinx.jupyter.JupyterExecutorImpl$runExecution$execThread$1.invoke(execution.kt:31)
at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)

The code to reproduce the problem is as follows.

@file:Repository("http://maven.jzy3d.org/releases")
@file:DependsOn("org.jzy3d:jzy3d-emul-gl:2.0.0")

import org.jzy3d.chart.Chart
import org.jzy3d.chart.EmulGLSkin
import org.jzy3d.chart.factories.EmulGLChartFactory
import org.jzy3d.colors.Color
import org.jzy3d.maths.Coord3d
import org.jzy3d.plot3d.primitives.*
import org.jzy3d.plot3d.rendering.canvas.*

val q: Quality = Quality.Advanced()
q.setAnimated(false)
q.setHiDPIEnabled(true) // need java 9+ to enable HiDPI & Retina displays
val chart: Chart = EmulGLChartFactory().newChart(q)

chart

You can run the code here to see the error:
http://35.223.178.227:8888/tree?token=5d6c7622a457c14fcad438dbe89ff57be4a564021d23e07e

Any help would be greatly appreciated! Thank you!

[haksunlinm]

I translated the code into Java and ran it without Jupyter and it works fine.

[haksunlinm]

I am doing some more testing.

This fails on some jupyter versions. It works on version 0.11.0.45.

@file:Repository("https://repo1.maven.org/maven2")
@file:DependsOn("org.apache.logging.log4j:log4j-api:2.20.0")
var c = org.apache.logging.log4j.util.Constants.EMPTY_BYTE_ARRAY
c

This works on all versions.

@file:Repository("https://repo1.maven.org/maven2")
@file:DependsOn("org.apache.logging.log4j:log4j-api:2.20.0")

var d = org.apache.logging.log4j.util.Constants.JAVA_MAJOR_VERSION
d

This works on all versions.

@file:Repository("https://repo1.maven.org/maven2")
@file:DependsOn("org.apache.logging.log4j:log4j-api:2.20.0")

var e = org.apache.logging.log4j.util.Constants.LOG4J2_DEBUG
e

I am thinking maybe there is some conflict of the binaries of the logging code somewhere...
Anyone got any idea please?

[haksunlinm]

My biggest problem is that org.apache.logging.log4j.util.Constants.EMPTY_BYTE_ARRAY sometimes can be accessed (after hitting SHIFT+ENTER multiple times) but sometimes can't. So, I am not sure whether this is a jar problem or a kotlin-kernel problem.

Can someone give me a pointer to know where to look, please?

[haksunlinm]

I suppose Kotlin-kernel uses some form of log4j for the logging mechanism, which may be using a different log4j-api jar.
Is there a way to check exactly which jar it is using when I call the command;
org.apache.logging.log4j.util.Constants.EMPTY_BYTE_ARRAY

One hypothesis is that it sometimes using the jar that I specified (version 2.20.0) but sometimes use the jar that comes with kotlin-jupyter which might be older and does not have that particular field. I am not sure why it would use a different jar each time that I hit SHIFT+ENTER though.

But the behavior of each execution is not consistent.

[ileasile]

Hi! Sorry for the delay. I can't reproduce the issue, and your server is unavailable (I would check classpath first). In the latest version of kernel, log4j 2.17.1 is used. But maybe in some of the previous ones, pre-2.15 is used, and conflict happens indeed...

[haksunlinm]

Hello Ilya, thank you for your message. Sorry for the late reply as I had been away. Can you try this new server/link please?
https://s22.nm.dev/hub/user-redirect/lab/tree/s2-public/Examples/Plotting/jzy3d/jzy3d_surface_chart.ipynb

Can you let me know how I may check the class path, please?
I checked the .m2 folder and the correct log4j version is there. No pre-2.15 version is the .m2 directory. Are there other paths that I should check please?

I built this jupyterhub instance using BASE_IMAGE=jupyterhub/singleuser and installing the latest kotlin jupyter kernel using pip install kotlin-jupyter-kernel. I am not exactly sure where and how I can find and remove an older version of log4j.

Can you provide more details, please? Your help is much appreciated.

[haksunlinm]

As for the code in that link,
https://s22.nm.dev/hub/user-redirect/lab/tree/s2-public/Examples/Plotting/jzy3d/jzy3d_surface_chart.ipynb

If you run the first cell and it succeeds, then the code in the second cell always succeeds. That is what we want.

However, if you do not run the first cell, or the first cell fails, then the second cell always fails because it loads the incorrect version of the log4j.

[haksunlinm]

[haksunlinm]

Hello, ileasile,

Can you provide more information, please?
That would be much appreciated.

Thanks!

[phiSgr]

how I may check the class path

kotlin-jupyter/docs/README.md

Line 249 in 4faf9b3

- `:classpath` - show current classpath