[WIP] Add part of the Keras API

[Avasil]

@shadaj
Hi, I added small part of Keras API - enough to solve MNIST example.
It is not working yet and I have lots of questions though :D
I skipped a lot of functionality but I hope it could be a good starting point.

When I run MnistExample it fails at runtime with:

[info] running (fork) me.shadaj.scalapy.tensorflow.MnistExample 
[info] x_train shape: List(60000, 28, 28, 1)
[info] 60000 train samples
[info] 10000 test samples
[error] 2020-04-06 19:09:48.182338: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[error] 2020-04-06 19:09:48.209693: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7ffc757f73b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
[error] 2020-04-06 19:09:48.209710: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
[error] Exception in thread "main" me.shadaj.scalapy.py.PythonException: <class 'TypeError'> 'NoneType' object is not iterable
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.$anonfun$throwErrorIfOccured$1(CPythonInterpreter.scala:110)
[error] 	at me.shadaj.scalapy.py.Platform$.Zone(Platform.scala:10)
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.throwErrorIfOccured(CPythonInterpreter.scala:96)
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.$anonfun$load$1(CPythonInterpreter.scala:119)
[error] 	at me.shadaj.scalapy.py.Platform$.Zone(Platform.scala:10)
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.load(CPythonInterpreter.scala:116)
[error] 	at me.shadaj.scalapy.py.package$.eval(package.scala:84)
[error] 	at me.shadaj.scalapy.py.AnyDynamics.applyDynamicNamed(Dynamic.scala:17)
[error] 	at me.shadaj.scalapy.py.AnyDynamics.applyDynamicNamed$(Dynamic.scala:14)
[error] 	at me.shadaj.scalapy.tensorflow.keras.layers.Layers$$anon$1$$anon$2.applyDynamicNamed(Layers.scala:1)
[error] 	at me.shadaj.scalapy.tensorflow.keras.layers.Layers.Conv2D(Layers.scala:1)
[error] 	at me.shadaj.scalapy.tensorflow.keras.layers.Layers.Conv2D$(Layers.scala:7)
[error] 	at me.shadaj.scalapy.tensorflow.keras.Keras$$anon$27$$anon$28.Conv2D(Keras.scala:20)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample$.delayedEndpoint$me$shadaj$scalapy$tensorflow$MnistExample$1(MnistExample.scala:53)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample$delayedInit$body.apply(MnistExample.scala:9)
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:392)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample$.main(MnistExample.scala:9)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample.main(MnistExample.scala)
[error] Nonzero exit code returned from runner: 1
[error] (scalaPyTensorFlowCrossJVM / Test / runMain) Nonzero exit code returned from runner: 1
[error] Total time: 9 s, completed Apr 6, 2020, 7:09:48 PM

I think it happens when creating second Conv2D layer.

I used local version of scalapy-numpy with these changes: shadaj/scalapy-numpy@master...Avasil:shape-reshape (needed shape and reshape` for the example)

Advice/help would be appreciated!

[Avasil]

input_shape should be provided via **kwargs (https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) but I didn't know how to do it with scalapy

[shadaj]

Hmm, this is something I need to work on supporting, probably with an annotation. In the meantime, you should be able to handle this by calling this.as[py.Dynamic].Conv2D(..., input_shape=input_shape, ...) since the dynamic API does support kwargs.

[Avasil]

I tried the following in dynamic API:

val models2 = py.module("tensorflow").keras.models
val layers2 = py.module("tensorflow").keras.layers

val model = models2.Sequential() // kerasA.models.as[py.Dynamic].Sequential() // kerasA.models.Sequential()
model.add(layers2.Conv2D(filters = 32, kernel_size = (3, 3), activation = "relu", input_shape = (28, 28, 1)))

And it fails with:

[error] Exception in thread "main" me.shadaj.scalapy.py.PythonException: <class 'TypeError'> Dimension value must be integer or None or have an __index__ method, got 32.0
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.$anonfun$throwErrorIfOccured$1(CPythonInterpreter.scala:110)
[error] 	at me.shadaj.scalapy.py.Platform$.Zone(Platform.scala:10)
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.throwErrorIfOccured(CPythonInterpreter.scala:96)
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.$anonfun$load$1(CPythonInterpreter.scala:119)
[error] 	at me.shadaj.scalapy.py.Platform$.Zone(Platform.scala:10)
[error] 	at me.shadaj.scalapy.py.CPythonInterpreter$.load(CPythonInterpreter.scala:116)
[error] 	at me.shadaj.scalapy.py.package$.eval(package.scala:84)
[error] 	at me.shadaj.scalapy.py.AnyDynamics.applyDynamicNamed(Dynamic.scala:17)
[error] 	at me.shadaj.scalapy.py.AnyDynamics.applyDynamicNamed$(Dynamic.scala:14)
[error] 	at me.shadaj.scalapy.py.package$$anon$11$$anon$12.applyDynamicNamed(package.scala:84)
[error] 	at me.shadaj.scalapy.py.AnyDynamics.applyDynamic(Dynamic.scala:11)
[error] 	at me.shadaj.scalapy.py.AnyDynamics.applyDynamic$(Dynamic.scala:8)
[error] 	at me.shadaj.scalapy.py.package$$anon$11$$anon$12.applyDynamic(package.scala:84)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample$.delayedEndpoint$me$shadaj$scalapy$tensorflow$MnistExample$1(MnistExample.scala:56)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample$delayedInit$body.apply(MnistExample.scala:10)
[error] 	at scala.Function0.apply$mcV$sp(Function0.scala:39)
[error] 	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
[error] 	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
[error] 	at scala.App.$anonfun$main$1$adapted(App.scala:80)
[error] 	at scala.collection.immutable.List.foreach(List.scala:392)
[error] 	at scala.App.main(App.scala:80)
[error] 	at scala.App.main$(App.scala:78)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample$.main(MnistExample.scala:10)
[error] 	at me.shadaj.scalapy.tensorflow.MnistExample.main(MnistExample.scala)

It casts 32 to Double somewhere. I didn't have this issue in previous versions (with JEP) so perhaps there is a bug on master?