I've an "Error with LightPipeline()" in Environment: Colab Spark 3.1.1 JSL 3.0.3. Spark NLP for Healthcare.

[JustHeroo]

Error with LightPipeline() in Environment: Colab Spark 3.1.1 JSL 3.0.3. Appreciate any pointers to resolve this issue. Thank you!
document = DocumentAssembler()
.setInputCol("TEXT")
.setOutputCol("document")
sentence = SentenceDetector()
.setInputCols('document')
.setOutputCol('sentence')
.setCustomBounds(['\n'])
embedding = BertSentenceEmbeddings.pretrained('sent_biobert_clinical_base_cased')
.setInputCols(["sentence"])
.setOutputCol("sentence_embeddings")
classsifierdl = ClassifierDLApproach()
.setInputCols(["sentence_embeddings"])
.setOutputCol("class")
.setLabelColumn("label")
.setMaxEpochs(5)
.setLr(0.001)
.setBatchSize(8)
.setEnableOutputLogs(True)
clf_pipeline = Pipeline(
stages = [
document,
sentence,
embedding,
classsifierdl
])
from sparknlp.base import LightPipeline
light_clfpipe = LightPipeline(clf_pipeline)
Py4JError Traceback (most recent call last)
in ()
1 from sparknlp.base import LightPipeline
2
----> 3 light_clfpipe = LightPipeline(clf_pipeline)
7 frames
/usr/local/lib/python3.7/dist-packages/py4j/protocol.py
in get_return_value(answer, gateway_client, target_id, name)
330 raise Py4JError(
331 "An error occurred while calling {0}{1}{2}. Trace:\n{3}\n".
--> 332 format(target_id, ".", name, value))
333 else:
334 raise Py4JError(

Py4JError: An error occurred while calling None.com.johnsnowlabs.nlp.LightPipeline. Trace:
py4j.Py4JException: Constructor com.johnsnowlabs.nlp.LightPipeline([class org.apache.spark.ml.Pipeline, class java.lang.Boolean]) does not exist
at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:179)
at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:196)
at py4j.Gateway.invoke(Gateway.java:237)
at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.base/java.lang.Thread.run(Thread.java:829)
(edited)

[JustHeroo]

First of all, you cannot feed the pipeline to Light pipeline as it needs to be fitted at first to become pipelineModel. You need to fit and then use the fitted clf stage in another pipeline to create a prediction pipeline and model and then feed to lightpipe.

check this out to learn more >> https://colab.research.google.com/github/JohnSnowLabs/spark-nlp-workshop/blob/master/tutorials/Certification_Trainings/Public/5.Text_Classification_with_ClassifierDL.ipynb