Spring Cloud Dataflow on Kubernetes - GKE

[sumitbansal88]

Description:
I am hosting SCDF on GKE by following below documentation -
https://dataflow.spring.io/docs/installation/kubernetes/kubectl/

PODS for all components are running but getting below error with SCDF server:

2024-08-08 03:22:40.081 WARN 1 --- [ main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'deleteTaskService' defined in class path resource [org/springframework/cloud/dataflow/server/config/features/TaskConfiguration$TaskDeleteServiceConfig.class]: Unsatisfied dependency expressed through method 'deleteTaskService' parameter 11; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'schedulerService' defined in class path resource [org/springframework/cloud/dataflow/server/config/features/SchedulerConfiguration.class]: Unsatisfied dependency expressed through method 'schedulerService' parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'kubernetesTaskPlatform' defined in class path resource [org/springframework/cloud/dataflow/server/config/kubernetes/KubernetesTaskPlatformAutoConfiguration.class]: Unsatisfied dependency expressed through method 'kubernetesTaskPlatform' parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'kubernetesTaskPlatformFactory' defined in class path resource [org/springframework/cloud/dataflow/server/config/kubernetes/KubernetesTaskPlatformAutoConfiguration.class]: Unsatisfied dependency expressed through method 'kubernetesTaskPlatformFactory' parameter 0; nested exception is org.springframework.boot.context.properties.ConfigurationPropertiesBindException: Error creating bean with name 'spring.cloud.dataflow.task.platform.kubernetes-org.springframework.cloud.dataflow.server.config.kubernetes.KubernetesPlatformProperties': Could not bind properties to 'KubernetesPlatformProperties' : prefix=spring.cloud.dataflow.task.platform.kubernetes, ignoreInvalidFields=false, ignoreUnknownFields=true; nested exception is org.springframework.boot.context.properties.bind.BindException: Failed to bind properties under 'spring.cloud.dataflow.task.platform.kubernetes.accounts.default.fabric8' to io.fabric8.kubernetes.client.Config
2024-08-08 03:22:40.087 INFO 1 --- [ main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2024-08-08 03:22:40.091 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2024-08-08 03:22:40.099 INFO 1 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
2024-08-08 03:22:40.101 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2024-08-08 03:22:40.165 ERROR 1 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter :

---

APPLICATION FAILED TO START

---

Description:

Failed to bind properties under 'spring.cloud.dataflow.task.platform.kubernetes.accounts.default.fabric8' to io.fabric8.kubernetes.client.Config:

Reason: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.String] to type [@org.springframework.boot.context.properties.NestedConfigurationProperty io.fabric8.kubernetes.client.Config]

Action:

Release versions:
My Kubernetes version is 1.29.6 and installing 2.11.4 dataflow and skipper. If I install below image the POD is successfull run but not able to run any tasks -

docker pull springcloud/spring-cloud-dataflow-server:latest

[cppwfs]

Can you provide us a log file?

Thank you!

[corneil]

@sumitbansal88 How are you deploying the components? At what point does it fail?

[sumitbansal88]

@cppwfs @corneil - I am deploying the components using kubectl and consuming manifests from:

https://github.com/spring-cloud/spring-cloud-dataflow/tree/main/src/kubernetes

I am successfully able to host the mariadb, rabbitmq, prometheus, grafana and skipper on GKE 1.29.6. While trying to run the dataflow server image the conatiner keeps crashing with below error:

Image Used: https://hub.docker.com/layers/springcloud/spring-cloud-dataflow-server/2.11.4-SNAPSHOT/images/sha256-9b172f83275c8c32250b618db7236a4267ab7c18a05f24d83e10d51b2dd936fa?context=explore

I have tried different 2.11.x versions and tried.

---

APPLICATION FAILED TO START

---

Description:

Failed to bind properties under 'spring.cloud.dataflow.task.platform.kubernetes.accounts.default.fabric8' to io.fabric8.kubernetes.client.Config:

Reason: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.String] to type [@org.springframework.boot.context.properties.NestedConfigurationProperty io.fabric8.kubernetes.client.Config]

Action:

Update your application's configuration

Attaching Log file -
server.txt

Manifests files -
server-m.txt

[sumitbansal88]

@corneil @cppwfs I am able to find a workaround if i remove below piece of code from my configuration -

management:
metrics:
export:
prometheus:
enabled: true
rsocket:
enabled: true
host: prometheus-rsocket-proxy
port: 7001

But I need all my metrics exported to prometheus and Grafana.

[corneil]

@sumitbansal88 I would like to suggest you try using the Carvel package installation.
The relevant scripts and readme are in https://github.com/spring-cloud/spring-cloud-dataflow/releases/download/v2.11.4/spring-cloud-dataflow-package-2.11.4.zip

In short it provides for configuration of database, broker and other properties and then deploys the application. Our acceptance tests uses this mechanism every day.

[corneil]

From the error message it seems you have added a property with name starting with spring.cloud.dataflow.task.platform.kubernetes.accounts.default.fabric8 which is not valid.
Your deployment file had 2 entries named application.yml please combine that into one entry.

apiVersion: v1
kind: ConfigMap
metadata:
  name: scdf-server
  labels:
    app: scdf-server
data:
  application.yaml: |
    management:
      metrics:
        export:
          prometheus:
            enabled: true
            rsocket:
              enabled: true
              host: prometheus-rsocket-proxy
              port: 7001
    spring:
      cloud:
        dataflow:
          metrics.dashboard:
            url: 'http://35.223.206.9:3000'
          task:
            platform:
              kubernetes:
                accounts:                  
                  default:
                    limits:
                      memory: 1024Mi
      datasource:
        url: jdbc:mariadb://${MARIADB_SERVICE_HOST}:${MARIADB_SERVICE_PORT}/mysql
        username: root
        password: ${database-password}
        driverClassName: org.mariadb.jdbc.Driver
        testOnBorrow: true
        validationQuery: "SELECT 1"

Stick with 2.11.4 which is the latest release.

[corneil]

@sumitbansal88 Can you confirm the remove of empty fabric8 element from yaml has fixed to issue?