Failed to find any class that implements Connector aws-sqs-sink-coonector

[L040639]

025-01-27 14:12:36,083 ERROR Uncaught exception in REST call to /connectors/my-source-connector/config (org.apache.kafka.connect.runtime.rest.errors.ConnectExceptionMapper) [qtp1945836927-27]
org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches org.apache.camel.kafkaconnector.awssqssink.CamelAwssqssinkSinkConnector, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='3.0.0', encodedVersion=3.0.0, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='3.0.0', encodedVersion=3.0.0, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector

I am trying add the camel aws sqs sink connnector to strimzi kafka connect but eventhough i have added the plugin correctly, getting above error i.e connector was not added .

@scholzj

Custom Image: -

FROM quay.io/strimzi/kafka:latest-kafka-3.0.0

USER root:root

# Create the plugins directory
RUN mkdir -p /opt/kafka/plugins

##copied camel aws sqs plugins .
COPY ./plugins/ /opt/kafka/plugins/


USER 1001

Kafkaconnect and connector configuration:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata: 
  name: my-connect-cluster
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  version: 3.7.0
  replicas: 1
  image: repo/<xxx>
  bootstrapServers: <xxxxx>
  tls:
    trustedCertificates:
      - secretName: lk-ca-root-bundle
        certificate: ca.crt
  logging:
    type: inline
    loggers:
      log4j.rootLogger: INFO
  readinessProbe: # (13)
    initialDelaySeconds: 15
    timeoutSeconds: 5
  livenessProbe:
    initialDelaySeconds: 15
    timeoutSeconds: 5
  config:
    group.id: <xxx>
    offset.storage.topic: connectoffsets
    config.storage.topic: connectconfigs
    status.storage.topic: connectstatus
    config.storage.replication.factor: 3
    offset.storage.replication.factor: 3
    status.storage.replication.factor: 3
    template:
    pod:
      metadata:
        annotations:
          eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxx:role/xxxxx
      imagePullSecrets:
        - name: kafka-artifactory
      terminationGracePeriodSeconds: 120
    serviceAccount:
      metadata:
        annotations:
          eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxx:role/xxxxx
  authentication:
    type: oauth
    clientId: xxxxxxxx
    clientSecret:
      key: secret
      secretName: <>
    scope: 'api://<>/.default'
    tokenEndpointUri: 'https://login.microsoftonline.com/<>/oauth2/v2.0/token'

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: my-source-connector
  labels:
    strimzi.io/cluster: my-connect-cluster
spec:
  class: org.apache.camel.kafkaconnector.awssqssink.CamelAwssqssinkSinkConnector
  config:
    topics: "testtopic"
    camel.kamelet.aws-sqs-sink.queueNameOrArn: "arn:aws:sqs:us-east-2:<xxx>:kafkaconnecttest"
    camel.kamelet.aws-sqs-sink.useSessionCredentials: "true"
    camel.kamelet.aws-sqs-sink.region: "us-east-2"

[scholzj]

I guess you would need to provide more details about how are you building the image and what are you adding to it and share full logs.

But one things that is definitely wrong is FROM quay.io/strimzi/kafka:latest-kafka-3.0.0 -> if you use Kafka 3.7.0 as your config suggests, you have to use the right container image as the base:

• The same Kafka version as you use
• The same Strimzi version you use

So both latest and 3.0.0 are wrong here.

[L040639]

i have added camel-aws-sqs-sink-kafka-connector-4.8.0-package.tar.gz into the plugins and building the image using docker and pushing that into our private registry and using the image in the kafka connect .

camel-aws-sqs-sink-kafka-connector-4.8.0-package.tar.gz was downloaded using below link and added my github repo and copying to the docker image in the plugin folders

https://camel.apache.org/camel-kafka-connector/4.8.x/reference/index.html

@scholzj which base image version i can use here .

[scholzj]

i have added camel-aws-sqs-sink-kafka-connector-4.8.0-package.tar.gz into the plugins and building the image using docker and pushing that into our private registry and using the image in the kafka connect .

camel-aws-sqs-sink-kafka-connector-4.8.0-package.tar.gz was downloaded using below link and added my github repo and copying to the docker image in the plugin folders

https://camel.apache.org/camel-kafka-connector/4.8.x/reference/index.html

That says nothing about the structure of the files.

@scholzj which base image version i can use here .

As I said - the one corresponding to the Strimzi version and Kafka version you use. You seem to be using Kafka 3.7.0. I do not know what Strimzi version are you using.

[L040639]

@scholzj Basically we are not using strimzi operator directly using a redhat streams for our openshift cluster , backend which strimzi version it is using not sure .

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  labels:
    operators.coreos.com/amq-streams.openshift-operators: ''
  name: amq-streams
  namespace: openshift-operators
spec:
  channel: stable
  installPlanApproval: Manual
  name: amq-streams
  source: redhat-operators
  sourceNamespace: openshift-marketplace
  startingCSV: amqstreams.v2.7.0-5

[scholzj]

Then you should not use a Strimzi image at all as the base image and you should use the corresponding Red Hat image instead. Please contact Red Hat support and they should be able to advice you which exact image corresponds to your product version.

[L040639]

@scholzj , Sure. Thanks for the input .