Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to load schema with references #111

Open
bbrcan opened this issue Sep 2, 2024 · 2 comments
Open

Failing to load schema with references #111

bbrcan opened this issue Sep 2, 2024 · 2 comments

Comments

@bbrcan
Copy link

bbrcan commented Sep 2, 2024

Hi there,

I have the following enum called Status:

{
  "schema": {
    "type": "enum",
    "namespace": "test",
    "name": "Status",
    "symbols": ["Pending", "Enroute", "Arrive", "Complete"]
  }
}

And the following schema called Route:

{
  "schema": {
    "type": "record",
    "namespace": "test",
    "name": "Route",
    "fields": [
      {
        "name": "orgId",
        "type": "string"
      },
      {
        "name": "status",
        "type": "Status"
      }
    ]
  },
  "references": [
    {
      "name": "test.Status",
      "subject": "Status",
      "version": 1
    }
  ]
}

When I try to load the Route schema via either GetSchemaByVersion or GetSchema, I get back:

schema registry error: Record "test.Route" field 2 ought to be valid Avro named type: unknown type name: "Status"

Note I am using sc.CodecCreationEnabled(true). Happy to disable this but then I'm not quite sure how I'd load the codec with the references.

I've also tried referencing the Status type as test.Status in the Route schema, to no avail.

Can you shed any light on why this might be occurring? Thank you

Using srclient 0.7.0, Golang 1.23

@bbrcan
Copy link
Author

bbrcan commented Sep 6, 2024

For those interested I've managed to get around this by the following:

  • Setting sc.CodecCreationEnabled(false) (this is default anyway)
  • For each reference, download the schema (you may have to do this recursively). Then use a regex or string replacement to substitute that into the parent schema.
  • Build the codec using your new "flattened" schema which now doesn't contain any references.

@AtakanColak
Copy link
Collaborator

Hi @bbrcan, thank you for raising this issue. It seems to be related to the behaviour around codec, and the underlying goavro package doesn't seem to have any documentation for reference types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants