Support loading Avro schema from file

[silverwind]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

For various sources and sinks like Kafka, one must currently inline the Avro schema via encoding.avro.schema or decoding.avro.schema into the config which is cumbersome because of escaping and general unwieldiness of big JSON strings.

Proposal

Add a option encoding.avro.schema_file that is treated mutually exclusive with encoding.avro.schema to load the schema from a relative or absolute file path:

encoding.avro.schema_file = "./schema.json"

• It should load valid JSON files (e.g. no escaping)
• Path resolution should work just like the file option of remap.

Version

0.35.0