Skip to content

Serde: Impl Avro type for timestamp-nanos logical type #872

Open
@ManuelRin

Description

@ManuelRin

Issue submitter TODO list

  • I've searched for an already existing issues here
  • I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

The UI doesn't currently support decoding an Avro timestamp-nanos logical type to JSON when it's displayed in the UI. It does support other (older) timestamp types such as timestamp-micros though. The timestamp-nanos logical type fields are currently displayed as their underlying long (nanoseconds since the Unix epoch).

In the latest specification Avro 1.12.0 the new logical type timestamp-nanos (and local-timestamp-nanos) was added via [AVRO-3884].

Details:

Describe the feature you're interested in

It would be great to see timestamp-nanos values correctly decoded to a point in time, like timestamp-micros is presented, but with the necessary precision of course. Same should be done for the local-timestamp-nanos logical type (local time instead of UTC).

According to JsonAvroConversion.java as of today, there is support for timestamp-micros but not yet support for timestamp-nanos which would likely work very similar.

TIMESTAMP_MICROS("timestamp-micros",
(node, schema) -> {
if (node.isIntegralNumber()) {
// TimeConversions.TimestampMicrosConversion for impl
long microsFromEpoch = node.longValue();
long epochSeconds = microsFromEpoch / (1_000_000L);
long nanoAdjustment = (microsFromEpoch % (1_000_000L)) * 1_000L;
return Instant.ofEpochSecond(epochSeconds, nanoAdjustment);
} else if (node.isTextual()) {
return Instant.parse(node.asText());
} else {
throw new JsonAvroConversionException(
"node '%s' can't be converted to timestamp-millis logical type"
.formatted(node));
}
},
(obj, schema) -> new TextNode(obj.toString()),
new SimpleFieldSchema(
new SimpleJsonType(
JsonType.Type.STRING,
Map.of(FORMAT, new TextNode(DATE_TIME))))
),

Describe alternatives you've considered

Alternative to this feature is to take the presented integer value of the underlying long and offset it from the Unix epoch 'by hand'.

Version you're running

4cf17a0 (1.1.0)

Additional context

Unit tests should be added as well, code search for 'timestamp-micros' provides the relevant locations where tests for 'timestamp-nanos' would also be appropriate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/serdeSerialization & Deserialization (plugins)good first issueUp for grabsscope/backendRelated to backend changesstatus/triage/completedAutomatic triage completedtype/enhancementEn enhancement/improvement to an already existing feature

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions