Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data API JsonCodec: Base64-binary support for vectors #1493

Open
tatu-at-datastax opened this issue Oct 3, 2024 · 0 comments
Open

Data API JsonCodec: Base64-binary support for vectors #1493

tatu-at-datastax opened this issue Oct 3, 2024 · 0 comments
Assignees

Comments

@tatu-at-datastax
Copy link
Contributor

(note: follow up to #1492)

After basic "JSON number array" input, also need to add support for optimized variant for vectors: Base64-encoded "packed floats" binary, in which client:

  1. First converts from float[] (or platform equivalent) into IEEE-754 packed (4 bytes per float) byte sequence
  2. Encodes resulting byte sequence into padded "MIME" Base64 String
  3. Encloses String into EJSON "$binary" wrapping -- same as blob values

and Data API will decode, unpack things, passes to CQL driver in structure it expects.

This issue DOES NOT change handling on return path: it will use "simple"/"plain" JSON Number array representation -- this because we do not know which representation was used for insert/update (unlike with Collections where we keep a copy in doc_json)

For further work we'd file another issue as necessary to allow requesting optimal representation in results.

@tatu-at-datastax tatu-at-datastax self-assigned this Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant