Skip to content

feat: Added ability to enforce numeric map keys are encoded as numbers rather than strings #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ console.log(buffer);
| sortKeys | boolean | false |
| forceFloat32 | boolean | false |
| forceIntegerToFloat | boolean | false |
| forceNumericMapKeys | boolean | false |
| ignoreUndefined | boolean | false |

To skip UTF-8 decoding of strings, `useRawBinaryStrings` can be set to `true`. In this case, strings are decoded into `Uint8Array`.
Expand Down
31 changes: 27 additions & 4 deletions src/Encoder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,13 @@ export type EncoderOptions<ContextType = undefined> = Partial<
*/
forceFloat32: boolean;

/**
* If `true`, numeric map keys will be encoded as ints/floats (as appropriate) rather than strings (the default).
*
* Defaults to `false`.
*/
forceNumericMapKeys: boolean;

/**
* If `true`, an object property with `undefined` value are ignored.
* e.g. `{ foo: undefined }` will be encoded as `{}`, as `JSON.stringify()` does.
Expand Down Expand Up @@ -82,6 +89,7 @@ export class Encoder<ContextType = undefined> {
private readonly forceFloat32: boolean;
private readonly ignoreUndefined: boolean;
private readonly forceIntegerToFloat: boolean;
private readonly forceNumericMapKeys: boolean;

private pos: number;
private view: DataView;
Expand All @@ -98,6 +106,7 @@ export class Encoder<ContextType = undefined> {
this.forceFloat32 = options?.forceFloat32 ?? false;
this.ignoreUndefined = options?.ignoreUndefined ?? false;
this.forceIntegerToFloat = options?.forceIntegerToFloat ?? false;
this.forceNumericMapKeys = options?.forceNumericMapKeys ?? false;

this.pos = 0;
this.view = new DataView(new ArrayBuffer(this.initialBufferSize));
Expand Down Expand Up @@ -383,13 +392,23 @@ export class Encoder<ContextType = undefined> {
return count;
}

private isNumber(value: string | number) {
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
return !isNaN(value as any) && !isNaN(parseFloat(value as any));
}

private encodeMap(object: Record<string, unknown>, depth: number) {
const keys = Object.keys(object);
// eslint-disable-next-line @typescript-eslint/no-unsafe-argument
const keys = Object.keys(object).map((key) => (this.forceNumericMapKeys && this.isNumber(key) ? Number(key) : key));
if (this.sortKeys) {
keys.sort();
if (keys.filter((k) => typeof k === "number").length > 0) {
keys.sort().sort((a, b) => +a - +b);
} else {
keys.sort();
}
}

const size = this.ignoreUndefined ? this.countWithoutUndefined(object, keys) : keys.length;
const size = this.ignoreUndefined ? this.countWithoutUndefined(object, Object.keys(object)) : keys.length;

if (size < 16) {
// fixmap
Expand All @@ -410,7 +429,11 @@ export class Encoder<ContextType = undefined> {
const value = object[key];

if (!(this.ignoreUndefined && value === undefined)) {
this.encodeString(key);
if (typeof key === "string") {
this.encodeString(key);
} else {
this.encodeNumber(key);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will automatically encode as an appropriate number (admittedly, based on int mode when that PR lands) so there isn't a lot of control over the type - potentially we want that to be configurable e.g. to always set them to uint64 or similar?

}
this.doEncode(value, depth + 1);
}
}
Expand Down
111 changes: 111 additions & 0 deletions test/encode-decode-map-number-keys.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/* eslint-disable @typescript-eslint/naming-convention */
import { deepStrictEqual, equal } from "assert";
import { encode } from "../src/encode";
import { decode } from "../src/decode";

const exampleMap = {
1: "1",
"102": "102",
a: "a",
20: "20",
} as Record<number | string, string>;

function getExpectedMsgPack(
key1: Uint8Array,
value1: Uint8Array,
key2: Uint8Array,
value2: Uint8Array,
key3: Uint8Array,
value3: Uint8Array,
key4: Uint8Array,
value4: Uint8Array,
) {
return new Uint8Array([
// fixmap of length 4: https://github.com/msgpack/msgpack/blob/master/spec.md#map-format-family
0b10000100,
...key1,
...value1,
...key2,
...value2,
...key3,
...value3,
...key4,
...value4,
]);
}

describe("map-with-number-keys", () => {
it(`encodes numeric keys as numbers when forced`, () => {
const expected = getExpectedMsgPack(
// This is the order Object.keys returns
encode(1),
encode("1"),
encode(20),
encode("20"),
encode(102),
encode("102"),
encode("a"),
encode("a"),
);

const encoded = encode(exampleMap, { forceNumericMapKeys: true });

equal(Buffer.from(encoded).toString("hex"), Buffer.from(expected).toString("hex"));
deepStrictEqual(decode(encoded), exampleMap);
});

it(`encodes numeric keys as strings when not forced`, () => {
const expected = getExpectedMsgPack(
// This is the order Object.keys returns
encode("1"),
encode("1"),
encode("20"),
encode("20"),
encode("102"),
encode("102"),
encode("a"),
encode("a"),
);

const encoded = encode(exampleMap);

equal(Buffer.from(encoded).toString("hex"), Buffer.from(expected).toString("hex"));
deepStrictEqual(decode(encoded), exampleMap);
});

it(`encodes numeric keys as strings with sorting`, () => {
const expected = getExpectedMsgPack(
encode("1"),
encode("1"),
encode("102"),
encode("102"),
encode("20"),
encode("20"),
encode("a"),
encode("a"),
);

const encoded = encode(exampleMap, { sortKeys: true });

equal(Buffer.from(encoded).toString("hex"), Buffer.from(expected).toString("hex"));
deepStrictEqual(decode(encoded), exampleMap);
});

it(`encodes numeric keys as numbers with sorting`, () => {
const expected = getExpectedMsgPack(
encode(1),
encode("1"),
encode(20),
encode("20"),
encode(102),
encode("102"),
encode("a"),
encode("a"),
);

const encoded = encode(exampleMap, { sortKeys: true, forceNumericMapKeys: true });

equal(Buffer.from(encoded).toString("hex"), Buffer.from(expected).toString("hex"));
deepStrictEqual(decode(encoded), exampleMap);
});
});