Skip to content
forked from kafkajs/zstd

ZStandard compression codec for KafkaJS

License

Notifications You must be signed in to change notification settings

emnify/kafkajs-zstd

This branch is 10 commits ahead of kafkajs/zstd:master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

6fd84a3 · Dec 31, 2023

History

26 Commits
Sep 29, 2023
Aug 31, 2023
Aug 31, 2023
Aug 31, 2023
Aug 31, 2023
Oct 9, 2020
Oct 9, 2020
Oct 7, 2020
Oct 9, 2020
Aug 31, 2023
Oct 9, 2020
Aug 31, 2023
Dec 31, 2023

Repository files navigation

@kafkajs/zstd

ZStandard codec for KafkaJS.

NOTE: This codec relies on cppzst for compression and decompression. Please divert any issues with cppzst there, and only open issues on this repo in case the version being used here needs to be updated.

This package is provided on a best-effort basis. See #2

Installation

npm install --save @kafkajs/zstd

Configuration

const {  CompressionTypes, CompressionCodecs } = require('kafkajs')
const ZstdCodec = require('@kafkajs/zstd')

// Both compressionParams and decompressionParams are optional
const compressionParams = { level: 1 }
const decompressionParams = {}
CompressionCodecs[CompressionTypes.ZSTD] = ZstdCodec(compressionParams, decompressionParams)

compressionParams

Optional Allows you to configure the compression level and training data.

{
    /*  
     * Higher compression level means faster at the cost of compression ratio or memory usage.
     * See https://facebook.github.io/zstd/
     */
    level: 3,

    /*
     * Training data for improving performance on small payloads.
     * See https://facebook.github.io/zstd/#small-data
     */
    dict: trainingData,
    dictSize: Buffer.byteLength(trainingData)
}

decompressionParams

Optional Allows you to configure the training data.

{
    /*
     * Training data for improving performance on small payloads.
     * See https://facebook.github.io/zstd/#small-data
     */
    dict: trainingData,
    dictSize: Buffer.byteLength(trainingData)
}

Testing

$ docker-compose up -d && node scripts/waitForKafka.js
$ npm test

# Interactive mode
$ npm run test:watch

# Verify type definitions
$ npm run test:types

# Lint
$ npm run lint

License

See LICENSE for more details.

About

ZStandard compression codec for KafkaJS

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 89.4%
  • TypeScript 10.6%