Skip to content
forked from dbry/lzw-ab

LZW streaming compressor/decompressor compatible with UNIX compress

License

Notifications You must be signed in to change notification settings

andrew-aladev/lzws

 
 

Repository files navigation

LZW streaming compressor/decompressor

Github Actions Codecov
Github Actions test status Codecov

LZW streaming compressor/decompressor based on LZW AB compatible with UNIX compress.

Compressor dictionaries

  • Linked list (idea from LZW AB). It has low memory usage <= 327 KB (16 bit codes). It is slow in general, recommended for small amount of data.
  • Sparse array (enabled by default). It has high memory usage <= 33.5 MB (16 bit codes). It is fast, recommended for large amount of data.

You can add your own implementation, see api for dictionary.

Why?

  • You can compress and decompress .Z (tar.Z) archives.
  • You can resurrect legacy apis (like Content-Encoding: compress in HTTP 1.1).
  • You can support ancient software and make your application looks like ancient software.

New features

  • Accurate ratio calculation without overhead, compressor provides smaller archive (compatible with UNIX compress).
  • Unaligned bit groups switch (only disabled mode is compatible with UNIX compress).
  • LSB/MSB switch (only LSB mode is compatible with UNIX compress).
  • Magic header switch (only enabled magic header is compatible with UNIX compress).

Dependencies

Operating systems: GNU/Linux, OSX, FreeBSD, Windows (Visual Studio, MinGW, Cygwin).

Popular OS Runtime dependencies Compilation dependencies
Ubuntu libgmp10/libtommath1 libgmp-dev/libtommath-dev
cmake, gcc/clang, asciidoc, docbook-xml
CentOS gmp/libtommath gmp-devel/libtommath-devel
cmake3, gcc/clang, asciidoc, docbook-dtds
ArchLinux gmp/libtommath gmp/libtommath
cmake, gcc/clang, asciidoc, docbook-xml
OSX gmp/libtommath gmp/libtommath
cmake, gcc/llvm, asciidoc
Windows libtommath msvc, windows sdk, ucrt, libtommath
cmake, curl, git, python, asciidoc, docbook-xml
xsltproc, docbook-bundle, zip, nsis

Installation

You can download and install prebuilt packages on releases page.

You can use scripts/release.sh, it will create package for your system from source. You can also use overlay for gentoo.

Installation for Windows (Visual Studio) from source is not so trivial:

  1. Install Visual Studio with components: msvc, windows sdk, ucrt.
  2. Install chocolatey with components: cmake, curl, git, python, xsltproc, docbook-bundle, zip, nsis.
  3. Install asciidoc.
  4. Download docbook-xml-4.5.zip and unpack it to C:\ProgramData\chocolatey\lib\docbook-bundle\docbook-4.5.
  5. Edit C:\ProgramData\chocolatey\lib\docbook-bundle\catalog.xml and append <nextCatalog catalog="./docbook-4.5/catalog.xml"/>.
  6. Launch x64 Native Tools Command Promp as admin.
git clone "https://github.com/libtom/libtommath.git" --depth 1 && cd "libtommath"
nmake -f "makefile.msvc" install && cd ".."
set "XML_CATALOG_FILES=C:\ProgramData\chocolatey\lib\docbook-bundle\catalog.xml"
set "INCLUDE=C:\devel\include;%INCLUDE%" && set "LIB=C:\devel\lib;%LIB%"
set "PATH=C:\devel\bin;C:\Program Files\Git\bin;C:\Program Files\CMake\bin;%PATH%"
cd build && bash "..\scripts\release.sh"

Please use the following command for OSX.

export XML_CATALOG_FILES=/usr/local/etc/xml/catalog

Quick start

cd build
cmake -DLZWS_COMPRESSOR_DICTIONARY="sparse-array" ".." && cmake --build "."
echo -n "TOBEORNOTTOBEORTOBEORNOT" | ./src/cli/lzws | ./src/cli/lzws -d
Option Values Default Description
LZWS_COMPRESSOR_DICTIONARY sparse-array, linked-list sparse-array dictionary implementation
LZWS_BIGNUM_LIBRARY gmp, tommath, "" (auto) "" (auto) bignum library
LZWS_SHARED ON, OFF ON build shared binaries
LZWS_STATIC ON, OFF OFF build static binaries
LZWS_CLI ON, OFF ON build cli
LZWS_TESTS ON, OFF ON build tests
LZWS_EXAMPLES ON, OFF OFF build examples
LZWS_MAN ON, OFF OFF build man
LZWS_ARGTABLE3_FALLBACK github, github-archive github-archive argtable3 fallback
bitbucket, bitbucket-archive
gitlab, gitlab-archive
LZWS_COVERAGE ON, OFF OFF enable coverage

Examples and docs

See examples, man and doc folders.

CI

Please visit scripts/test-images. See test script scripts/toolchains.sh and test script scripts/coverage_toolchains.sh for CI.

Real world testing

See lzws archive collector.

License

Copyright (c) 2016 David Bryant, 2018+ other authors, all rights reserved (see AUTHORS). Distributed under the BSD Software License (see LICENSE).

Notice

Releases with GMP provided without static linked binaries (GMP LGPL license limitation).

About

LZW streaming compressor/decompressor compatible with UNIX compress

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C 74.9%
  • CMake 21.4%
  • Shell 3.7%