[RFC] Use libcramjam for compression codecs

[milesgranger]

See if there was any interest in using libcramjam.

There's probably further use/simplification available here, and could anyway help libcramjam's implementation. It's presently used in python's cramjam package that has served quite well.

datafusion-orc could make use of other codecs as well with low overhead that are already part of libcramjam (brotli, bzip2, xz, etc and the ISA-L backed zlib/deflate/gzip algorithms which are quite a bit faster than flate2 w/ zlib-ng [1] (at the expense of more C code and reduced compression level options 0, 1, 3)

I was blissfully unaware there existed a MIT friendly LZO implementation so might add that to libcramjam as well going forward.

[1] https://github.com/milesgranger/isal-rs/tree/main?tab=readme-ov-file#benchmarks

[Xuanwo]

Hi, thank you for bringing this up! As discussed at #120, we are moving to https://github.com/datafusion-contrib/orc-rust to get it ready been accepted into Apache ORC. Would you like to start a PR there instead? Thank you and sorry for any inconvenience.

[Xuanwo]

Hi, thank you for bringing this up! As discussed at #120, we are moving to https://github.com/datafusion-contrib/orc-rust to get it ready been accepted into Apache ORC. Would you like to start a PR there instead? Thank you and sorry for any inconvenience.

also cc @waynexia, would you like to remove code in this repo and make it clear in README?

[milesgranger]

Thanks for the heads up, moved now. :)