Snappy decompression performance

[markbaumgarten]

Ran a few tests and found that the snappy decompress function is a bit of a bottleneck.

Using python-snappy instead helped me a lot:

....
import snappy
...

def decompress(data: typing.BinaryIO) -> bytes:
    return snappy.uncompress(data.read())

[cclgroupltd]

Hi there, thanks for that. I think last time I checked that module required the snappy C library installed (and was a pain to use on Windows). Do you know if that's still the case?

[markbaumgarten]

I am only using ubuntu....so I dont know..sorry. /Mark

…

On Mon, Jan 9, 2023, 17:14 CCL Solutions Group ***@***.***> wrote: Hi there, thanks for that. I think last time I checked that module required the snappy C library installed (and was a pain to use on Windows). Do you know if that's still the case? — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUZW2WP5LVDS2KOZGZXHYTWRQ2PNANCNFSM6AAAAAATRGVZ7Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[cclgroupltd]

Yeah, I've had a look and it's still not exactly easy to install on Windows and requires an external dependency. So I might look at adding a "try import" around the C bindings and fall back to the pure Python version if that fails so we don't end up locking users out.