Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

does not work on big files #11

Open
davidsanda opened this issue Nov 30, 2022 · 2 comments
Open

does not work on big files #11

davidsanda opened this issue Nov 30, 2022 · 2 comments

Comments

@davidsanda
Copy link

davidsanda commented Nov 30, 2022

outb64.xz.zip
remove .zip prefix and unpack in 7zip manager. Try doing the same in the minlzdec. It fails. Not really sure where the issue is.
Packed as xz in 7zip

@fizbin
Copy link
Contributor

fizbin commented Apr 26, 2023

The failure has nothing to do with the file's size, though it is quite large.

This compressed file violates one of the restrictions listed in https://github.com/ionescu007/minlzma/#limitations-and-restrictions:

  • The LZMA2 property byte must indicate the LZMA properties lc = 3, pb = 2, lc = 0

(This is actually a documentation typo: it should read lc = 3, pb = 2, lp = 0 )

Specifically, this file uses a property byte that indicates lc = 5, pb = 3, lp = 1. I don't know what compression utility could create such a file.

If I decompress the file with some other lzma decompression tool and recompress it with xz using default options then the resulting file decompresses with minlzdec just fine. However, that compressed file is then 4291368 bytes instead of 4203108 bytes in the original. (so roughly 2% larger)

If I use xz -8 to compress the file, I get much closer to the original at 4209788 bytes.

I was unable to find a combination of options to xz that could exactly reproduce the original file. Also, every set of options I tried with xz produced a file that minlzdec could decompress; how are you compressing the original text?

@fizbin
Copy link
Contributor

fizbin commented Apr 26, 2023

Having re-read your comment, I tried again with 7-zip.

Indeed, 7-zip will by default compress this file with a property byte other than the usual one of lc = 3, pb = 2, lp = 0 and will produce a compressed file which minlzdec cannot decompress. However, xz will not produce a file that minlzdec can't handle, even with the -e option, nor will python's lzma module unless you supply a custom filter chain.

Figuring out why 7-zip uses an unusual LZMA2 property byte with this file will require a deeper dive into the 7-zip source code than I have time for tonight.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants