Custom Brotli Dictionaries #212
Conversation
fuzzing for brotli
Compress Validator Modules
| /// Points to data owned by Go. | ||
| ptr: *mut u8, | ||
| /// The length in bytes. | ||
| len: *mut usize, |
There was a problem hiding this comment.
It's not clear to me why the len is also a pointer - couldn't it just be a usize?
There was a problem hiding this comment.
Added another comment :)
There was a problem hiding this comment.
Ah right; I somehow missed that that actually happened. The comment probably isn't necessary; should be implied by the type; I just missed it.
| /// Returns a pointer to a compression-ready instance of the given dictionary. | ||
| pub fn ptr(&self, level: u32) -> Option<*const EncoderPreparedDictionary> { | ||
| match self { | ||
| Self::StylusProgram if level == 11 => Some(STYLUS_PROGRAM_DICT.0), |
There was a problem hiding this comment.
Why the check if level == 11? brotli can still use a custom dictionary at lower levels can't it?
There was a problem hiding this comment.
It seems you need a different dictionary for each level. Ours is a level 11 and that's what cargo stylus uses
There was a problem hiding this comment.
You know, we should probs make this return a result given that. I'll add a commit
There was a problem hiding this comment.
Ah, I see what you mean. The binary dictionary we supply isn't level-specific, but the thing prepared by BrotliEncoderPrepareDictionary is. In that case it seems like maybe we'd want to either allow the possibility to prepare dictionaries at other levels, or else only allow compression at level 11. But I don't have a strong opinion; as-is is fine with me.
Adds support for custom brotli dictionaries, allowing us to dramatically cut Stylus contract sizes.
Resolves STY-67