Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[write-fonts] reproduce and fix non deterministic gvar serialization #750

Merged
merged 4 commits into from
Dec 20, 2023

Conversation

anthrotype
Copy link
Member

Meant to reproduce the issue discussed at googlefonts/fontc#647

If we are lucky enough the CI should fail, otherwise flip the coin again and you'll see...

Meant to reproduce the issue discussed at googlefonts/fontc#647

If we are lucky enough the CI should fail, otherwise flip the coin again and you'll see...
@anthrotype
Copy link
Member Author

anthrotype commented Dec 20, 2023

nice, it did fail the first time :)

https://github.com/googlefonts/fontations/actions/runs/7277928604/job/19831021279?pr=750

assertion `left == right` failed
  left: Some(All)
 right: Some(Some([1, 3]))

@anthrotype
Copy link
Member Author

anthrotype commented Dec 20, 2023

Finally it took me a few rounds but I managed to also make the CI randomly pass without actually changing anything :)

https://github.com/googlefonts/fontations/actions/runs/7277928604/job/19831276618

This is to match Python's max() built-in that returns the first item when multiple items are maximum, whereas the Rust Iterator::max_by_key returns the last. Both would work, but this way we match Python fonttools more closely.
Copy link
Member

@cmyr cmyr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find cosimo, and I think that matching the behaviour of fonttools is the right call.

This was interesting, and I'm glad it made me go and look more closely into the rust HashMap implementation. :)

write-fonts/src/tables/gvar.rs Outdated Show resolved Hide resolved
@anthrotype
Copy link
Member Author

This particular test glyph is funny: basically, there are 7 tuple variations, 5 of them have PackedPointNumbers::All, 2 of them have PackedPointNumbers::Some([1, 3]). Using either one as the shared point numbers will save the same number of bytes (4 in this particular case).
Now, because of the indeterminacy of hashmap order (which uses a random hash seed that changes at every execution), we would end up using either one at every execution, leading to a gvar with the same total size but different checksum.
So I decided that between the two possible valid shared point numbers, we should use the first one that fits the bill, following the ordering of the tuple variations themselves, i.e. the set of packed point numbers that first occurs when iterating over the tuple variations.
One could also take the last, and Rust max_by_key does exactly that, but since this whole piece of code was meant to port and match fonttools, I think it made more sense to match Python's max() behavior and take the first in case of ties.

@cmyr
Copy link
Member

cmyr commented Dec 20, 2023

Oh one other idea here: you should be able to get closer to ensuring that this test fails consistently by just running it in a loop for some number of iterations.

(and another clarification is that (if I'm understanding correctly) the hashmap does not use a new seed on each execution, but each call to HashMap::new() uses a different seed)

@anthrotype
Copy link
Member Author

each call to HashMap::new() uses a different seed

oh good to know.

you should be able to get closer to ensuring that this test fails consistently

good point, but I think now that I fixed it it's no longer going to fail/pass inconsistently so maybe it's not worth doing the loop any more?

@anthrotype anthrotype changed the title [write-fonts] reproduce non deterministic gvar serialization [write-fonts] reproduce and fix non deterministic gvar serialization Dec 20, 2023
@anthrotype anthrotype merged commit 7fd954e into main Dec 20, 2023
9 checks passed
@anthrotype anthrotype deleted the repro-gvar-indeterminism branch December 20, 2023 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants