Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce HashCompressor for name compression #396

Merged
merged 4 commits into from
Oct 21, 2024
Merged

Introduce HashCompressor for name compression #396

merged 4 commits into from
Oct 21, 2024

Commits on Oct 16, 2024

  1. [base/message_builder] Introduce 'HashCompressor' 

    I noticed that 'TreeCompressor' will likely have a lot of overhead, as
it has to copy each label into its internal tree and it maintains a
separate hashmap for every known name (where each name likely has one or
two parents at the most, even though hashmaps have a large capacity).

'HashCompressor' is an alternative name compressor built on top of the
'hashbrown' crate, which offers lower-level hash table access that lets
entries indirectly reference the built message for hashing and equality
checks.  It maintains a single hashmap, it should be faster, and it does
not require as much memory.

The implementation here makes it clear that domain names should have
been stored in reverse order, at least in the wire format.  A decent
chunk of the logic goes into correctly reversing the domain name.  In
fact, the implementation likely quadratic runtime because of this (at
least when 'Name' or 'RelativeName' are used, as backward iteration on
them has quadratic runtime).

While 'std' essentially exposes the same data structures as 'hashbrown',
it does not currently offer a way to perform low-level hash table access
(although there is a nightly "raw entry" feature).  This may change in
the future.

If not for the added 'hashbrown' dependency, I would suggest deprecating
'TreeCompressor' entirely.
    @bal-e
    bal-e committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    905c438 View commit details
    Browse the repository at this point in the history

  2. [base/message_builder] Mention 'HashCompressor' in API docs

    @bal-e
    bal-e committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    b1028b4 View commit details
    Browse the repository at this point in the history

  3. Add internal docs for HashCompressor 

    The feature flag to enable it has been changed to be more specific.
    @bal-e
    bal-e committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    c8034cf View commit details
    Browse the repository at this point in the history

  4. Move 'HashCompressor' under the 'std' feature

    @bal-e
    bal-e committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    072474a View commit details
    Browse the repository at this point in the history