Refine BAR format

[gardners]

Currently we waste space in BARs by having 7 bytes for file size. Only the log2 of the file size is ever needed, so we can save 6 bytes.

These extra bytes can be used to provide more of the BID prefix, so that collisions become exceedingly improbably. At present with 64 bits, we expect 50% chance of a collision once there are about 2^32 bundles in existence. Adding 48 bits to get 112 bits means that we would need 2^56 bundles, or approximately 8 million for every person on the planet. This is clearly a helpful move.

We can also reduce the version field to 32 bits if we change the manifest semantics to monotonically increasing values instead of using a notional milli-second clock.

Probably also makes sense to have a short bit field somewhere (eg the upper 3 bits of (log2(filesize)-10), which will never be used to indicate version of BAR format. This could probably be done retrospectively now using the MSB of the file size in this way.

[gardners]

The main part of this issue (extending the BID prefix by using log2(filesize) has been substantially addressed as part of the work on issue #9.

The new BAR format is:

120 bits - manifest ID prefix.
8 bits - log2(length) of associated file.
56 bits - low 56 bits of version number.
16 bits - min latitude (-90 - +90).
16 bits - min longitude (-180 - +180).
16 bits - max latitude (-90 - +90).
16 bits - max longitude (-180 - +180).
8 bits - TTL of bundle in hops (0xff = unlimited distribution)

Note that we still allow a long version number, because we have not changed the version number allocation logic, so this issue is not quite ready for closure.

[quixotique]

See issue #29 relating to Rhizome version numbers. Any BAR refinements arising from versioning changes ought to be dealt with in that issue, so this issue can be closed if there are no other improvements to be made to the BAR format at the moment.

Should the BAR format be refined to include some version bits, so that future adjustments can be made without damaging back compatibility? Possibly not.

If Rhizome BAR advertisements are always sent to/from a well-known MDP port number, then upgrading the BAR format can simply change to a new port number, and the software can continue to service and/or send to the old port for a while to support compatibility during the upgrade period. This proposal would consume more network bandwidth during the transition period, but not forever. BAR format changes ought to be rare events, so it would be a shame to permanently sacrifice precious BAR bits for this purpose if another mechanism can be employed.

I propose closing this issue.

[gardners]

No objection from me. I personally like the 56bit version field, and we are not in desperate need of freeing up more bytes, and there is a certain neatness of keeping the BARs 2^5=32 bytes instead of, say, 29 bytes.

I will close the issue and as suggested any followup activity should occur in issue #29