Fix "junk" data to allow removal of readsb from feed-in containers

[mikenye]

Currently can't upgrade bordercontrol until this issue is resolved, or we backport readsb into the current version.

[mikenye]

From: https://discord.com/channels/207038656311984139/1231792328050606120/1231792330466656267

Hi Team. I've been working through adding CRC checks to pw-feeder by working through the 1090MHz Riddle book.

We can easily perform CRC checks on the ADS-B messages (DF17/DF18) as we know the interrogator ID is 0, so we can calculate the parity and reject the frame if the remainder isn't 0.

For Address Parity: i'm thinking that we wait until we've received a DF17/18 message and we have a list of ICAO addresses. When we get a message with address parity, we can then check the computed parity (address) against the list of known addresses. If the computed parity doesn't match any addresses, discard the frame.

For Parity/interrogator identifier: Unsure what we can do with this. Reading about 1090MHz, it appears that the valid interrogator numbers are 0 through 63, so we could reject where the PI is outside of this. However, even if the PI is within 0 to 63, that doesn't necessarily mean the frame is OK...

Another thought I had is dropping frames of types we don't care about. Do we have a list of frames that we use for our data? Obviously ident, position, altitude, etc. Do we use any others (eg: navigational uncertainty etc)?

Anyway, i wanted to leave my thoughts here so we can discuss and figure out how we want to proceed.

[bluntelk]

FYI, CRC checks are already performed (and busted frames discarded) in pw_ingest on ADSB messages.

https://github.com/plane-watch/pw-pipeline/blob/085bbf6657b2f95c894991ae6ab399fddbe15c1a/lib/tracker/mode_s/crc.go#L68

[bluntelk]

dump1090 using a scoring method to determine if the other types of frames are valid.

[shaneshort]

FYI, CRC checks are already performed (and busted frames discarded) in pw_ingest on ADSB messages.

it costs less to not send the data in the first place rather than reject it on ingress I would've thought?

[mikenye]

FYI, CRC checks are already performed (and busted frames discarded) in pw_ingest on ADSB messages.

it costs less to not send the data in the first place rather than reject it on ingress I would've thought?

We need to do it both places. Not everyone will run the up-to-date feeder, or even run our feeder. Also, we're still getting "junk" data coming through that's being filtered with readsb before it hits pw-ingest, so something somewhere is letting that junk through.