"detect OF devices in Nearby state" can't search for known key

[thisiscam]

Recently the 5 bytes matching trick discussed in #46 appears to stop working.
I also just tried #50 and seems like I can't find devices that are in nearby state using the example device_scanner.py

[malmeloo]

Hmm that's strange, I recently used the scanner for another project of mine and could've sworn it received multiple "nearby" reports. That said, I have received a report from someone else saying that the accessory key generation is not working properly, which might be related to your problem. I've been quite busy but will investigate a bit more soon, will keep you posted.

[thisiscam]

Hmm that's strange, I recently used the scanner for another project of mine and could've sworn it received multiple "nearby" reports. That said, I have received a report from someone else saying that the accessory key generation is not working properly, which might be related to your problem. I've been quite busy but will investigate a bit more soon, will keep you posted.

To clarify, I can see there are nearby devices, but I can't match them with my own devices with known private keys.

[malmeloo]

Ahh alright, thanks for the clarification. I'll look into it!

[malmeloo]

Could you try whether #91 fixes your issue? You can install it as a drop-in replacement using pip install -U git+https://github.com/malmeloo/FindMy.py@fix/nearby-device-discovery.

[thisiscam]

Could you try whether #91 fixes your issue? You can install it as a drop-in replacement using pip install -U git+https://github.com/malmeloo/FindMy.py@fix/nearby-device-discovery.

Unfortunately that didn't fix the problem. I was already doing a timeshift adjustment in my own code. Is it possible that Apple changed their "5 matching bytes" protocol?

[thisiscam]

Some possibilities:

1. The rotating key algorithm was changed
2. The 5 matching bytes protocol was changed
3. I'm not using the correct decrypted plist??

Reason 3) is possible is that one of my Mac was updated to macOS 15 recently and I can no longer decrypt the plist on that machine. Ref: MartinPham/FindMySync#38
So I ended up using an older mac that's still on macOS 13 to decrypt the plist on that machine. Interestingly, I found this new decrypted plist file to use the same private key as the one I had been using before, prior to the macOS15 update. So maybe this explains away the possibility of 3) a little bit, but I'm not sure if there are catches.

[malmeloo]

That's really strange, and I'm honestly not sure what's going wrong. Of course it's possible that Apple changed the spec, but that's assuming that they somehow have the ability to update both airtags and 3rd party tags over the air.

I'm picking up my own 3rd party tag soon to do some experiments on this.

[thisiscam]

I'm leaning towards 1) at the moment. If I put my phone on flight mode, I get one (new) separated device, which presumably is my airtag. But it also doesn't match with any known public key in the time shift range.

[thisiscam]

Ok this should eliminate possibility of 2):

I put my phone flight mode, got one separated device:

SEPARATED Device - DC:B0:6B:C1:E1:20
  Public key:   XLBrweEg0MRQwAt5gk0zk0rRDOQtdW2vr1WCqw==
  Lookup key:   IcctnBNQnY0H3Pqq7JQzGtIQxDNvFAs2yWb2TlH3lkw=
  Status byte:  0
  Hint byte:    0
  Extra data:

I checked that

>>> import base64
>>> s = "XLBrweEg0MRQwAt5gk0zk0rRDOQtdW2vr1WCqw=="
>>> base64.b64decode(s)[1:6].hex()
'b06bc1e120'

So it seems like the 5 bytes trick is still valid!

It then seems likely that they changed the rotating key algorithm. They could be changing only the official airtag's algorithm afaik. I can see an airtag software version number on my phone --- not sure if recently got updated though!

[malmeloo]

The public key generated by the library is partially derived from those bytes in the mac address, so I'm not surprised it checks out ;-). And that's interesting, though I guess it makes sense that Apple would want to update the firmware on their own tags. Theoretically speaking it would be possible for Apple to use a new, updated algorithm only on their devices / AirTags, since owner devices are aware of the type of accessory that they're tracking.

If possible, could you try to re-pair the tag, then dump the plist again and see if that fixes the issue, at least temporarily?

[thisiscam]

The public key generated by the library is partially derived from those bytes in the mac address, so I'm not surprised it checks out ;-). And that's interesting, though I guess it makes sense that Apple would want to update the firmware on their own tags. Theoretically speaking it would be possible for Apple to use a new, updated algorithm only on their devices / AirTags, since owner devices are aware of the type of accessory that they're tracking.

If possible, could you try to re-pair the tag, then dump the plist again and see if that fixes the issue, at least temporarily?

Yeah, I re-paired, dumped and decoded the plist on macOS 13. The plist correctly reflects that I have repaired. But that did not fix the issue.

[malmeloo]

That unfortunate; I'll probably pick up a tag next week and experiment a bit on my own. Will let you know how it goes.

[thisiscam]

Sorry for the ping --- is there an update to this? Should I an opensource tag instead?

[malmeloo]

Depends on what you call an update - I've been able to pair a tag to one of my apple accounts, but for whatever reason I cannot get it to show up on my hackintosh. My best guess is that's because I only booted it up after having logged out of the device I paired it with. I don't exactly have regular access to an iPhone or iPad, let alone one I can use with a temporary account, so it's all taking way longer than necessary. Currently I'm just regularly saving the data this tag is broadcasting, so as soon as that plist is dumped I should know more.

[thisiscam]

Thanks for the update! I am fairly certain that the BLE nearby device tracking functionality is not working for official airtag and official clones anymore. In the mean time while you look at this, do you have any recommendation for a open source drop in replacement tag? As in, do you happen to know (to a good degree of certainty) that particular hardware should work with the BLE nearby device tracking?

[malmeloo]

The BLE device tracking should work with any "static" OpenHaystack tag. There are a bunch of people working on different firmwares for different tags, so it mostly depends on which hardware you can get your hands on. I am personally using these NRF52810 tags with this firmware, but there are likely cheaper options available.

[kimi4422]

嗯，这很奇怪，我最近将扫描仪用于我的另一个项目，并且可以肯定它收到了多个“附近”报告。话虽如此，我收到了其他人的报告，称附件密钥生成无法正常工作，这可能与您的问题有关。我一直很忙，但很快就会进行更多调查，并随时向您汇报。

Brother, I have some problems that need your help. Please leave your contact information and I can pay you.

[malmeloo]

嗯，这很奇怪，我最近将扫描仪用于我的另一个项目，并且可以肯定它收到了多个“附近”报告。话虽如此，我收到了其他人的报告，称附件密钥生成无法正常工作，这可能与您的问题有关。我一直很忙，但很快就会进行更多调查，并随时向您汇报。

Brother, I have some problems that need your help. Please leave your contact information and I can pay you.

If it's related to this issue then please leave your question here, that way everyone can benefit from the shared knowledge. If it's something really private you want to share, feel free to use my git email.

[thisiscam]

Just an update on my end: I actually tried to pair my official airtag with an iOS 17 device, and got the plist out from a macOS 13 --- it still does not work. At this point, the only possibility is that they changed the key rotation.

[thisiscam]

Just an update on my end: I actually tried to pair my official airtag with an iOS 17 device, and got the plist out from a macOS 13 --- it still does not work. At this point, the only possibility is that they changed the key rotation.

Actually, I spoke too soon! For the above comment, I was testing iOS 17 + macOS 13 with the latest version of the FindMy.py library, which wasn't working.

However, after switching back to FindMy.py version 0.6 (prior to the nearby scan was implemented) and using my old code from #46, everything is working again! This suggests that an issue in the current FindMy.py scanner code. Upon inspecting the scanner class, I couldn't find the "five consecutive bytes check" in the code. (nvm, found it. Turns out problem was just that the key matching needs a time window)

Since I am able to get everything to work again using iOS 17 + macOS 13 combo, it appears that Apple hasn't changed the AirTag key rotation algorithm itself. But, I am likely using the wrong master key after upgrading to iOS 18 + macOS 15, since the new keychain app does not allow easy extraction of the BeaconStore key to decode the plists, and I had to use macOS 13 to do the plist decoding! Interestingly, if I pair the official AirTag with iOS 18 but extract the plist using macOS 13, I get a plist file that still allows querying the AirTag's location from Apple's server. However, it’s possible that Apple is concealing the true private key with the iOS18 + macOS13 combo.

[malmeloo]

That's great news, thanks for investigating! So, just to summarize:

• Nearby scanning needs a fix in the library
• iOS 17 + macOS 13 works
• iOS xx + macOS 15 does not work, because the keys can't be dumped
• iOS 18 + macOS 13 does not work when scanning, but fetching locations does work

Correct?

[thisiscam]

That's great news, thanks for investigating! So, just to summarize:

* Nearby scanning needs a fix in the library

Right. The fix is in #97

* iOS 17 + macOS 13 works

* iOS xx + macOS 15 does not work, because the keys can't be dumped

* iOS 18 + macOS 13 does not work when scanning, but fetching locations does work

Correct?

Correct.

[malmeloo]

Cool, thanks for confirming. I'll look at your PR tomorrow when I have my laptop available.

I think it might be time to seriously start looking into how keychain syncing works. That way we should able to obtain the keys directly without having to dump them from a mac.

[thisiscam]

Cool, thanks for confirming. I'll look at your PR tomorrow when I have my laptop available.

I think it might be time to seriously start looking into how keychain syncing works. That way we should able to obtain the keys directly without having to dump them from a mac.

Keychain (as in the macOS app) syncing? Or do you simply mean how the airtag communicate with an iOS device to decide on the master private key? I do suspect that we can potentially have an middle man when the airtag pairs for the first time. Not sure how feasible that is though!

[malmeloo]

Setting up an AirTag still requires communication with Apple servers for registration, and I'd much prefer trying to reverse engineer a service running on macOS than one that runs exclusively on iOS 😅. I also think that keychain syncing would be more useful in general, because it would allow people to use AirTags without having to reset and pair them through this library. That said, both are interesting to look into.

[thisiscam]

Setting up an AirTag still requires communication with Apple servers for registration, and I'd much prefer trying to reverse engineer a service running on macOS than one that runs exclusively on iOS 😅. I also think that keychain syncing would be more useful in general, because it would allow people to use AirTags without having to reset and pair them through this library. That said, both are interesting to look into.

I'm lacking knowledge of what is "keychain syncing"?

[malmeloo]

Keychain as in the icloud keychain that stores secrets in your account. Since the AirTag master secrets are stored in the keychain, that should make it possible to pull them directly from an Apple account after logging in, instead of having to dump them from a mac.