Shared(.appStorage), protected data availability and the possibility of corrupted state

[KaiOelfke]

Probably you've seen this blog post that was shared a lot recently on UserDefaults and app prewarming.

https://christianselig.com/2024/10/beware-userdefaults/

The summary is that iOS in some cases launches apps in the background after a reboot before the first unlock to speed up the perceived app launch once the user interacts with the app. Before the first unlock anything on the file system that has completeUntilFirstUserAuthentication is not available.

I already once spend quite a lot of time to debug some Core Data setup issue in a group container due to this where I learned that anything in an shared application group container has at least completeUntilFirstUserAuthentication. So one must deal with prewarming issues carefully.

When I read the blog post I first thought this would only apply to app group user defaults. But I verified in a sample project that UserDefaults.standard also is affected. So I guess it's also created with completeUntilFirstUserAuthentication by the system.

The biggest problem as written in the article, is if some state is initialized while the device is locked as UserDefaults returns nil. Then the initialValue or some fallback will be used. And at a later launch some mutation based on this initialValue can write incorrect data back to the UserDefaults store.

E.g. the value on disk for an Int is 42, while locked the initialValue of 0 is used instead, and some increment action then writes 1 to the store instead of 43.

It's possible that this doesn't always apply to appStorage as there's the willEntereForground notification observation:

let willEnterForeground: (any NSObjectProtocol)?
    if let willEnterForegroundNotificationName {
      willEnterForeground = NotificationCenter.default.addObserver(
        forName: willEnterForegroundNotificationName,
        object: nil,
        queue: nil
      ) { _ in
        didSet(load(initialValue: initialValue))
      }
    } else {
      willEnterForeground = nil
    }

So I guess if the mutation happens after a foreground event the event will have already loaded the actual value. But in app extensions and some prewarming or background scenarios there can be a write before the app enters foreground. For example background URL session or background push may trigger some work without the app opening.

My question would be, if that's something you'd address in the library or not. Fundamentally UserDefaults has the same issues so I'd understand, if you would say that developers need to be aware of it and make their own adjustments. On the other hand this is a quite annoying thing to deal with and also something many developers won't be aware of as the whole thing is not documented much by Apple. So it would be nice to have protections against this scenario abstracted away in the PersistenceKey.

As far as I understand guarding against this scenario would require checking, if the device is in a state where protected data is unavailable and then refreshing the state once that changes. For the first check one can write an empty file to a temp dir with completeUntilFirstUserAuthentication, which will throw if it's not available. I don't know, if it's possible to use a DispatchSource to be notified when such a file becomes available in app extensions.

If UIApplication is available one can use isProtectedDataAvailable, protectedDataDidBecomeAvailableNotification, protectedDataWillBecomeUnavailableNotification. One can also use a Keychain item to check this. Maybe there's more ways to do this check.

https://nemecek.be/blog/104/checking-if-device-is-locked-or-sleeping-in-ios

Not really related to this I also checked the fileStorage implementation and saw that for loading the initialValue is used as fallback, if something is thrown and for saving thrown errors are just ignored with try?. Would it not make sense here to use the reportIssue API?

[KaiOelfke]

I now had a time to make a repro sample. It's still annoying to test as it requires a device and rebooting.

To reproduce the prewarming scenario:

1. Launch app on device with live activities enabled (simulator doesn't work)
2. Schedule live activity, leave app (activity should be on lock screen and dynamic island)
3. Optionally give permission for lock screen activity
4. Power off device
5. Turn on device without unlocking it
6. Wait 10-60s
7. Unlock device
8. Live activity should reappear
9. Open app (it should've been already launched via prewarming in step 6)
10. Investigate app state
11. Cancel activities, force quit app and relaunch for a clean state for further test iterations

The sample allows to test various setups with UserDefaults, Shared and AppStorage and different launch configurations where Shared or UserDefaults is accessed in a locked prewarming state or not.

Compared to my previous testing there's an apparent fix for iOS 18 where later accesses to UserDefaults update the in memory cache from the file cache so only the initial value is wrong. I have to test this sample again on an iOS 17 device later.

AppStorage performs better in my sample as it refreshes correctly some time after the unlocking. While Shared in most cases use the default value potentially overriding actual values when the value gets mutated.

I don't see this as a swift-sharing library issue, but as a very complicated iOS app lifecycle issue where it would be very desirable to have mitigations in place. I can share my custom persistence keys that I use as a workaround in my apps.

Live-Activity-Example-main.zip

This discussion should be moved to swift-sharing by the way.

[jshier]

This is an issue for all storage solutions across iOS. There's no great way to deal with it (I can find a good article later). Often apps just kill themselves in the prewarming scenario, as you can't otherwise say you don't support it.

[KaiOelfke]

This is an issue for all storage solutions across iOS.

Yes, that's correct for everything using the file system, which is basically everything.

Often apps just kill themselves in the prewarming scenario, as you can't otherwise say you don't support it.

I thought about that approach as well. It works, but it's not great obviously and defeats the purpose of this prewarming. Plus I think you lose the live activity then? I already have workarounds in my app that seem to work fine. My code checks, if it's a prewarming state by checking the file system access for completeFileProtectionUntilFirstUserAuthentication, if not prewarming everything continues as usual and otherwise any file system access is delayed until certain signals (e.g. notifications about the application state or protected data availability) happen. One remaining issue is that 3rd party frameworks are not really controllable other than initializing them later. I actually had end users reporting me that their RevenueCat entitlements were gone and it was due to RevenueCat's user defaults access. I could reproduce this bug and fixed it by delayed RevenueCat setup to after unlocking in the case of prewarming.

[stephencelis]

@KaiOelfke We will at least surface errors soon (#40), so you could use the presence of such an error as a guard against writes to that state. We're open to other ideas, though.

[KaiOelfke]

I saw the PR. I think that's already a great improvement to make errors accessible. With that I should be able to improve my custom shared wrapper key. I'm not sure, if it would work for UserDefaults / AppStorage as there's no file system errors but wrong nil values for any key.

We're open to other ideas, though.

Either you say this is out of scope of the library and library adopters need to handle it themselves e.g. by checking error presence and reloading with the changes in the PR. Or you can consider to add code like my workaround. I can share my unpolished code, if helpful. But the basic idea is:

1. Have a function that writes a temporary empty dummy file with completeFileProtectionUntilFirstUserAuthentication, which fails or works to determine if the device was never unlocked. There may be other ways to detect a never unlocked after booting state.
2. If it was unlocked everything operates normally
3. If it's locked do not load or save anything, expose the error, observe certain signals to run the function from 1 again and then refresh all shared values from their underlying source e.g. file / app storage, if the unlock has happened

With this a library user can show an error UI e.g. in a widget or elsewhere and as soon as the device is unlocked and the shared code runs all values get refreshed.

If you choose to not do anything other than exposing the errors for the file system storage I'd at least consider updating app storage values as in my tests with the sample @AppStorage refreshed itself after the unlock in my sample on iOS 18. So @AppStorage had a better experience out of the box for this specific scenario.