Fail to migrate BSManagedDocuments when -wal and -shm are present

[Motti-Shneor]

I need to (custom) migrate my documents upon open.

After much struggle I got it to work - with one last problem. If my document packages contain, persistentStore-wal and persistentStore-shm files, the migration "succeeds" but fails to save.

In one place in the internet I read that CoreData might mistake the old -wal files of the old persistent store for new model - and fail saving.

First, I don't understand why -wal and -sha files exist after I save a document.
Second, doesn't BSManagedDocument provide some temporary path for the migration, or alternatively help with a hook for this? I do not really understand the migration process, but I presume the new persistent store finally replaces the old one, and the old one gets deleted in the end - so why aren't the -wal and shm files deleted with it?

A few words about those 2 files will also be very helpful. When are they safe to delete etc.

[mikeabdullah]

Hi Motti,

I don't provide any hooks for migration for BSManagedDocument since it's highly app-dependent how you want to handle the migration. For example, my approach would likely be to subclass NSDocumentController to detect that the document being opened is in an old format, throw up some UI offering to migrate it, and work from there. But I can see that others want to make the process more automatic.

The wal and shm files aren't controlled by BSManagedDocument; they're entirely Core Data's business.

[Motti-Shneor]

While I'd do the same for robust document versioning, I need to update my model for technical reasons, and migration should be completely automatic and transparent to the user. I do it very simply. In my Document subclass of BSManagedDocument, I override

-(BOOL)configurePersistentStoreCoordinatorForURL:(NSURL *)url
                                          ofType:(NSString *)fileType
                              modelConfiguration:(NSString *)configuration
                                    storeOptions:(NSDictionary *)storeOptions
                                           error:(NSError *__autoreleasing *)error {
    // support light migration
    NSMutableDictionary *options = [@{NSMigratePersistentStoresAutomaticallyOption : @YES,
                                      NSInferMappingModelAutomaticallyOption : @YES } mutableCopy];
    if (storeOptions) {
        [options addEntriesFromDictionary:storeOptions];
    }
    return [super configurePersistentStoreCoordinatorForURL:url ofType:fileType  modelConfiguration:configuration storeOptions:options error:error];
}

Providing all the migration machinery in my model and code. It worked for me until now (7 migrations) but now I face real hard time with the 8th. Few relations became required (non-optional to-one relation). My old documents had "orphan" entities. My migrations fail, and I decided to cleanup all orphans BEFORE attempting the migration, to find the source of the problem. So I cleanup (using the old model) and save.

CoreDataEditor allows me to look into the clean DB - and it is good. No orphans. BUT, when I try to migrate, the migration process finds orphaned entities that have already been deleted!!! Even worse, it complains about more instances of entities than exist in the database! It gives me exact ObjectID's that don't exist. It's driving me mad, and I just can't make it WHERE does Core Data bring in the persistent store and data anymore. The errors all indicate the good, clean persistent store.

Can you hint? Can it be related to the way BSManagedDocument does its background writes, and autosave-in-place? or maybe application saved-states are my problem? the -wal? the -shm?

[mikeabdullah]

I can't hint because I honestly haven't a clue what's happening. BSManagedDocument provides a pretty standard Core Data stack, and integrates that with Apple's document-handling logic. If you're doing something custom, I'm afraid you need to understand what's going on and figure it out for yourself.