realm crash: Realm file decryption failed

[tomholub]

I'm still getting this and have to reset storage to continue using the app.

The interesting thing is that it seemed to happen for both apps at the same time this morning. So I wonder, maybe the encryption key we use from keychain is not stable?

Unable to open a realm at path '/var/mobile/Containers/Data/Application/850D5417-D347-4FCC-BDB4-A1AE3E1A2200/Documents/encrypted.realm': Realm file decryption failed Path:Exception backtrace:
0   FlowCrypt                           0x00000001049df9fc _ZN5realm4util6detail26ExceptionWithBacktraceBaseC2Ev + 64
1   FlowCrypt                           0x00000001049ee55c _ZN5realm4util22ExceptionWithBacktraceISt13runtime_errorEC2IJRKNSt3__112basic_stringIcNS5_11char_traitsIcEENS5_9allocatorIcEEEEEEEDpOT_ + 104
2   FlowCrypt                           0x00000001049ee310 _ZN5realm4util4File11AccessErrorC2ERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEESB_ + 84
3   FlowCrypt                           0x00000001049ee29c _ZN5realm15InvalidDatabaseC2ERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_ + 88
4   FlowCrypt                           0x00000001049e80ec _ZN5realm15InvalidDatabaseC1ERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEES9_ + 44
5   FlowCrypt                           0x00000001049e79fc _ZN5realm9SlabAlloc11attach_fileERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEERNS0_6ConfigE + 4156
6   FlowCrypt                           0x0000000104aabf14 _ZN5realm2DB4openERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEbNS_9DBOptionsE + 4028
7   FlowCrypt                           0x0000000104aaf544 _ZN5realm2DB4openERNS_11ReplicationERKNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEENS_9DBOptionsE + 244
8   FlowCrypt                           0x0000000104ab7124 _ZN5realm2DB6createENSt3__110unique_ptrINS_11ReplicationENS1_14default_deleteIS3_EEEERKNS1_12basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEENS_9DBOptionsE + 324
9   FlowCrypt                           0x0000000104cc79f0 _ZN5realm5_impl16RealmCoordinator7open_dbEv + 1444
10  FlowCrypt                           0x0000000104cc963c _ZN5realm5_impl16RealmCoordinator12do_get_realmENS_5Realm6ConfigERNSt3__110shared_ptrIS2_EENS_4util8OptionalINS_9VersionIDEEERNS8_17CheckedUniqueLockE + 92
11  FlowCrypt                           0x0000000104cc9444 _ZN5realm5_impl16RealmCoordinator9get_realmENS_5Realm6ConfigENS_4util8OptionalINS_9VersionIDEEE + 692
12  FlowCrypt                           0x0000000104e73204 _ZN5realm5Realm16get_shared_realmENS0_6ConfigE + 156
13  FlowCrypt                           0x00000001049498f8 +[RLMRealm realmWithConfiguration:queue:error:] + 1968
14  FlowCrypt                           0x00000001055aa4e4 $sSo8RLMRealmC13configuration5queueABSo0A13ConfigurationC_So012OS_dispatch_C0CSgtKcfCTO + 128
15  FlowCrypt                           0x000000010564cba8 $s10RealmSwift0A0V13configuration5queueA2C13ConfigurationV_So012OS_dispatch_D0CSgtKcfC + 200
16  FlowCrypt                           0x00000001045b4328 $s9FlowCrypt16EncryptedStorageC8validateyyKF + 276
17  FlowCrypt                           0x000000010447d544 $s9FlowCrypt12GlobalRouterC24validateEncryptedStorage33_57D5652C71D18AB9D7F3E85674F97089LLyyyyXEF + 156
18  FlowCrypt                           0x000000010447d010 $s9FlowCrypt12GlobalRouterC7proceedyyF + 64
19  FlowCrypt                           0x0000000104372150 $s9FlowCrypt11AppDelegateC11application_29didFinishLaunchingWithOptionsSbSo13UIApplicationC_SDySo0k6LaunchJ3KeyaypGSgtF + 316
20  FlowCrypt                           0x0000000104372240 $s9FlowCrypt11AppDelegateC11application_29didFinishLaunchingWithOptionsSbSo13UIApplicationC_SDySo0k6LaunchJ3KeyaypGSgtFTo + 184
21  UIKitCore                           0x000000018326b044 8388EB03-002B-3B35-A78A-6A022894292E + 3313732
22  UIKitCore                           0x0000000183452b38 8388EB03-002B-3B35-A78A-6A022894292E + 5311288
23  UIKitCore                           0x000000018343b4a4 8388EB03-002B-3B35-A78A-6A022894292E + 5215396
24  UIKitCore                           0x0000000183298e44 8388EB03-002B-3B35-A78A-6A022894292E + 3501636
25  UIKitCore                           0x00000001830f162c _UIScenePerformActionsWithLifecycleActionMask + 104
26  UIKitCore                           0x000000018321cb7c 8388EB03-002B-3B35-A78A-6A022894292E + 2993020
27  UIKitCore                           0x000000018333d380 8388EB03-002B-3B35-A78A-6A022894292E + 4174720
28  UIKitCore                           0x000000018357df8c 8388EB03-002B-3B35-A78A-6A022894292E + 6537100
29  UIKitCore                           0x0000000183509710 8388EB03-002B-3B35-A78A-6A022894292E + 6059792
30  UIKitCore                           0x00000001830f2430 8388EB03-002B-3B35-A78A-6A022894292E + 1770544
31  UIKitCore                           0x00000001831b2350 8388EB03-002B-3B35-A78A-6A022894292E + 2556752
32  UIKitCore                           0x00000001830f4340 _UISceneSettingsDiffActionPerformChangesWithTransitionContext + 276
33  UIKitCore                           0x00000001831dedf4 8388EB03-002B-3B35-A78A-6A022894292E + 2739700
34  UIKitCore                           0x0000000183666260 8388EB03-002B-3B35-A78A-6A022894292E + 7488096
35  UIKitCore                           0x0000000183122c60 8388EB03-002B-3B35-A78A-6A022894292E + 1969248
36  UIKitCore                           0x0000000183166524 8388EB03-002B-3B35-A78A-6A022894292E + 2245924
37  UIKitCore                           0x000000018329dd70 8388EB03-002B-3B35-A78A-6A022894292E + 3521904
38  UIKitCore                           0x00000001831dd0b4 8388EB03-002B-3B35-A78A-6A022894292E + 2732212
39  FrontBoardServices                  0x000000019262ce20 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 171552
40  FrontBoardServices                  0x0000000192652cdc 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 326876
41  FrontBoardServices                  0x000000019260d6b4 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 42676
42  FrontBoardServices                  0x000000019260ecf4 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 48372
43  libdispatch.dylib                   0x00000001807a7660 A5CBAAB3-E389-3548-BAAC-FAB18411B94A + 18016
44  libdispatch.dylib                   0x00000001807ab118 A5CBAAB3-E389-3548-BAAC-FAB18411B94A + 33048
45  FrontBoardServices                  0x000000019260ef94 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 49044
46  FrontBoardServices                  0x000000019260e3d4 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 46036
47  FrontBoardServices                  0x00000001926129e4 6FDEC0C3-E500-3E06-948E-E34BDA520801 + 63972
48  CoreFoundation                      0x0000000180b57020 B2D21CFD-378C-36D5-BAF7-3F70599CFEFC + 765984
49  CoreFoundation                      0x0000000180b67ce0 B2D21CFD-378C-36D5-BAF7-3F70599CFEFC + 834784
50  CoreFoundation                      0x0000000180aa2054 B2D21CFD-378C-36D5-BAF7-3F70599CFEFC + 24660
51  CoreFoundation                      0x0000000180aa77f4 B2D21CFD-378C-36D5-BAF7-3F70599CFEFC + 47092
52  CoreFoundation                      0x0000000180abb3b8 CFRunLoopRunSpecific + 600
53  GraphicsServices                    0x000000019c44b38c GSEventRunModal + 164
54  UIKitCore                           0x000000018345b6a8 8388EB03-002B-3B35-A78A-6A022894292E + 5346984
55  UIKitCore                           0x00000001831da7f4 UIApplicationMain + 2092
56  FlowCrypt                           0x00000001043ba1bc $s9FlowCryptyyXEfU_ + 364
57  FlowCrypt                           0x00000001043ba218 $ss5Error_pIgzo_ytsAA_pIegrzo_TR + 24
58  FlowCrypt                           0x00000001043ba26c $ss5Error_pIgzo_ytsAA_pIegrzo_TRTA + 28
59  libswiftObjectiveC.dylib            0x00000001ae902a20 $s10ObjectiveC15autoreleasepool8invokingxxyKXE_tKlF + 64
60  FlowCrypt                           0x00000001043ba034 main + 88
61  dyld                                0x000000010b539a24 start + 520.

[tomholub]

My suspicion seems right realm/realm-swift#5615

[tomholub]

Also, could this have been accidentally triggered?

    if ProcessInfo().arguments.contains(AppReset.reset.rawValue) {
        AppReset.resetKeychain()
        AppReset.resetUserDefaults()
    }

[tomholub]

relevant code

//
//  KeyChainService.swift
//  FlowCrypt
//
//  Created by Anton Kharchevskyi on 25.11.2019.
//  Copyright © 2017-present FlowCrypt a. s. All rights reserved.
//

import FlowCryptCommon
import Foundation
import Security

// keychain is used to generate and retrieve encryption key which is used to encrypt local DB
// it does not contain any actual data or keys other than the db encryption key

protocol KeyChainServiceType {
    func getStorageEncryptionKey() throws -> Data
}

struct KeyChainService: KeyChainServiceType {
    private static var logger = Logger.nested(in: Self.self, with: "Keychain")

    // the prefix ensures that we use a different keychain index after deleting the app
    // because keychain entries survive app uninstall
    private static var keychainIndex: String = {
        // todo - verify if this is indeed atomic (because static) or if there can be a race condition
        let prefixStorageIndex = "indexSecureKeychainPrefix"
        let storageEncryptionKeyIndexSuffix = "-indexStorageEncryptionKey"
        if let storedPrefix = UserDefaults.standard.string(forKey: prefixStorageIndex) {
            return storedPrefix + storageEncryptionKeyIndexSuffix
        }
        guard let randomBytes = CoreHost().getSecureRandomByteNumberArray(12) else {
            fatalError("could not get secureKeychainPrefix random bytes")
        }
        let prefix = Data(randomBytes)
            .base64EncodedString()
            .replacingOccurrences(of: "[^A-Za-z0-9]+", with: "", options: [.regularExpression])

        logger.logInfo("LocalStorage.secureKeychainPrefix generating new prefix")
        UserDefaults.standard.set(prefix, forKey: prefixStorageIndex)
        return prefix + storageEncryptionKeyIndexSuffix
    }()

    private let keyByteLen = 64

    private func generateAndSaveStorageEncryptionKey() throws {
        Self.logger.logInfo("generateAndSaveStorageEncryptionKey")

        guard let randomBytes = CoreHost().getSecureRandomByteNumberArray(keyByteLen) else {
            let message = "KeyChainService getSecureRandomByteNumberArray bytes are nil"
            throw AppErr.general(message)
        }

        let key = Data(randomBytes)
        let query: [CFString: Any] = [
            kSecClass: kSecClassGenericPassword,
            kSecAttrAccount: KeyChainService.keychainIndex,
            kSecValueData: key
        ]
        let addOsStatus = SecItemAdd(query as CFDictionary, nil)
        guard addOsStatus == noErr else {
            let message = "KeyChainService SecItemAdd osStatus = \(addOsStatus), expected 'noErr'"
            throw AppErr.general(message)
        }
    }

    func getStorageEncryptionKey() throws -> Data {
        let query: [CFString: Any] = [
            kSecClass: kSecClassGenericPassword,
            kSecAttrAccount: KeyChainService.keychainIndex,
            kSecReturnData: kCFBooleanTrue!,
            kSecMatchLimit: kSecMatchLimitOne
        ]
        var keyFromKeychain: AnyObject?
        let findOsStatus = SecItemCopyMatching(query as CFDictionary, &keyFromKeychain)
        guard findOsStatus != errSecItemNotFound else {
            try generateAndSaveStorageEncryptionKey() // saves new key to storage
            return try getStorageEncryptionKey() // retries search
        }

        guard findOsStatus == noErr else {
            let message = "KeyChainService SecItemCopyMatching status = \(findOsStatus), expected 'noErr'"
            throw AppErr.general(message)
        }

        guard let validKey = keyFromKeychain as? Data else {
            let message = "KeyChainService keyFromKeychain not usable as Data. Is nil?: \(keyFromKeychain == nil)"
            throw AppErr.general(message)
        }

        guard validKey.count == keyByteLen else {
            let message = "KeyChainService validKey.count != \(keyByteLen), instead is \(validKey.count)"
            throw AppErr.general(message)
        }

        return validKey
    }
}

[tomholub]

I'll refactor this to have more control over initialization of Realm.

We seem to be initializing encrypted storage / realm all over the app, and many times with EncryptedStorage() everywhere.

Instead I'll do it once during app startup, and then dependency-inject already initialized storage. Plus make the key chain parts a MainActor. That should help clean things up.

[tomholub]

@sosnovsky to address this issue, I will want to only initialize EncryptedStorage() (and Realm) once. Either I'll do it once per view controller, or maybe I'll pass it between view controllers during init of the VC, not sure.

To do that, I'll create a Context struct that would contain storage in it. In the future, it may contain more things that we need in almost every VC. The Context would be created in the beginning (either starting the app, or init of each VC) and it would be dependency-injected into every place that needs storage.

Does that sound reasonable?

[tomholub]

Just noticed, we already pass session: SessionType? everywhere in a similar manner. So this would go into the context as well. As follows:

struct AppContext {
    let encryptedStorage: EncryptedStorageType
    let session: SessionType?
}