Runtime crash when using `JSONDecoder().decode` after updating to XCode 15

[aramayyes]

Description

Hello. We get a EXC_BAD_ACCESS (code=1, address=0x444d49) error in libswiftCore.dylib when using JSONDecoder().decode. This error occurs occasionally (not every time) and only if Optimization Level is set to No Optimization[-Onone]. Everything worked fine on XCode 14. The error has appeared after updating to XCode 15.

Steps to reproduce

   let (data, _) = try await internalExecute(
      request,
      interceptors: interceptors
    ) // Returns a data from API using `URLSession.shared`

    do {
      let jsonDecoder = JSONDecoder()
      jsonDecoder.dateDecodingStrategy = .custom { _ in
        ...
        return Date() // Doesn't matter
      }

      let result = try jsonDecoder.decode(decodingType, from: data) // <== Crash
      return result
    } catch let error as DecodingError {
    ...

Expected behavior

No crash like it was in XCode 14 :)

Screenshots

Environment

• Swift compiler version info swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-5.9.0.128.108 clang-1500.0.40.1) Target: arm64-apple-macosx14.0
• Xcode version info Xcode 15.0 - Build version 15A240d
• Deployment target:

Logs

libswiftCore.dylib`Swift.Array<τ_0_0 where τ_0_0: Swift.Decodable>.init(from: Swift.Decoder) throws -> Swift.Array<τ_0_0>:
    0x18c9b178c <+0>:   str    x28, [sp, #-0x60]!
    0x18c9b1790 <+4>:   stp    x27, x26, [sp, #0x10]
    0x18c9b1794 <+8>:   stp    x25, x24, [sp, #0x20]
    0x18c9b1798 <+12>:  stp    x23, x22, [sp, #0x30]
    0x18c9b179c <+16>:  stp    x20, x19, [sp, #0x40]
    0x18c9b17a0 <+20>:  stp    x29, x30, [sp, #0x50]
    0x18c9b17a4 <+24>:  add    x29, sp, #0x50
    0x18c9b17a8 <+28>:  sub    sp, sp, #0x30
    0x18c9b17ac <+32>:  mov    x22, x21
    0x18c9b17b0 <+36>:  mov    x23, x2
    0x18c9b17b4 <+40>:  mov    x24, x1
    0x18c9b17b8 <+44>:  ldur   x19, [x1, #-0x8]
    0x18c9b17bc <+48>:  ldr    x8, [x19, #0x40]
    0x18c9b17c0 <+52>:  mov    x9, sp
    0x18c9b17c4 <+56>:  add    x8, x8, #0xf
    0x18c9b17c8 <+60>:  and    x8, x8, #0xfffffffffffffff0
    0x18c9b17cc <+64>:  sub    x25, x9, x8
    0x18c9b17d0 <+68>:  mov    sp, x25
    0x18c9b17d4 <+72>:  mov    x9, sp
    0x18c9b17d8 <+76>:  sub    x26, x9, x8
    0x18c9b17dc <+80>:  mov    sp, x26
    0x18c9b17e0 <+84>:  adrp   x8, 209966
    0x18c9b17e4 <+88>:  add    x8, x8, #0x5a0            ; _swiftEmptyArrayStorage
    0x18c9b17e8 <+92>:  stur   x8, [x29, #-0x48]
    0x18c9b17ec <+96>:  ldp    x27, x28, [x0, #0x18]
    0x18c9b17f0 <+100>: stur   x0, [x29, #-0x80]
    0x18c9b17f4 <+104>: mov    x1, x27
    0x18c9b17f8 <+108>: bl     0x18cbdd37c               ; __swift_project_boxed_opaque_existential_0Tm
    0x18c9b17fc <+112>: ldr    x9, [x28, #0x20]
    0x18c9b1800 <+116>: sub    x8, x29, #0x78
    0x18c9b1804 <+120>: mov    x20, x0
    0x18c9b1808 <+124>: mov    x21, x22
    0x18c9b180c <+128>: mov    x0, x27
    0x18c9b1810 <+132>: mov    x1, x28
    0x18c9b1814 <+136>: blr    x9
    0x18c9b1818 <+140>: mov    x22, x21
    0x18c9b181c <+144>: cbz    x21, 0x18c9b1838          ; <+172>
    0x18c9b1820 <+148>: adrp   x0, 209966
    0x18c9b1824 <+152>: add    x0, x0, #0x5a0            ; _swiftEmptyArrayStorage
    0x18c9b1828 <+156>: bl     0x18cc71388               ; swift_bridgeObjectRelease
    0x18c9b182c <+160>: ldur   x0, [x29, #-0x80]
    0x18c9b1830 <+164>: bl     0x18cbdd2ac               ; __swift_destroy_boxed_opaque_existential_1Tm
    0x18c9b1834 <+168>: b      0x18c9b1918               ; <+396>
    0x18c9b1838 <+172>: ldp    x27, x28, [x29, #-0x60]
    0x18c9b183c <+176>: sub    x0, x29, #0x78
    0x18c9b1840 <+180>: mov    x1, x27
    0x18c9b1844 <+184>: bl     0x18cbdd37c               ; __swift_project_boxed_opaque_existential_0Tm
    0x18c9b1848 <+188>: ldr    x8, [x28, #0x18]
    0x18c9b184c <+192>: mov    x20, x0
    0x18c9b1850 <+196>: mov    x0, x27
    0x18c9b1854 <+200>: mov    x1, x28
    0x18c9b1858 <+204>: blr    x8
    0x18c9b185c <+208>: tbz    w0, #0x0, 0x18c9b186c     ; <+224>
    0x18c9b1860 <+212>: adrp   x20, 209966
    0x18c9b1864 <+216>: add    x20, x20, #0x5a0          ; _swiftEmptyArrayStorage
    0x18c9b1868 <+220>: b      0x18c9b1908               ; <+380>
    0x18c9b186c <+224>: ldp    x27, x28, [x29, #-0x60]
    0x18c9b1870 <+228>: sub    x0, x29, #0x78
    0x18c9b1874 <+232>: mov    x1, x27
    0x18c9b1878 <+236>: bl     0x18cbddda0               ; __swift_mutable_project_boxed_opaque_existential_1
    0x18c9b187c <+240>: mov    x20, x0
    0x18c9b1880 <+244>: ldr    x9, [x28, #0xa0]
    0x18c9b1884 <+248>: mov    x8, x26
    0x18c9b1888 <+252>: mov    x0, x24
    0x18c9b188c <+256>: mov    x1, x24
    0x18c9b1890 <+260>: mov    x2, x23
    0x18c9b1894 <+264>: mov    x21, x22
    0x18c9b1898 <+268>: mov    x3, x27
    0x18c9b189c <+272>: mov    x4, x28
    0x18c9b18a0 <+276>: blr    x9
    0x18c9b18a4 <+280>: mov    x22, x21
    0x18c9b18a8 <+284>: cbnz   x21, 0x18c9b1940          ; <+436>
    0x18c9b18ac <+288>: ldr    x8, [x19, #0x20]
    0x18c9b18b0 <+292>: mov    x0, x25
    0x18c9b18b4 <+296>: mov    x1, x26
    0x18c9b18b8 <+300>: mov    x2, x24
    0x18c9b18bc <+304>: blr    x8
    0x18c9b18c0 <+308>: mov    x0, #0x0
    0x18c9b18c4 <+312>: mov    x1, x24
    0x18c9b18c8 <+316>: bl     0x18cbd7d48               ; type metadata accessor for Swift.Array
    0x18c9b18cc <+320>: mov    x1, x0
    0x18c9b18d0 <+324>: sub    x20, x29, #0x48
    0x18c9b18d4 <+328>: mov    x0, x25
    0x18c9b18d8 <+332>: bl     0x18c97005c               ; Swift.Array.append(__owned τ_0_0) -> ()
->  0x18c9b18dc <+336>: ldp    x27, x28, [x29, #-0x60]
    0x18c9b18e0 <+340>: sub    x0, x29, #0x78
    0x18c9b18e4 <+344>: mov    x1, x27
    0x18c9b18e8 <+348>: bl     0x18cbdd37c               ; __swift_project_boxed_opaque_existential_0Tm
    0x18c9b18ec <+352>: ldr    x8, [x28, #0x18]
    0x18c9b18f0 <+356>: mov    x20, x0
    0x18c9b18f4 <+360>: mov    x0, x27
    0x18c9b18f8 <+364>: mov    x1, x28
    0x18c9b18fc <+368>: blr    x8
    0x18c9b1900 <+372>: tbz    w0, #0x0, 0x18c9b186c     ; <+224>
    0x18c9b1904 <+376>: ldur   x20, [x29, #-0x48]
    0x18c9b1908 <+380>: sub    x0, x29, #0x78
    0x18c9b190c <+384>: bl     0x18cbdd2ac               ; __swift_destroy_boxed_opaque_existential_1Tm
    0x18c9b1910 <+388>: ldur   x0, [x29, #-0x80]
    0x18c9b1914 <+392>: bl     0x18cbdd2ac               ; __swift_destroy_boxed_opaque_existential_1Tm
    0x18c9b1918 <+396>: mov    x0, x20
    0x18c9b191c <+400>: mov    x21, x22
    0x18c9b1920 <+404>: sub    sp, x29, #0x50
    0x18c9b1924 <+408>: ldp    x29, x30, [sp, #0x50]
    0x18c9b1928 <+412>: ldp    x20, x19, [sp, #0x40]
    0x18c9b192c <+416>: ldp    x23, x22, [sp, #0x30]
    0x18c9b1930 <+420>: ldp    x25, x24, [sp, #0x20]
    0x18c9b1934 <+424>: ldp    x27, x26, [sp, #0x10]
    0x18c9b1938 <+428>: ldr    x28, [sp], #0x60
    0x18c9b193c <+432>: ret    
    0x18c9b1940 <+436>: ldur   x0, [x29, #-0x48]
    0x18c9b1944 <+440>: bl     0x18cc71388               ; swift_bridgeObjectRelease
    0x18c9b1948 <+444>: sub    x0, x29, #0x78
    0x18c9b194c <+448>: bl     0x18cbdd2ac               ; __swift_destroy_boxed_opaque_existential_1Tm
    0x18c9b1950 <+452>: b      0x18c9b182c               ; <+160>

and

libswiftCore.dylib`swift_isUniquelyReferenced_nonNull_native:
->  0x18cc729d8 <+0>:  ldr    x8, [x0, #0x8]
    0x18cc729dc <+4>:  tbnz   x8, #0x3f, 0x18cc729f0    ; <+24>
    0x18cc729e0 <+8>:  lsr    x8, x8, apple/swift#32
    0x18cc729e4 <+12>: cmp    x8, #0x0
    0x18cc729e8 <+16>: cset   w0, eq
    0x18cc729ec <+20>: ret    
    0x18cc729f0 <+24>: cmn    w8, #0x1
    0x18cc729f4 <+28>: b.ne   0x18cc72a00               ; <+40>
    0x18cc729f8 <+32>: mov    w0, #0x0
    0x18cc729fc <+36>: ret    
    0x18cc72a00 <+40>: lsl    x8, x8, apple/swift#3
    0x18cc72a04 <+44>: ldp    x8, xzr, [x8, #0x10]
    0x18cc72a08 <+48>: and    x9, x8, #0x100000000
    0x18cc72a0c <+52>: bfxil  x9, x8, apple/swift#33, apple/swift#31
    0x18cc72a10 <+56>: cmp    x9, #0x0
    0x18cc72a14 <+60>: cset   w0, eq
    0x18cc72a18 <+64>: ret  

[cdf1982]

Hello,

I stumbled on this issue while investigating a rare crash (5 Users in six week, in a pool of at least a couple thousands that have updated to the latest macOS) happening only on Sonoma.

Do you have an URL you're trying to decode?

In my case – and I still need to figure out a solution – there is an optional URL property that fails to decode (only with certain values, not clear which ones) on any build compiled with Xcode 15, producing the following error:

 dataCorrupted(
            Swift.DecodingError.Context(
                codingPath: [_JSONKey(stringValue: "Index 3", intValue: 3), CodingKeys(stringValue: "webcamStreamURL", intValue: nil)],
                debugDescription: "Invalid URL string.",
                underlyingError: nil
            )
        )

I suspect that, in my case and possibly yours, this might be related to a change introduced in Xcode 15 beta 5:

For apps linked on or after iOS 17 and aligned OS versions, URL parsing has updated from the obsolete RFC 1738/1808 parsing to the same RFC 3986 parsing as URLComponents. This unifies the parsing behaviors of the URL and URLComponents APIs. Now, URL automatically percent- and IDNA-encodes invalid characters to help create a valid URL.

What perplexes me is that my app targets 10.14, and my understanding of "linked on" is that it should mean targeting, but instead just running on Sonoma (the equivalent of iOS 17) seem to trigger the new URL parsing behavior.

I just moments ago realised that this might be the cause of the problem I've been investigating for six weeks (it's impossible for me to reproduce, as it appears that just some URLs trigger the crash), so I don't have a solution yet: for "regular" URL parsing the documentations says to use init(string:encodingInvalidCharacters:) with false, but how do I tell JSONDecoder to do it is still unclear to me, if it's possible at all.

If you're decoding URLs, this might be worth investigating.

[ktoso]

Probably the more appropriate place for the question is in foundation, I moved the issue.

[cdf1982]

Thank you @ktoso, I think it's appropriate: I was just looking at JSONDecoder source and on line 702 it indeed appears that it's using URL(string: string), so without an option to use the new separate initialiser with encodingInvalidCharacters:false, I am not sure how I'll get out of this situation, especially since it's unclear to me what previously saved User content is triggering the crash on decoding.

Way above my level of understanding, but perhaps JSONDecoder needs a URL decoding strategy that allows to revert back to the previous behavior.

[cdf1982]

I am way out of my league, but I don't think I'm right about line 702 (I suppose the #if FOUNDATION_FRAMEWORK directive and the "Reenable once URL and Decimal has been moved" todo next to it comment mean that code is currently not executed).

But I do wonder if the currently shipping implementation of JSONDecoder accounts for the switch to RFC 3986 in the default implementation of URL.init, and if it doesn't my humble opinion is that a fallback to the previous method could be really useful.

Consider what follows not much more than pseudocode, but basically I think it could be reasonable that a decoding attempt that fails with the new method is retried with the previous implementation:

    private func unwrapURL(from mapValue: JSONMap.Value, for codingPathNode: _JSONCodingPathNode, _ additionalKey: (some CodingKey)? = nil) throws -> URL {
        try checkNotNull(mapValue, expectedType: URL.self, for: codingPathNode, additionalKey)

        let string = try self.unwrapString(from: mapValue, for: codingPathNode, additionalKey)
        if let url = URL(string: string) {
            return url
        } else if let urlDecodedTheOldWay = URL(string: string, encodingInvalidCharacters: false) {
            return urlDecodedTheOldWay
        } else {
            throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: codingPathNode.path(with: additionalKey),
                                                                    debugDescription: "Invalid URL string."))
        }
    }

Again, I might be completely missing the point for my crash, but this seems reasonable to investigate considering that switching to Xcode 15 is what introduced this crash for me as well.

[aramayyes]

Hello,

I stumbled on this issue while investigating a rare crash (5 Users in six week, in a pool of at least a couple thousands that have updated to the latest macOS) happening only on Sonoma.

Do you have an URL you're trying to decode?

In my case – and I still need to figure out a solution – there is an optional URL property that fails to decode (only with certain values, not clear which ones) on any build compiled with Xcode 15, producing the following error:

 dataCorrupted(
            Swift.DecodingError.Context(
                codingPath: [_JSONKey(stringValue: "Index 3", intValue: 3), CodingKeys(stringValue: "webcamStreamURL", intValue: nil)],
                debugDescription: "Invalid URL string.",
                underlyingError: nil
            )
        )

I suspect that, in my case and possibly yours, this might be related to a change introduced in Xcode 15 beta 5:

For apps linked on or after iOS 17 and aligned OS versions, URL parsing has updated from the obsolete RFC 1738/1808 parsing to the same RFC 3986 parsing as URLComponents. This unifies the parsing behaviors of the URL and URLComponents APIs. Now, URL automatically percent- and IDNA-encodes invalid characters to help create a valid URL.

What perplexes me is that my app targets 10.14, and my understanding of "linked on" is that it should mean targeting, but instead just running on Sonoma (the equivalent of iOS 17) seem to trigger the new URL parsing behavior.

I just moments ago realised that this might be the cause of the problem I've been investigating for six weeks (it's impossible for me to reproduce, as it appears that just some URLs trigger the crash), so I don't have a solution yet: for "regular" URL parsing the documentations says to use init(string:encodingInvalidCharacters:) with false, but how do I tell JSONDecoder to do it is still unclear to me, if it's possible at all.

If you're decoding URLs, this might be worth investigating.

Hello cdf1982, thank you for investigation.

I don't have any URLs in my models so I guess mine is another case.

[cdf1982]

Thank you for letting me know. I had hoped we were onto something both for mine and your issue.

[aramayyes]

Hello. Are there any news?

[kperryua]

Hello!

@cdf1982's case is one that definitely deserves its own issue. ("linked on XXX" by the way refers to "built using the XXX SDK")

Unfortunately @aramayyes I'm afraid it's rather difficult to deduce just from these crash points what the likely cause is here without a (semi-)reproducible case in hand. Perhaps you could provide an example payload, Decodable type, and JSONDecoder invocation that together produce this crash?