Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NIOFileSystem crashes when loading more than 2GB #2878

Open
weissi opened this issue Sep 10, 2024 · 1 comment
Open

NIOFileSystem crashes when loading more than 2GB #2878

weissi opened this issue Sep 10, 2024 · 1 comment

Comments

@weissi
Copy link
Member

weissi commented Sep 10, 2024 •
edited
Loading

Expected behavior

When requesting impossible things (e.g. loading more than 2GB into a ByteBuffer), NIOFileSystem should raise an error and not crash.

Actual behavior

Crash

Steps to reproduce

request.body = try await Data(buffer: ByteBuffer(
    contentsOf: "/tmp/4GBFile",
    maximumSizeAllowed: .gibibytes(4)
))

problem

Thread 8 Crashed:: NIO-SGLTN-0-#7
0   0x102fc5490 Swift runtime failure: arithmetic overflow + 0 [inlined]
1   0x102fc5490 ByteBuffer._setBytes(_:at:) + 420 (ByteBuffer-core.swift:491)
2   0x102fc9190 ByteBuffer.setBytes(_:at:) + 88 (ByteBuffer-core.swift:1007)
3   0x102fbd7d8 closure #1 in ByteBuffer.setBuffer(_:at:) + 72 (ByteBuffer-aux.swift:399)
4   0x102fbd810 partial apply for closure #1 in ByteBuffer.setBuffer(_:at:) + 28
5   0x102fc72a0 ByteBuffer.withUnsafeReadableBytes<A>(_:) + 448 (ByteBuffer-core.swift:748)
6   0x102fbd764 ByteBuffer.setBuffer(_:at:) + 196 (ByteBuffer-aux.swift:398)
7   0x102fbd950 ByteBuffer.writeBuffer(_:) + 308 (ByteBuffer-aux.swift:412)
8   0x102fbe534 ByteBuffer.writeImmutableBuffer(_:) + 204 (ByteBuffer-aux.swift:512)
9   0x1038a6400 ReadableFileHandleProtocol.readToEnd(fromAbsoluteOffset:maximumSizeAllowed:) + 364 (FileHandleProtocol.swift:389)
10  0x10387d38d closure #1 in ByteBuffer.init<A>(contentsOf:maximumSizeAllowed:fileSystem:) + 1 (ByteBuffer+FileSystem.swift:32)
11  0x10387d595 partial apply for closure #1 in ByteBuffer.init<A>(contentsOf:maximumSizeAllowed:fileSystem:) + 1
12  0x1038df7e5 closure #1 in FileSystemProtocol.withFileHandle<A>(forReadingAt:options:execute:) + 1 (FileSystemProtocol.swift:329)
13  0x1038df9a1 partial apply for closure #1 in FileSystemProtocol.withFileHandle<A>(forReadingAt:options:execute:) + 1
14  0x103911ef1 withUncancellableTearDown<A>(_:tearDown:) + 1 (Cancellation.swift:40)
15  0x1038df3b5 FileSystemProtocol.withFileHandle<A>(forReadingAt:options:execute:) + 1 (FileSystemProtocol.swift:328)
16  0x10387cfa9 ByteBuffer.init<A>(contentsOf:maximumSizeAllowed:fileSystem:) + 1 (ByteBuffer+FileSystem.swift:31)
17  0x10387d705 ByteBuffer.init(contentsOf:maximumSizeAllowed:) + 1 (ByteBuffer+FileSystem.swift:48)
@glbrntt
Copy link
Contributor

glbrntt commented Sep 10, 2024

Oof. Yeah, we should throw an error and suggest using the streaming APIs instead.

@glbrntt glbrntt mentioned this issue Sep 10, 2024
Open
clintonpi added a commit to clintonpi/swift-nio that referenced this issue Oct 9, 2024
@clintonpi
[NIOFileSystem] Avoid crash when reading more than ByteBuffer capacity
d003730 
Motivation:

As described in issue (apple#2878)[apple#2878], NIOFileSystem crashes when reading more than `ByteBuffer` capacity.

Modifications:

- Add a new static property, `byteBufferCapacity`, to `ByteCount` representing the maximum amount of bytes that can be written to `ByteBuffer`.
- Throw a `FileSystemError` in `ReadableFileHandleProtocol.readToEnd(fromAbsoluteOffset:maximumSizeAllowed:)` when `maximumSizeAllowed` is more than `ByteCount.byteBufferCapacity`.

Result:

NIOFileSystem will `throw` instead of crashing.
clintonpi added a commit to clintonpi/swift-nio that referenced this issue Oct 9, 2024
@clintonpi
[NIOFileSystem] Avoid crash when reading more than ByteBuffer capacity
18f70f7 
Motivation:

As described in issue [apple#2878](apple#2878), NIOFileSystem crashes when reading more than `ByteBuffer` capacity.

Modifications:

- Add a new static property, `byteBufferCapacity`, to `ByteCount` representing the maximum amount of bytes that can be written to `ByteBuffer`.
- Throw a `FileSystemError` in `ReadableFileHandleProtocol.readToEnd(fromAbsoluteOffset:maximumSizeAllowed:)` when `maximumSizeAllowed` is more than `ByteCount.byteBufferCapacity`.

Result:

NIOFileSystem will `throw` instead of crashing.
@clintonpi clintonpi mentioned this issue Oct 9, 2024
Merged
glbrntt pushed a commit that referenced this issue Oct 10, 2024
@clintonpi
Throw error when the max read amount is greater than ByteBuffer can…
611fa09 
… tolerate (#2911)

Motivation:

As described in issue
[#2878](#2878), NIOFileSystem
crashes when reading more than `ByteBuffer` capacity.

Modifications:

- Add a new static property, `byteBufferCapacity`, to `ByteCount`
representing the maximum amount of bytes that can be written to
`ByteBuffer`.
- Throw a `FileSystemError` in
`ReadableFileHandleProtocol.readToEnd(fromAbsoluteOffset:maximumSizeAllowed:)`
when `maximumSizeAllowed` is more than `ByteCount.byteBufferCapacity`.

Result:

NIOFileSystem will `throw` instead of crashing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@weissi @glbrntt