ResourceHandler cannot handle files over 2GB in size.

[joakime]

Jetty version(s)
12.1.0.alpha1

Jetty Environment
core

Java version/vendor (use: java -version)

openjdk 17.0.11 2024-04-16
OpenJDK Runtime Environment Temurin-17.0.11+9 (build 17.0.11+9)
OpenJDK 64-Bit Server VM Temurin-17.0.11+9 (build 17.0.11+9, mixed mode, sharing)

OS type/version
Linux / Ubuntu

Description
When using ResourceHandler from jetty core (no environment), and attempting to serve a file over 2GB in size, the size is truncated to exactly 2GB in size, the rest of the bytes are not even attempted to be sent.

The Resource Base Directory

$ ls -la docroot/
total 2097168
drwxrwxr-x 2 joakim joakim       4096 Jan  8 13:36 .
drwxrwxr-x 3 joakim joakim       4096 Jan  8 13:36 ..
-rw-rw-r-- 1 joakim joakim 2147483657 Jan  8 13:36 huge.mkv

Request

GET /context/huge.mkv HTTP/1.1
Host: local
Connection: close

Response

HTTP/1.1 200 OK
Date: Wed, 08 Jan 2025 19:35:12 GMT
Last-Modified: Wed, 08 Jan 2025 19:35:12 GMT
Accept-Ranges: bytes
Content-Length: 2147483647
Connection: close

Notice that the file on disk is 10 bytes larger than 2GB, but those last 10 bytes are not sent.

[joakime]

I added a testcase to branch fix/12.1.x/resourcehandler-over-2GB that demonstrates this bug.

See commit 1fa8f95

[joakime]

Also of note, the response has no Content-Type header.

I chose huge.mkv to specifically not fall into a known mime-type.
Perhaps this should have been returned as application/octet-stream as a fallback if no mime-type is known?

[joakime]

Testing on jetty-12.0.x HEAD with the same unit test shows a different bug.

See branch fix/12.0.x/resourcehandler-over-2GB for testcase (introduced in commit b620df7)

Exception in thread "main" java.lang.OutOfMemoryError: Required array length 2147483639 + 180 is too large
	at java.base/jdk.internal.util.ArraysSupport.hugeLength(ArraysSupport.java:752)
	at java.base/jdk.internal.util.ArraysSupport.newLength(ArraysSupport.java:745)
	at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:100)
	at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:132)
	at [email protected]/org.eclipse.jetty.server.LocalConnector$LocalEndPoint.waitForResponse(LocalConnector.java:460)
	at [email protected]/org.eclipse.jetty.server.LocalConnector$LocalEndPoint.waitForResponse(LocalConnector.java:357)
	at [email protected]/org.eclipse.jetty.server.LocalConnector.getResponse(LocalConnector.java:225)
	at [email protected]/org.eclipse.jetty.server.LocalConnector.getResponse(LocalConnector.java:185)
	at [email protected]/org.eclipse.jetty.server.handler.ResourceHandlerTest.testOver2GBFile(ResourceHandlerTest.java:780)

[joakime]

The 12.0.x branch is working fine.

I updated the testcase to not use LocalConnector due to the size of the response. (see commit eba500a)

The 12.1.x branch is still broken.