How to handle multipart/form-data with Oathkeeper

[Intevel]

How to handle multipart/form-data with Oathkeeper

Currently I am trying to send larger files with a POST request, the request goes through Oathkeeper.
However, the connection breaks after about 5 seconds.

I'm trying to figure out how Oathkeeper deals with this or how it will be dealt with in the future.
Unlike with Websockets, Oathkeeper stops the requests after about 5 seconds ignoring Connection: keep-alive

SSE Writes

This problem is related to #877, the connection keep-alive header is simply ignored. Oathkeeper should respect the Connection: keep-alive even if the HTTP read timeout is set.

Read Timeout

Obviously, if you set the read timeout higher, the request will work. However, this is not a real fix. In general it should be clear that you don't know how long a file upload takes

serve:
  proxy:
    timeout:
      read: 60s

Is there a way to configure this as rule, and not for every route?

---

Long story, short.

In general, this seems to be a problem, even with other proxy software. However, file uploads are important, and increasing the timeout is not a proper solution.

How will Oathkeeper handle this in the future, or is there currently already a way to handle it better? Maybe I'm missing something. As said in #877 (comment) and #877 (comment) the Connection: keep-alive header should be implemented or Content-Type streams should ignore the timeout settings.

[Intevel]

@aeneasr

[vinckr]

Hello @Intevel
sorry for the late answer, this was not an easy question. To be honest I don't know enough to give you a confident answer here.

Ory Oathkeeper does allow for the configuration of the Content-Type header in its error handling pipeline. This includes the multipart/form-data MIME type. Here is an example of how you can specify the matching conditions for the Content-Type header:

config:  
 when:  
 - request:  
 header:  
 content_type:  
 - multipart/form-data  
 # OR  
 - application/x-www-form-urlencoded  
 # OR  
 - application/json

If this configuration key is left empty, then all Content-Type headers will match.

For more detailed information, I would recommend checking the documentation.

By the way Ory now also offers support for companies running Ory software on-premise, read more here: https://www.ory.sh/support/

[Intevel]

The problem in general is not that the requests with files are blocked, the problem is that it blocks on longer uploads. I would like to disable the HTTP timeout for multipart form data completely.

[vinckr]

Have you had a look at this issue: #441? Its a bit older but there are still plans to revamp, and your input could be valuable there.

[Intevel]

Sorry for the late reply, surely I will give some input in #441.

Nevertheless, I have the problem now, and that I can turn off the timeout should be a config option. Waiting for that to come with the next generation of Oathkeeper would be a bit annoying.

[Intevel]

I could set the read-timeout up to 290 years, but actually I don't want to set the duration for the full server.
https://pkg.go.dev/time#Duration

My Idea was to give users the option to change the read-timeout on specific routes, not for the whole server.

I think this will be difficult due to the limitation of the Go HTTP server.
Because the Go HTTP Server sets the limitation for the whole server: https://github.com/ory/oathkeeper/blob/cae282488070984986391fb496f614f0c802e895/cmd/server/server.go#L85C4-L85C15