Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Help - how does multipart works under the hood #1070

Open
Rantoledo opened this issue Nov 17, 2024 · 3 comments
Open

Help - how does multipart works under the hood #1070

Rantoledo opened this issue Nov 17, 2024 · 3 comments
Labels
help wanted Extra attention is needed

Comments

@Rantoledo
Copy link

💬 Question here

How does @fastify/multipart work?
I'm trying to upload a file approximately 20-50 mib size.
I'm planning on using multipart as a content type, so I used @fastify/multipart.
But, I have nginx reverse proxy, so I'm limited to 1mb to the request size.
My question is does multipart send the data in chunks? does it do that for me? my testing saying it doesn't, but just wanted to ask to make sure.

  • node version: 22
  • fastify version: 5.1.0
  • os: Linux
@Rantoledo Rantoledo added the help wanted Extra attention is needed label Nov 17, 2024
@Rantoledo Rantoledo changed the title Help Help - how does multipart works under the hood Nov 17, 2024
@climba03003
Copy link
Member

climba03003 commented Dec 13, 2024

My question is does multipart send the data in chunks?

Yes, by the nature of stream it is always send by chunk. The only difference is the chunk size.

does it do that for me?

By default, using the Stream API should be process by chunk.
However, @fastify/multipart offers the ability to buffer in memory. So, it may gives some negative image.

my testing saying it doesn't, but just wanted to ask to make sure.

You should probably post the code you test. There is no way to ensure your implementation is doing things right.

But, I have nginx reverse proxy, so I'm limited to 1mb to the request size.

Since your setup involve nginx as extra layer.
nginx by default buffered the whole request body before sending to next server.
You should either disable proxy_request_buffering or set proxy_http_version to 1.1.

That maybe the reason why your test always received the whole payload instead of chunk.

Refs: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_request_buffering
Refs: https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version

@Rantoledo
Copy link
Author

@climba03003
So, you're saying that each chunk is sent in a separate request, and the only reason my backend receives a single request with the entire file is because of nginx?

@climba03003
Copy link
Member

climba03003 commented Jan 3, 2025

Possibly, yes.

If you want to test things work properly, you should reduce the intermediate parts (e.g. nginx) between client and server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants