inflexible partial_post_size

[leoliu]

I am reviewing my setting of partial_post_size. I currently have it at 1048576 from a long time ago.

But since Yaws will always read the first chunk eagerly. It is neither performant nor safe given most of my POST data are within 10k except one. I have one route for media uploads accepting file size < 100M. Doing it at 10k per chunk is decidedly slow (which was the reason to bump to 1048576).

So Yaws needs to provide more flexibility i.e. allowing an appmod to decide how much to read since it has the knowledge to make the decision, for example, by allowing the get_more tuple to specify chunk size.

Another popular HTTP server cowboy doesn't read any POST data initially but provides a function cowboy_req:read_body/2 to do that when needed with complete control over chunk size.

[vinoski]

This doesn't address the programmatic flexibility portion of your issue, but perhaps use a separate virtual server just for the media uploads? Then it could have its own large partial_post_size setting.

[leoliu]

Indeed. To keep it simple I just bump partial_post_size for that server for now. But letting appmod decide chunk size is a good feature to have ;)