Optimize part size for checksummed read

[monthonk]

The prefetcher stores data received from each input stream as a part in the part queue structure. Usually, the part size is pretty big (8 MB or more) and the checksum validation always has to be done against an entire part even if we only read a small portion of that part.

This makes checksummed read much slower than non-checksummed read. We could make it more efficient by making the part smaller or ideally align the part size to the read size so that we don't have to compute the checksum on unnecessary bytes.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

[jamesbornholt]

I think if you initialize preferred_part_size to 128KiB, this can just be

Suggested change

	let min_part_size = 128 * 1024;
	self.preferred_part_size = length.max(min_part_size);
	self.preferred_part_size = self.preferred_part_size.max(length);

Also worth a comment on why we chose 128KiB.

[jamesbornholt]

Also, should we set a maximum here, like 1MiB?

[monthonk]

For choosing 128k, it's the linux readahead size and seem to be a reasonable minimum value. I will update the comment, also happy to change if you have better suggestion.

I'm not sure we should put a maximum value though. If the read size is really big then we will have to combine data from multiple parts and the extend operation is quite expensive.

[monthonk]

but if we change the logic the use max value between last preferred_part_size and current length I think there should be a maximum, otherwise it will keep growing bigger.

[jamesbornholt]

The effective maximum is the client's part size anyway, so probably we should enforce it here just to be explicit.