feat: support zstd compression in miniostorage

[joseph-sentry]

we want to use zstd compression when compressing files for storage in
object storage because it performs better than gzip which is what we
were using before

these changes are only being made to the minio storage service because
we want to consolidate the storage service functionality into this one
so both worker and API will be using this backend in the future (API was
already using this one)

we have to manually decompress the zstd compressed files in read_file
but HTTPResponse takes care of it for us if the content encoding of the
file is gzip

the is_already_gzipped argument is being deprecated in favour of
compression_type and is_compressed, also the ability to pass a str to
write_file is being deprecated. we're keeping track of the use of these
using sentry capture_message

Fixes: codecov/engineering-team#2257

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.06%. Comparing base (00e90bb) to head (54bfa03).
Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #405      +/-   ##
==========================================
- Coverage   90.69%   90.06%   -0.63%     
==========================================
  Files         394      324      -70     
  Lines       12002     9251    -2751     
  Branches     2018     1649     -369     
==========================================
- Hits        10885     8332    -2553     
+ Misses       1028      858     -170     
+ Partials       89       61      -28     

Flag	Coverage Δ
shared-docker-uploader	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Swatinem]

As we have this abstraction in place for all the different storage backends, we should add support for this to the other variants as well.
Reason is that API is using the minio backend to access GCS. I have no idea how that even works, but thats the code that runs in production, and we have to make sure things work out correctly using that.

Other than that, I left some individual comments mostly in the tests about:

• The is_already_gzipped flag should assume gzip
• We should verify that the application/x-gzip handling works properly
• This applies zstd decompression only in the "return as bytes" codepath, but not the "save to file".

[Swatinem]

We have gone a few iterations on the archive service, and there is a bunch of PRs open:

• this one
• Clean up Archive Storage Tests #351
• Clean up ArchiveService #415

Now the circumstances have changed a little in the sense that it looks like we might want to consolidate the storage layer as only using minio as the storage adapter, as that already runs against the GCS server in api and staging worker.
So we can just remove all the other storage adapters.

IMO, we could:

• take Clean up ArchiveService #415 as that is some general cleanup and reduces the surface area (I would like to land this together with Store v2 reports in GCS instead of Redis codecov-api#960, as that changes the user of said APIs)
• remove all the other storage adapters, moving exclusively to minio
• take the remaining parts of Clean up Archive Storage Tests #351, which would remove all the cumbersome vcr tests and replace those with running against a live minio server in docker
• adapt this PR to add a better story around compression, and move the default to zstd
• then as a followup, we can also consolidate the ArchiveService layer across worker, api and shared (the latter two should be handled by Store v2 reports in GCS instead of Redis codecov-api#960)

[Swatinem]

I wonder if an uptodate urllib3 would also handle zstd automatically for us.
We have quite an outdated urllib3 dependency primarily for reasons of compatiblity with vcr snapshots.

I’m pretty sure that Content-Encoding: zstd is properly standardized and transparently handled by browsers by now for example.

[Swatinem]

Another thing I remembered is that we should add a test that Content-Encoding: gzip; Content-Type: application/x-gzip works correctly.
The gcp backend had a workaround specifically for that. I suspect that whatever http abstraction that was using was not doing the transparent decompression correctly, and also triggered a bogus checksum mismatch error in that case.

[Swatinem]

I was looking up Content-Encoding: zstd support:

• According to https://caniuse.com/zstd, its supported in all the latest browsers except Safari (proving once again that Safari is the new IE6 in terms of lagging behind on features and being a pain for webdev)
• ☝🏻 because of that, we might consider sticking to gzip when "rewriting uploads as readable", as those are the only files which are downloadable by customers and support folks. all other files are purely internal and won’t ever be exposed to customers directly.
• It is indeed supported out of the box in urllib > 2.0.0: https://urllib3.readthedocs.io/en/2.1.0/changelog.html#added, https://urllib3.readthedocs.io/en/2.1.0/advanced-usage.html#zstandard-encoding
• ☝🏻 so as long as we are stuck with a <2 version, your workaround to do zstd decompression manually is needed
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding#directives lists a few more officially supported options here. Like I mentioned in review, the initial upload is user-initiated through the CLI or bash script, so those clients are free to chose whatever they want, and we have to support that accordingly.

[Swatinem]

nice, this looks great!

feel free to add a test for gzip chunking, and/or add an explicit check for the urllib3 version.
otherwise its also fine to do those as followups, and lets ship this (carefully to staging first :-)