Hardcoded decode cache makes logging jpeg defacto more memory intensive than raw images

[Wumpf]

If a user sends all their images as jpeg, e.g. rr.log_image(f'{label}', img, jpeg_quality=75), they will observe higher memory usage until the tensor decode cache fills up than they would have with uncompressed images.

This cache has a hardcoded size right now of 4gb on desktop and 1gb on web

rerun/crates/re_viewer_context/src/tensor/tensor_decode_cache.rs

Line 62 in be7a031

impl Cache for TensorDecodeCache {

A bunch of action points:

• document decoding caching behavior to not make any false promises (point out that viewer may use more memory, saved files won't, etc.)
• make cache size configurable
• default to a smaller cache - 4gb seems excessive. The reason we have a cache as big is that jpeg decoding is too slow for smooth playback. There's a bunch of ways to address this
  • look into faster decompression libraries
  • more clever caching than just a size budget
    • if an image wasn't used for very long it should be evicted
    • don't cache an image if its decode didn't take very long to begin with
  • ?

[Wumpf]

A whole different solution for this popped up now:
In #7457 we successfully read back texture values from a video. The code is written in a way that we could do this with any rgb texture (and formats in the future).
Also, it turns out that it seems that by now the only reason we have to keep around decompressed images is inspection via mouse.
-> We can make this a gpu-memory only cache, thus solving this issue

this ofc still means up to 4gb of vram used by the cache for which all previous notes are still relevant, but this solves the issue at hand nonetheless. On a system wide perspective (very relevant for shared ram/vram architectures) it halves the memory needed.
In my experience running out of vram also swaps fine into ram without issues.

[Wumpf]

turns out our decode cache is cpu-only right now, creating gpu textures only on the fly if needed.
That makes above idea a lot less attractive. I still think it would be the right move, but also means we don't get "half the memory consumption" either.