Android APp storage usage is very High, APP crashing episodes etc..

[ebildebil]

HI,

I am on the latest version of Immich server and Android apps. Server v1.120.2 and App1.120.1.
I only have external libraries.
Size of Library is about 2.4TB.
179K photos (resolutions upto 50MP and 8k) and 19K videos. (resolutions upto 4k.)

Immich is using 822GB for Thumbnails and trascoded videos on the server.

On my phone, xiaomi 14, Immich app regularly uses about 20-35GB of storage in cache. I have enabled prefer remote images on phone app. Since it becomes so large, I keep deleting the cache very week.

Is this kind of High Storage usage Normal? Especially on phone?

Also, The android App crashes everynow and then while browing the library.

I have scan jobs schedued every 15 mins. Main Immich is running on a nuc13 + 64GB ram and serves only the API and has the DB and redis. (Ofcourse had other contianers runing but has 51GB RAM available)

h# free -h
               total        used        free      shared  buff/cache   available
Mem:            62Gi       9.9Gi       1.4Gi       898Mi        51Gi        51Gi
Swap:           95Gi       2.7Gi        93Gi

The ML and transcoding is done on 2 other machines with an rtx490 and rtx3060 and with 192GB and 128GB RAM.
So, the API machine does not seem to be resource constrained.

Another error i saw in the microservices container on the 4090 machine:

2024-11-13 00:00:07 [Nest] 7  - 11/13/2024, 12:00:07 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: ./lib/jpegli/decode_scan.cc:539: Failed to decode DCT block
2024-11-13 00:00:07 [Nest] 7  - 11/13/2024, 12:00:07 AM   ERROR [Microservices:JobService] Error: VipsJpeg: ./lib/jpegli/decode_scan.cc:539: Failed to decode DCT block

Any advise or suggestions?

[NicholasFlamy]

That's a much larger library than most users. This probably has not been tested before.

[NicholasFlamy]

But, 823 GB for a 2.4TB library, feels, quite much for thumbs and pretranscoded videos.

For that I suggest changing your thumbnail and transcoding settings.

[NicholasFlamy]

But, 823 GB for a 2.4TB library, feels, quite much for thumbs and pretranscoded videos.

For that I suggest changing your thumbnail and transcoding settings.

By this I mean in the immich admin settings.

[ebildebil]

But, 823 GB for a 2.4TB library, feels, quite much for thumbs and pretranscoded videos.

For that I suggest changing your thumbnail and transcoding settings.

By this I mean in the immich admin settings.

Got it. I will try this when there is more time. Sort of dread re-doing it for my entire library.
Currently Thimbnails are all WebP, 720 and preview are at 4k. Quality at 80.

Videos are all set to transcode at -crf=28, Preset=fast, Codec=HEVC, Audio=AAC, target resolution=1080p, maxbitrate=5000k

[NicholasFlamy]

I'd suggest dropping WebP quality and resolution
My suggestions:

(Sorry for the weird screenshot, I took it on my phone.)

For videos, that seems like good settings, but maybe setting the preset to medium (will use less space but take longer to encode) or setting the crf higher (lower quality and filesize but same speed) could help.

[ebildebil]

Thank you. I will try this. No apologies needed, I am grateful for your engagement. :)
For video's I'll also see if I can play around a bit with b-frames and ref frames. I had used t-darr for batch processing stuff a couple of years back. time to try some of those old settings and see what works well.

[NicholasFlamy]

I have enabled prefer remote images on phone app. Since it becomes so large, I keep deleting the cache very week.

I think that would load thumbnails from the server and cache them, instead of loading thumbnails from a local copy of the asset. So I think it does nothing unless you have that 2.4 TB collection on your phone.

[ebildebil]

I can ofcourse try to disable "prefer remote images" and see how that works. But considering I have only external libraries (i.e, assets not on phone, but on my server/nas etc), would that really help? I will anyhow try and report back.

I still feel, regardless of the library size, there should be some way for the app to consume space more reasonably. The entire library on phone is ofcourse impossible :-)

[ebildebil]

I will anyhow try and report back.

Now I have disabled - Prefer remote images. Cleared all Storage cache. It dropped from 19.6GB to 145 MB.
Switched to 5G on the phone.
Browsed around for a few minutes and played a few videos and the app is already at 7GB. (All in about ~45 mins)

[NicholasFlamy]

Yep, don't think that will help. I think an option to disable caching or limit caching is needed.

[alextran1502]

Since we want the app to be able to be used in offline conditions, the thumbnail is cached, which is why the storage is high.

If you have a lot of videos, the transcode video space can be pretty high. Transcoding is for browser playback compatibility, not to reduce the file size

[ebildebil]

Thank you for the response Alex. Makes sense.
Perhaps there can be an option to not cache or limit/rotate cache for users that can't set aside this kind of space and access it while online? Portable devices have limited storage. Is that something that can be considered?

For transcoded videos, I agree with the reasoning that it makes it browser compatible.
But, often times if I am not in the same lan as the server, a high size of the transcoded video is very difficult to play because of bandwith constraints.
Browser compatibility does not help much if the bandwidth isn't available to stream, which is also a general day to day reality. These days, even kids and other family members take videos at high resolutions and albums are shared across regions.
1080p or 4k shots can easily become 5GB for 10-15 min clip. An option for, On the fly stanscoding and HLS can help.

[cheeregowh]

I’m experiencing a oppsite issue. I often access external networks, and the transfer speed is quite slow. Every time I open the app after some time, the previously downloaded preview images are automatically cleared. As a result, my device has to reconnect to the NAS network to re-download the preview images, causing delays in loading them, which negatively impacts the user experience.

What could be the reason for this? Doesn't the app have a caching mechanism? I noticed that within a few hours of closing the app, the storage usage of the app doesn’t change, and the preview images can be quickly displayed when I reopen it. However, after a longer period, such as the next day, I find that the thumbnails have been cleared, and the app’s storage usage decreases, indicating that the cached data was automatically cleared.