ImageWrapper overhead

[gabors]

Thanks for the excellent cache library.
When using for caching images I noticed that you take the UIImage data as PNG or JPEG then serialize it into a JSON like

{"object":{"image":"\/9j\/4AAQSkZJRgABAQAASABIAAD\/4QBYRXhpZ...

This transformation I'm afraid is expensive, the resulting cache files also are easily double in size from the original binary. Then when we retrieve the data from the cache we have to deserialize it again.

So what happens now:

1. Fetch data from URL
2. Load it into UIImage
3. Send it to Cache:
   3.1 Convert back to Data as PNG or JSON, depending on Alpha channel, again
   3.2 Wrap it into a Codable
   3.3 Serialize into JSON text (larger size)
   3.4 Save to cache (disk and memory)

then in reverse when retrieving..

I prefer the way your previous Swift 3 version of the library worked where it would cache the actual data we fetch from a URL.

[onmyway133]

@gabors Hi this is a valid concern. I will check it

[gabors]

Appreciate it.
I was hoping that if can save a Data instance the cache that would work but when I checked the code the DiskStorage uses a private DataSerializer static method to convert Data to JSON string.
IMHO if data was written to disk as a byte array instead we could skip a lot of unnecessary serialization.

[onmyway133]

@gabors I was using JSONEncoder and ImageWrapper to make all APIs the same. But I think we can implement our own Encoder and Container. I will have a look

[gabors]

Yeah I appreciate that.
In case of setObject if we pass in a T which is a Data we could pass that to fileManager directly perhaps? Skip serialization to JSON.

This would not work with the current ImageWrapper per se but if we receive a Data (which is an image) from a URL we could cache that directly.

func setObject<T: Codable>(_ object: T, forKey key: String, expiry: Expiry? = nil) throws { let expiry = expiry ?? config.expiry let data = try DataSerializer.serialize(object: object) let filePath = makeFilePath(for: key) _ = fileManager.createFile(atPath: filePath, contents: data, attributes: nil) try fileManager.setAttributes([.modificationDate: expiry.date], ofItemAtPath: filePath) }

[gabors]

@onmyway133

Hi, please take a look if you want at https://github.com/gabors/Cache/commit/4ece09fae2bb6dc278e9592c9e1c1d9ccce3e775

in my quick test with 110 thumbnail images, the ImageWrapper based cache took up 65MB and when saving the Data directly and bypassing serialization (for Data) the cache takes up 12MB.

[annjawn]

How about storing the images in the file system and just save the directory path (as a String) using Cache. Again, keep in mind that I am only caching some specific images only, for images appearing on tableview's and collections views through an internet URL I am using SDWebImage which has it's own internal caching and default picture mechanism. That is what I am doing in my production app and I have not noticed any overhead as well. I also maintain the expiration for the image file in the path using Cache and when I call removeExpiredObjects I also remove the image file from the file system.

In the previous version of my app, I was using the same mechanism but I was using UserDefaults since then I have migrated over to Cache. Cache is wonderful, but my idea is that caching large blobs of data beats the purpose of having fast accessible persistent data that it provides, as a suitable alternative I use the file system to store images or other large files and simply cache the file path using cache.

[gabors]

Storing the image data in the file system separately is an option but then you need to take care of expired files cleanup yourself, which sounds like a lot of work, and that's what this library is supposed to help with.

[annjawn]

@gabors agree, but remember that the idea behind Codable is to streamline how you access Structured data (data from a database or API and so on). Flat files, media files, video files etc. by nature are unstructured. Caching video, media or any unstructured files is a hot button issue with Moble App caching community where many people are completely for and against the idea (more against than for). In my opinion, your application shouldn't be holding large swaths of cache data (like videos or images), especially if it is temporary in nature. Small number of images (such as profile photos, profile banner photo etc) are excusable since they don't change all that frequently and are best candidates for storing in the device the way I explained, everything else is just a 'no bueno', but that's just me. By the way read/write from FS is not all that difficult, a small singleton in your project can achieve that honestly. But again, all of this depends on what your app's requirements are of course. 🙂

[gabors]

@annjawn My use case if caching hundreds of smallish thumbnail images that are accessed VERY frequently by the app so i cache them for several hours.

[annjawn]

Well in that case you need to look at the SDWebImage library. It has a pretty strong inbuilt caching system for images, compression, decompression options and much more. Very easy to use and well thought out library. I use both SDWebImage and Cache in my app which is live in AppStore and they both, although used for different purposes, work very well.

[gabors]

@annjawn Thanks I'll check it out again. Last time I looked it had a lot of functionality I did not need, as I have my own compression/decompression options.
All I was looking for was just caching the Data as-is on the file system after fetching from my secured API endpoints. Without any additional serialization.

[onmyway133]

@gabors Hi, sorry for late reply. I've just made a big refactoring to address this issue #189. Now we have the ability to save UIImage/NSImage without overhead, and the APIs are very type safe and flexible now

[onmyway133]

@gabors You can now take a look at https://github.com/hyperoslo/Cache/releases/tag/5.0.0