[BUG] Image metadata is lost when converting from AVCapturePhoto to UIIMage

[bogdan]

I am trying to extract the image metadata (specifically EXIF) from a photo made by the library.
I clearly see the metadata being present in AVCapturePhoto internally in the library.

But I can not see any metadata when working with UIImage object returned by the library.

Is there a way to preserve the metadata?

Ideally, I would also want the metadata to include geolocation, but it seems like it is not returned by AVCapturePhoto#capturePhoto and there is no way to configure it but to only manually attach it:

    let photoSettings = AVCapturePhotoSettings()

    if let location = location {
        let gpsMetadata = createGPSMetadata(from: location)
        photoSettings.embeddedThumbnailPhotoFormat = [AVVideoCodecKey: AVVideoCodecType.jpeg]
        var metadata = photoSettings.metadata
        metadata[kCGImagePropertyGPSDictionary as String] = gpsMetadata
        photoSettings.metadata = metadata
    }

    photoOutput.capturePhoto(with: photoSettings, delegate: self)

Please advice on geolocation if you want it to be part of the library or not.

[FulcrumOne]

Hey @bogdan,

I agree that in some cases this can indeed be useful, I will think about how to make it part of the library without losing the simplicity of the code; we could probably return a struct ImageData as a result, containing both metadata and UIImage.

As for the latter idea, I do agree that it could be useful, but it seems like we would need to think of a more abstract solution to avoid over-complicating the code; I think giving the developer the ability to customise both photo and video settings would be more suitable here. What is your opinion on this?

[bogdan]

Based on my research UIImage can contain the metadata so there is no need for additional object and API change.

The metadata is lost somewhere during these transformations:

        let capturedCIImage = prepareCIImage(ciImage, parent.attributes.cameraFilters)
        let capturedCGImage = prepareCGImage(capturedCIImage)
        let capturedUIImage = prepareUIImage(capturedCGImage)

While UIImage can be made directly from AVCapturePhoto (didn't try at runtime):

UIImage(data: photo.fileDataRepresentation())

As far as I can see, a more complex transformation currently used is to apply filters.
I believe even with filters, the metadata can be gradually transfered from photo.fileDataRepresentation() even with filters applyed.

Here is what ChatGPT suggested, but I didn't try. I have very little swift experience to try things quickly:

https://gist.github.com/bogdan/b461fb5a20aa161d10e996945db0ea20

[bogdan]

@FulcrumOne

I researched it a little more and found that UIImage wrapper kills metadata: whatever you put to the constructor of UIImage can have a metadata, but after a call to any of UIImage#pngData #jpegData #heicData the metadata is lost.
Moreover, re-encoding causes a quality loss too.

I didn't find any way to recover metadata from UIImage. So probably you are right about:

we could probably return a struct ImageData as a result, containing both metadata and UIImage

However, I see UIImage as the design issue as it enforces re-encoding.
It would be so much easier if the library would provide a callback like:

    func onImageCaptured(_ action: @escaping (UIImage, MCamera.Controller) -> ()) -> Self {
        config.imageCapturedAction = {
            action(UIImage(data: $0)!, $1)
        }
        return self
    }

    func onImageCaptured(_ action: @escaping (Data, MCamera.Controller) -> ()) -> Self { 
        config.imageCapturedAction = action
        return self 
    }

Instead of the one with UIImage argument. This will be easier to use as converting Data is easy: UIImage(data: ...).

You may maintain two callback formats easily deprecating the previous one if you want.