🐛 The resolution of the PhotoFile is different than the format photo resolution #3031
Comments
|
Guten Tag, Hans here. Note New features, bugfixes, updates and other improvements are all handled mostly by |
|
Hi @mrousavy, our company (Spaces Experiences) has been a sponsor for around four months and we are very happy that this library exists, so thank you! This issue I posted here has priority for us, if you have any time taking a look would be much appreciated. |
|
Hi! |
|
Hi, thanks for your quick reply. Yes, with only |
|
fwiw visionCamera version 4.3.2 with my Galaxy S22 doesn't have this issue. I tried disabling/enabling video and preview, and verified both front and back camera format and photo resolutions match. |
|
Yes good point @JustinHaut, I also tested on a Galaxy A41 and there it does indeed also match. |
|
Okay thanks, good research on This might fix this issue. |
|
@mrousavy I can reproduce it with the example app with Pixel Fold Pixel 8 Pro Pixel 7a Samsung Galaxy A53 And I cannot reproduce it on these devices: Samsung Galaxy S23 Ultra SHARP AQUOS sense2 SH-01L |
Good to hear! Let me know if can help in any way once you get to it (like testing or something). To add I used the |
|
The problem here is that the Formats API on Android is not 100% accurate, which sucks - I know. On Android, there is no API to figure out which combination of output resolutions is supported. On iOS there is, which is why the But on Android, it's always only an estimate, the OS might not be able to support 4k Photo and 4k Video at the same time, only individually. I try to get an accurate list of supported video/photo resolutions here; ..but again, CameraX might choose a different resolution if it feels like it. I raised this concern with the Google/CameraX team, and after many months they came back to me and said they are going to implement an API that actually gives you accurate resolution (and FPS) combinations - which is what VisionCamera's format API is about. So we might see this sometime soon, but for now this is all we have on Android unfortunately. |
|
Hey, thanks for your explanation and your time! That's a bummer.... seems kind of stupid for Android to not have such a API, but I don't know too much of the Android camera world so idk 🤷 (maybe it's really hard to implement with all the different vendors who might not fully comply) It does make it really hard to manually select a format, since the format you select can be one that is not compatible thus resulting in the photo resolution (and sometimes aspect ratio) being different than expected. When looking at the Pixel 7a it's a massive difference instead of 4000x2000 photo you get a 1920x1080 photo. So you think you are getting the highest resolution picture while instead you might have a format which actually has a lower photo resolution than maybe a other one where the video resolution is lower (which is compatible). At least if I understand it correctly? 🤔 I was thinking is there a running issue for this in the CameraX issue tracker, maybe we as community could upvote it so that they might prioritize this? Furthermore, I was wondering do you have any advice on how to deal with this? Like should formats with a high resolution and and high video ouput be avoided (because they might be invalid)? Yet again, thanks for your time. |
|
Formats with high resolutions should not be avoided, no. But the useCameraFormat/getAvailableCameraDevices should just be accurate in telling us which video/photo resolution combinations are supported. |
|
Thanks for your reply @mrousavy! Hopefully, the Android team will pick this up somewhere in the near future. I misunderstood some things on how this works and have since thought about it some more. |
|
This has been changed recently and now works exactly like on iOS - preview will try to match the format's video resolution and aspect ratio, with the priority being the aspect ratio. |
|
Hey can you check if release 4.4.0 changed anything? |
|
Hey, thanks again for you effort! I just tested it on my device, and indeed the preview confirms to the format's video resolution and aspect ratio. The issue with the resolution being lower for the |
|
The code from the sample project boils down to this: val rotation = fragmentCameraBinding.viewFinder.display.rotation
// CameraProvider
val cameraProvider = cameraProvider ?: throw IllegalStateException("Camera initialization failed.")
// CameraSelector
val cameraSelector =
CameraSelector.Builder().requireLensFacing(CameraSelector.LENS_FACING_BACK).build()
// Preview
val previewResolutionSelector =
ResolutionSelector.Builder()
.setAllowedResolutionMode(ResolutionSelector.PREFER_CAPTURE_RATE_OVER_HIGHER_RESOLUTION)
.build()
preview =
Preview.Builder()
.setResolutionSelector(previewResolutionSelector)
.setTargetRotation(rotation)
.build()
// ImageCapture
val imageCaptureResolutionSelector =
ResolutionSelector.Builder()
.setAllowedResolutionMode(ResolutionSelector.PREFER_HIGHER_RESOLUTION_OVER_CAPTURE_RATE)
.build()
imageCapture =
ImageCapture.Builder()
.setCaptureMode(ImageCapture.CAPTURE_MODE_MAXIMIZE_QUALITY)
.setResolutionSelector(imageCaptureResolutionSelector)
.setTargetRotation(rotation)
.build()
// Must unbind the use-cases before rebinding them
cameraProvider.unbindAll()
try {
// A variable number of use-cases can be passed here -
// camera provides access to CameraControl & CameraInfo
camera = cameraProvider.bindToLifecycle(this, cameraSelector, imageCapture, preview)
val previewResolution = preview?.resolutionInfo?.resolution
if (previewResolution != null) {
Log.i("DingenisCamera", "${previewResolution.width}x${previewResolution.height}")
}
val imageCaptureResolution = imageCapture?.resolutionInfo?.resolution
if (imageCaptureResolution != null) {
Log.i("DingenisCamera", "${imageCaptureResolution.width}x${imageCaptureResolution.height}")
}
// Attach the viewfinder's surface provider to preview use case
preview?.setSurfaceProvider(fragmentCameraBinding.viewFinder.surfaceProvider)
observeCameraState(camera?.cameraInfo!!)
} catch (exc: Exception) {
Log.e(TAG, "Use case binding failed", exc)
}
Which gives me the following preview and image capture resolution, this matches the highest resolution format from react-native-vision-camera (but there CameraX choses a different resolution): |
|
Alright @mrousavy, I think I found the culprit! I was playing around with the configuration of the In react-native-vison-camera the format says it has a |
|
Oh wow interesting, great research! I'm thinking what a good solution to this would be, maybe I'll just always allow minFps of 20... |
|
Thanks! Good question, well to set it to a min of 20 seems like a simple solution, so i like it 😁😂 . I don't know if there are any other formats which have an even lower Another solution could be to change the |
|
I can also confirm that lowering the Pixel 7a Pixel Fold |
|
@Dingenis let's do this;
cool? |
|
Created a PR to automatically fix this: #3063 |
|
@mrousavy Nice! 🔥 I think this is intended behaviour, looking at the description of
The line In combination with how |
|
Ahh okay - so I think we shouldn't hardcode it to 20 then, but instead use the minFps value here.... wdyt? |
|
Yes, I think that could be nice! I do wonder how that affects the device, for example that high resolution format for my Galaxy A53 has a min fps of 8. Will the device chose to output the highest fps for that resolution if we have a range of [8, 30]? Or will it act "lazy" and chose a lower fps while a higher fps is possible? I mean that sounds insane but it's unclear from the API documentation on how that works. |
|
It seems that it choses the highest value possible (link) |
|
One thing I'm worried about is if devices have multiple overlapping FPS Ranges. For example, a device could report
I'm not sure if devices do this (could you confirm by logging fps ranges in the native kotlin code for CameraDevice?), but if it does, what would the difference between [8, 30] and [12, 30] be? Should I choose 8 or 12 for min FPS if the user wants 30 FPS? |
|
Sure thing @mrousavy, here you go! From what I can find, is that on Android the camera itself manages it's frame rate internally, so using Say the user really wants 30 fps for like video capture, then I think this is a argument for changing the It would be a lot nicer if each format would have a accurate |
|
Awesome thank you so much. Created a PR to use |
|
I now wonder if this is a good idea to always have a low minFps. So now with this change, all existing apps will throttle their frame rates in dark lighting conditions. |
|
So I have two ideas:
|
|
With option 2 you have full control, with option 1 you cannot select the min FPS - it will always be the minimum from the format. Which might not be a good idea since we don't always want to drop as low as 8, sometimes we just want a smaller range like [20, 30]. I mean yea we can also select a format with that range tbh. But maybe on some devices there are no multiple FPS ranges, only one that says like [1, 30]... |
|
See #3074 |
|
Good ideas @mrousavy! 🔥 |
What's happening?
When taking a photo the PhotoFile has a different resolution than the format.
Reproduceable Code
Relevant log output
Camera Device
{ "formats": [], "sensorOrientation": "landscape-right", "hardwareLevel": "limited", "maxZoom": 4, "minZoom": 1, "maxExposure": 20, "supportsLowLightBoost": false, "neutralZoom": 1, "physicalDevices": [ "wide-angle-camera" ], "supportsFocus": true, "supportsRawCapture": false, "isMultiCam": false, "minFocusDistance": 0, "minExposure": -20, "name": "1 (FRONT) androidx.camera.camera2", "hasFlash": false, "hasTorch": false, "position": "front", "id": "1" }Device
Samsung Galaxy A53 (Android 14)
VisionCamera Version
4.3.1
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information
The text was updated successfully, but these errors were encountered: