🐛 [V3] (Android) NullPointerException Camera2 #1785

bjornhutter · 2023-09-08T09:00:09Z

What's happening?

Hi! First and foremost thanks for the great work on this library <3

I think i've might stumbled upon a Camera2 problem, it seems related to other issues mentioned in this Camera2 Rewrite PR #1674

I'm getting this error pretty much as soon as the camera goes active. E.g. if i set isActive={false} the component renders, but when i change to isActive={true} the crash happens.

I'm using Expo development builds (SDK 49) and EAS to build the app but all that seems fine.

I've not been able to test it in the example app because i can't seem to get it to build properly. But my current project is more or less basically the same, the difference being that it's built through Expo EAS.

I've prebuilt the code and tried looking for the problem and it seems to originate from CameraSession.kt:

private suspend fun updateRepeatingRequest() {
    mutex.withLock {
      val session = captureSession
      if (session == null) {
        // Not yet ready. Start session first, then it will update repeating request.
        startRunning()
        return
      }

      val fps = fps
      val videoStabilizationMode = videoStabilizationMode
      val lowLightBoost = lowLightBoost
      val hdr = hdr

      val repeatingRequest = getPreviewCaptureRequest(fps, videoStabilizationMode, lowLightBoost, hdr)
      Log.d(TAG, "Setting Repeating Request..")
      session.setRepeatingRequest(repeatingRequest, null, null) // <-- Error right here (Line:548)
    }
  }

Note: I've used "react-native-vision-camera": "^2.15.4", before on my ASUS Zenfone 8 without problems.

Reproduceable Code

// CameraPage.tsx it's basically the same as the example app.
<ReanimatedCamera
  style={[StyleSheet.absoluteFill, styles.camera]}
  device={device}
  isActive={true}
/>

// Some maybe relevant packages (package.json):
"expo": "^49.0.0",
"react-native": "0.72.4",
"react-native-vision-camera": "^3.0.0",
"react-native-worklets-core": "^0.2.0",
"react-native-reanimated": "~3.3.0",

// babel.config.json
module.exports = function (api) {
  api.cache(true);
  return {
    presets: ['babel-preset-expo'],
    plugins: [
      'react-native-reanimated/plugin',
      'react-native-worklets-core/plugin',
    ],
  };
};

Relevant log output

LOG  Loading react-native-worklets-core...
LOG  Worklets loaded successfully
LOG  re-rendering camera page without active camera
LOG  re-rendering camera page without active camera
LOG  re-rendering camera page without active camera
LOG  re-rendering camera page without active camera
LOG  re-rendering camera page without active camera
LOG  Re-rendering camera page with active camera. Device: "BACK (0)" (4560x2052 @ 30fps)
LOG  Re-rendering camera page with active camera. Device: "BACK (0)" (4560x2052 @ 30fps)
LOG  Camera initialized!
LOG  Re-rendering camera page with active camera. Device: "BACK (0)" (4560x2052 @ 30fps)
ERROR  Your app just crashed. See the error below.
java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.Integer.intValue()' on a null object reference
  android.hardware.camera2.impl.CameraDeviceImpl.setRepeatingRequest(CameraDeviceImpl.java:1342)
  android.hardware.camera2.impl.CameraCaptureSessionImpl.setRepeatingRequest(CameraCaptureSessionImpl.java:330)
  com.mrousavy.camera.core.CameraSession.updateRepeatingRequest(CameraSession.kt:548)
  com.mrousavy.camera.core.CameraSession.startRunning(CameraSession.kt:526)
  com.mrousavy.camera.core.CameraSession.access$startRunning(CameraSession.kt:51)
  com.mrousavy.camera.core.CameraSession$startRunning$1.invokeSuspend(Unknown Source:14)
  kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
  kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
  android.os.Handler.handleCallback(Handler.java:942)
  android.os.Handler.dispatchMessage(Handler.java:99)
  android.os.Looper.loopOnce(Looper.java:241)
  android.os.Looper.loop(Looper.java:358)
  android.os.HandlerThread.run(HandlerThread.java:67)

Camera Device

{
  "formats": [REMOVED BECAUSE OUTPUT IS TOO LONG - WILL NOT ACCEPT TICKET],
  "hardwareLevel": "level-3",
  "pixelFormats": [
    "RAW_SENSOR",
    "JPEG",
    "PRIVATE",
    "YUV_420_888",
    "RAW_PRIVATE",
    "RAW10",
    "HEIC"
  ],
  "maxZoom": 4,
  "minZoom": 1,
  "supportsLowLightBoost": false,
  "sensorOrientation": "landscape-right",
  "supportsDepthCapture": false,
  "neutralZoom": 1,
  "supportsFocus": true,
  "supportsRawCapture": true,
  "isMultiCam": false,
  "hasTorch": true,
  "devices": [
    "wide-angle-camera"
  ],
  "hasFlash": true,
  "name": "BACK (0)",
  "position": "back",
  "id": "0"
}

Device

ASUS Zenfone 8, Android 13

VisionCamera Version

3.0.0

Can you reproduce this issue in the VisionCamera Example app?

I can reproduce the issue in the VisionCamera Example app.

Additional information

I am using Expo
I have enabled Frame Processors (react-native-worklets-core)
I have read the Troubleshooting Guide
I agree to follow this project's Code of Conduct
I searched for similar issues in this repository and found none.

mrousavy · 2023-09-11T09:30:34Z

Huh this is weird. What happens if you remove pretty much all the captureRequest.set calls from getPreviewCaptureRequest, so that it only returns an empty/unconfigured Capture Request? does that crash?

bjornhutter · 2023-09-11T10:51:59Z

After commenting out all captureRequest.set calls it does not crash anymore and the camera renders. I'll try to pin point which call (or calls) that might cause the error.

mrousavy · 2023-09-11T11:51:48Z

Yea please try that out and let me know, thank you!

bjornhutter · 2023-09-12T10:14:38Z

Hi again and thanks for the help 👊

Okay I think i've localized the issue, It seems to be related to Video Stabilization.

If I comment out this line:

captureRequest.set(CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE, videoStabilizationMode?.toDigitalStabilizationMode())

The app does not crash anymore, e.g. it fixes this error:

java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.Integer.intValue()' on a null object reference

But when I try to start a recording it fails on this (no crash):

LOG  Camera initialized!
LOG  Re-rendering camera page with active camera. Device: "BACK (0)" (4560x2052 @ 30fps)
LOG  calling startRecording()...
LOG  called startRecording()!
LOG  Re-rendering camera page with active camera. Device: "BACK (0)" (4560x2052 @ 30fps)
LOG  Re-rendering camera page with active camera. Device: "BACK (0)" (4560x2052 @ 30fps)
ERROR  Recording failed! {"cause": {"message": "prepare failed.", "stacktrace": "java.io.IOException: prepare failed.
        at android.media.MediaRecorder._prepare(Native Method)
        at android.media.MediaRecorder.prepare(MediaRecorder.java:1416)
        at com.mrousavy.camera.core.RecordingSession.start(RecordingSession.kt:92)
        at com.mrousavy.camera.core.CameraSession.startRecording(CameraSession.kt:269)
        at com.mrousavy.camera.CameraView_RecordVideoKt.startRecording(CameraView+RecordVideo.kt:47)
        at com.mrousavy.camera.CameraViewModule$startRecording$1.invokeSuspend(CameraViewModule.kt:85)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
"}, "code": "capture/unknown", "message": "An unknown error occurred while trying to start a video recording! prepare failed.", "userInfo": null}

If i comment out BOTH of the Video Stabilization calls I can record videos again e.g:

// Video Stabilization
// captureRequest.set(CaptureRequest.CONTROL_VIDEO_STABILIZATION_MODE, videoStabilizationMode?.toDigitalStabilizationMode()) <-- This gives java.lang.NullPointerException
// captureRequest.set(CaptureRequest.LENS_OPTICAL_STABILIZATION_MODE, videoStabilizationMode?.toOpticalStabilizationMode()) <-- This makes start recording fail, (if the other one is commented out?)

The rest of the captureRequests.set calls seems to be working properly.

mrousavy · 2023-09-13T11:02:33Z

That's interesting. And you're 100% sure that the device/format you're using supports the video stabilization mode you're trying to use? As in, the format you're passing to the <Camera> has the stabilization mode listed here:

react-native-vision-camera/package/src/CameraDevice.ts

Lines 122 to 125 in 4e96eb7

    
             /** 
        
              * All supported video stabilization modes 
        
              */ 
        
             videoStabilizationModes: VideoStabilizationMode[];

?

bjornhutter · 2023-09-14T10:15:38Z

This is format that I'm passing to the <Camera>:

{
  "pixelFormats": [
    "rgb",
    "native",
    "yuv",
    "unknown"
  ],
  "videoStabilizationModes": [
    "off",
    "standard",
    "off",
    "cinematic-extended"
  ],
  "autoFocusSystem": "contrast-detection",
  "supportsPhotoHDR": false,
  "photoHeight": 2052,
  "supportsVideoHDR": false,
  "maxISO": 3200,
  "minISO": 25,
  "minFps": 1,
  "videoWidth": 3840,
  "videoHeight": 2160,
  "photoWidth": 4560,
  "fieldOfView": 66.53434119576576,
  "maxFps": 30
}

The modes are listed in the type so that seems fine? The weird thing being that there are 2 "off" modes?
export type VideoStabilizationMode = 'off' | 'standard' | 'cinematic' | 'cinematic-extended' | 'auto';

I'm not using the videoStabilizationMode prop on the <Camera> component, does it have to be explicitly set to something?

FYI: I'm resolving the format the exact same way as in the example app.

mrousavy · 2023-09-14T11:47:32Z

Did you try setting videoStabilizationMode on the <Camera> component to see if that resolved the issue?

bjornhutter · 2023-09-14T11:54:01Z

I'm on it, will report back asap!

bjornhutter · 2023-09-14T12:31:31Z

Yeah explicitly setting the videoStabilizationMode on the <Camera> seems to have resolved the issue 😄🙌

If I'm passing an empty string "", null, undefined or a mode that does not exist on the format being passed to the camera in my case "auto" for instance to the videoStabilizationMode prop I get the same null pointer exception again java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.Integer.intValue()' on a null object reference

I also (kind of) found the culprit for the startRecording issue that I got before:

 ERROR  Recording failed! {"cause": {"message": "prepare failed.", "stacktrace": "java.io.IOException: prepare failed.
        at android.media.MediaRecorder._prepare(Native Method)
        at android.media.MediaRecorder.prepare(MediaRecorder.java:1416)
        at com.mrousavy.camera.core.RecordingSession.start(RecordingSession.kt:92)
        at com.mrousavy.camera.core.CameraSession.startRecording(CameraSession.kt:269)
        at com.mrousavy.camera.CameraView_RecordVideoKt.startRecording(CameraView+RecordVideo.kt:47)
        at com.mrousavy.camera.CameraViewModule$startRecording$1.invokeSuspend(CameraViewModule.kt:85)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
"}, "code": "capture/unknown", "message": "An unknown error occurred while trying to start a video recording! prepare failed.",

The problem seems to be cause by having the startRecording React.useCallback async e.g.

// Using async caused the start recording to fail
const startRecording = React.useCallback(async () => {
  // ...
try {
      // ...

      camera.current.startRecording({
        flash,
        onRecordingError: (error) => {
          console.error('Recording failed!', error);
          onStoppedRecording();
        },
        onRecordingFinished: (video) => {
          console.log(`Recording successfully finished!' ${video.path}`);
          onMediaCaptured(video, 'video');
          onStoppedRecording();
        },
      });

      // ...

// This works
const startRecording = React.useCallback(() => {
  // ...

But the weird thing being that I could start recording when commenting out both of the video stabilization captureRequest calls and still having the start recoding callback be async... ? 🫠

mrousavy · 2023-09-14T20:37:11Z

Interesting, yea I probably need to add some sanity checks before that so the user can't make those mistakes. Thanks for your help here, imma leave this open until I added those checks!

bjornhutter · 2023-09-15T06:52:30Z

Yeah that sounds good. Thanks all the same for the help troubleshooting and your work on the library, it's much appreciated 💪

mrousavy · 2024-01-13T19:09:41Z

Hey! I think this is fixed in the latest PR where I added some sanity checks!

bjornhutter added the 🐛 bug Something isn't working label Sep 8, 2023

mrousavy closed this as completed Jan 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 [V3] (Android) NullPointerException Camera2 #1785

🐛 [V3] (Android) NullPointerException Camera2 #1785

bjornhutter commented Sep 8, 2023

mrousavy commented Sep 11, 2023

bjornhutter commented Sep 11, 2023

mrousavy commented Sep 11, 2023

bjornhutter commented Sep 12, 2023

mrousavy commented Sep 13, 2023

bjornhutter commented Sep 14, 2023

mrousavy commented Sep 14, 2023

bjornhutter commented Sep 14, 2023

bjornhutter commented Sep 14, 2023

mrousavy commented Sep 14, 2023

bjornhutter commented Sep 15, 2023

mrousavy commented Jan 13, 2024

🐛 [V3] (Android) NullPointerException Camera2 #1785

🐛 [V3] (Android) NullPointerException Camera2 #1785

Comments

bjornhutter commented Sep 8, 2023

What's happening?

Reproduceable Code

Relevant log output

Camera Device

Device

VisionCamera Version

Can you reproduce this issue in the VisionCamera Example app?

Additional information

mrousavy commented Sep 11, 2023

bjornhutter commented Sep 11, 2023

mrousavy commented Sep 11, 2023

bjornhutter commented Sep 12, 2023

mrousavy commented Sep 13, 2023

bjornhutter commented Sep 14, 2023

mrousavy commented Sep 14, 2023

bjornhutter commented Sep 14, 2023

bjornhutter commented Sep 14, 2023

mrousavy commented Sep 14, 2023

bjornhutter commented Sep 15, 2023

mrousavy commented Jan 13, 2024