Best practice to pause ARDK camera requests to Android OS when the app is onPause

Include the following details (edit as applicable):

  • Issue category: ARDK AR Camera Management
  • Device type & OS version: Android 10/11
  • Host machine & OS version: Windows
  • Issue Environment : On Device
  • ARDK version: 2.0.1
  • Unity version: 2021.3.6f1

Description of the issue:

Hi. If I launch an intent from my ARDK app to open another app that uses the camera (let’s say I want to share a screencapture in SnapChat, for instance), my app goes to the background but keeps requesting for camera access to the Android OS (it calls the CameraManager.openCamera method). During a few seconds the OS throws warnings like this, once by call:

E/ACameraManager: openCamera: connect camera device failed: Status(-8, EX_SERVICE_SPECIFIC): '8: connectHelper:1760: Too many cameras already open, cannot open camera "0"'
I/native: I0000 00:00:1658946133.719157   24090 logger.h:28] CameraMetrics: OpenCamera: 2.557917ms
I/native: I0000 00:00:1658946133.719335   24089 cameras.cc:1385] Camera 0 changed state RECOVERING->CLOSED; Camera device closed successfully. i=17
I/native: I0000 00:00:1658946133.719418   24089 cameras.cc:1385] Camera 0 changed state CLOSED->RECOVERING; Attempting recovery while opening the camera from:  Code=-10011; Message=RESOURCE_EXHAUSTED: ACameraManager_openCamera ACAMERA_ERROR_MAX_CAMERA_IN_USE [type.googleapis.com/util.ErrorSpacePayload='CameraStatusErrorSpaceClass::ACAMERA_ERROR_MAX_CAMERA_IN_USE'] i=18
I/native: I0000 00:00:1658946134.219405   24090 logger.h:28] CameraMetrics: CloseCamera: 886ns
E/ACameraManager: openCamera: connect camera device failed: Status(-8, EX_SERVICE_SPECIFIC): '8: connectHelper:1760: Too many cameras already open, cannot open camera "0"'
I/native: I0000 00:00:1658946134.221045   24090 logger.h:28] CameraMetrics: OpenCamera: 1.469687ms
I/native: I0000 00:00:1658946134.221211   24089 cameras.cc:1385] Camera 0 changed state RECOVERING->CLOSED; Camera device closed successfully. i=19
I/native: I0000 00:00:1658946134.221258   24089 cameras.cc:1385] Camera 0 changed state CLOSED->RECOVERING; Attempting recovery while opening the camera from:  Code=-10011; Message=RESOURCE_EXHAUSTED: ACameraManager_openCamera ACAMERA_ERROR_MAX_CAMERA_IN_USE [type.googleapis.com/util.ErrorSpacePayload='CameraStatusErrorSpaceClass::ACAMERA_ERROR_MAX_CAMERA_IN_USE'] i=20
I/native: I0000 00:00:1658946134.721273   24090 logger.h:28] CameraMetrics: CloseCamera: 938ns
E/ACameraManager: openCamera: connect camera device failed: Status(-8, EX_SERVICE_SPECIFIC): '8: connectHelper:1760: Too many cameras already open, cannot open camera "0"'
I/native: I0000 00:00:1658946134.724415   24090 logger.h:28] CameraMetrics: OpenCamera: 2.868125ms
I/native: I0000 00:00:1658946134.724731   24089 cameras.cc:1385] Camera 0 changed state RECOVERING->CLOSED; Camera device closed successfully. i=21
I/native: I0000 00:00:1658946134.724775   24089 cameras.cc:1385] Camera 0 changed state CLOSED->RECOVERING; Attempting recovery while opening the camera from:  Code=-10011; Message=RESOURCE_EXHAUSTED: ACameraManager_openCamera ACAMERA_ERROR_MAX_CAMERA_IN_USE [type.googleapis.com/util.ErrorSpacePayload='CameraStatusErrorSpaceClass::ACAMERA_ERROR_MAX_CAMERA_IN_USE'] i=22
W/CameraManagerGlobal: ignore the torch status update of camera: 2
W/CameraManagerGlobal: ignore the torch status update of camera: 4
W/CameraManagerGlobal: ignore the torch status update of camera: 5
W/CameraManagerGlobal: ignore the torch status update of camera: 6

Eventually, after several tries, Android OS disables the camera “by policy”. That either means the device has an enterprise policy installed that disables cameras (that’s not my case) or, on recent Android releases, the app is trying to open the camera in the background.

E/ACameraManager: openCamera: connect camera device failed: Status(-8, EX_SERVICE_SPECIFIC): '6: connectHelper:1818: Camera "0" disabled by policy'
I/native: I0000 00:00:1658946135.229534   24090 logger.h:28] CameraMetrics: OpenCamera: 4.447448ms
I/native: I0000 00:00:1658946135.229746   24089 cameras.cc:1385] Camera 0 changed state RECOVERING->CLOSED; Camera device closed successfully. i=23
I/native: I0000 00:00:1658946135.229790   24089 cameras.cc:1385] Camera 0 changed state CLOSED->ERROR; Could not open camera:  Code=-10012; Message=UNAVAILABLE: ACameraManager_openCamera ACAMERA_ERROR_CAMERA_DISABLED [type.googleapis.com/util.ErrorSpacePayload='CameraStatusErrorSpaceClass::ACAMERA_ERROR_CAMERA_DISABLED'] i=24

As a result of the camera being disabled, when I resume my app, the ar camera feed is either frozen or black.

The camera doesn’t get disabled if I launch intents to apps that don’t use camera by default (for instance whatsapp), but it seems that ARDK keeps doing camera requests and, eventually, it throws something like this:

E/native: E0000 00:00:1658944724.346748   21336 camera_image_stream.cc:270] FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata.
    === Source Location Trace: === 
    third_party/arcore/ar/camera/android/metadata_cache.cc:52
    third_party/arcore/ar/camera/android/metadata_extraction.cc:53
    
    === Source Location Trace: === 
    third_party/arcore/ar/camera/android/camera_image_stream.cc:204
E/native: E0000 00:00:1658944724.347638   21338 camera_image_stream.cc:270] FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata.
    === Source Location Trace: === 
    third_party/arcore/ar/camera/android/metadata_cache.cc:52
    third_party/arcore/ar/camera/android/metadata_extraction.cc:53
    
    === Source Location Trace: === 
    third_party/arcore/ar/camera/android/camera_image_stream.cc:204
E/native: E0000 00:00:1658944724.380299   21336 camera_image_stream.cc:270] FAILED_PRECONDITION: Failed to extract the native metadata, status=DEADLINE_EXCEEDED: Timed out waiting for metadata.
    === Source Location Trace: === 
    third_party/arcore/ar/camera/android/metadata_cache.cc:52
    third_party/arcore/ar/camera/android/metadata_extraction.cc:53

Around 50% of the times that this last error happens, my ARDK app closes too.

I’ve tried to disable the ARDK camera requests by:

  • Calling DisableFeatures() on AR Session Manager and AR Rendering Manager
  • Disabling Unity Camera Gameobject (the AR Camera)
  • Disabling all AR Gameobjects in my scene.

But it didn’t work. I’ve also tried to stop the camera device directly on Android, overwriting the onPause unity activity method, but it doesn’t matter because, even if I close the camera device, the Unity activity keeps requesting to open it again on the background.

I’ve also added this permission to the Android Manifest to try to avoid the camera being disabled “by policy”:

<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>

But it didn’t help either.

I would like to share videos of my ARDK game, but this is blocking me a little bit, since I don’t want the user to be oblied to restart the app 50% of the times after sharing a video. However, I am quite lost with this issue, so I would like to know if you can help me to solve it. This is happening, at least, in Android 10 and 11 devices.

Thank you very much in advance.

I made a mistake when disabling AR Session Manager features. I was enabling them back right after. I modified it and now it seems that, once I call arSessionManager.DisableFeatures(), ARDK stops rendering the camera feed, and no calls are made to open the camera device. That solves my problem.

I’ll keep the issue open for a few days, in case someone from Niantic could confirm if this approach is correct or if there are better alternatives. Thanks in advance.

That’s great news Chankeiro! We’ll make sure to note this in our own best practices documents going forward

Thanks @Bill_McLaughlin , but I was actually asking for the best practice or Niantic’s recommendation. Disabling arSessionManager seems to work in this case , but I am not sure if that covers all the situations or if that’s even a good approach. What do you think?