I’m looking to render the AR camera to a texture to be able to run some additional computer vision algorithms and/or image processing.
With just arfoundation it’s quite easy to acquire an AR camera frame by CPU or GPU (Image capture | AR Foundation | 5.1.1), but with Niantic’s ARDK patching/overwriting arfoundation functions this doesnt seem to be possible.
ARDK 2.5 still documented how to do this using the ARRenderingManager and also the ARKitFrameRenderer in the ARDK2.5 Examples and also on this forum under “Get AR Scene Camera output to Texture / Material + to Camera”
None of this seems to work for ardk3+. When going down into the rabbit hole I get to UnityEngine.XR.ARSubsystems > XRCameraSubsystem which is what links the unity camera to the device camera, but here I can see that a function like TryAcquireLatestCpuImage exists but doesn pass it on to arcore, but instead throws a “NotSupportedException: getting camera image is not supported by this implementation”
Any suggestions where else to look or what other methods to use? I basically just want one camera render both to the screen as well as a rendertexture I can use for additional processing.
whole stack trace:
NotSupportedException: getting camera image is not supported by this implementation
Niantic.Lightship.AR.Subsystems.Playback.LightshipPlaybackCameraSubsystem+LightshipPlaybackProvider.TryAcquireLatestCpuImage (UnityEngine.XR.ARSubsystems.XRCpuImage+Cinfo& cinfo) (at Library/PackageCache/com.nianticlabs.lightship@25b2beb33d/Runtime/Subsystems/Playback/LightshipPlaybackCameraSubsystem.cs:264)
UnityEngine.XR.ARSubsystems.XRCameraSubsystem.TryAcquireLatestCpuImage (UnityEngine.XR.ARSubsystems.XRCpuImage& cpuImage) (at Library/PackageCache/com.unity.xr.arfoundation@5.1.0/Runtime/ARSubsystems/CameraSubsystem/XRCameraSubsystem.cs:494)
UnityEngine.XR.ARFoundation.ARCameraManager.TryAcquireLatestCpuImage (UnityEngine.XR.ARSubsystems.XRCpuImage& cpuImage) (at Library/PackageCache/com.unity.xr.arfoundation@5.1.0/Runtime/ARFoundation/ARCameraManager.cs:299)
ARCameraToTexture.OnCameraFrameReceived (UnityEngine.XR.ARFoundation.ARCameraFrameEventArgs eventArgs) (at Assets/ARCameraToTexture.cs:31)
UnityEngine.XR.ARFoundation.ARCameraManager.InvokeFrameReceivedEvent (UnityEngine.XR.ARSubsystems.XRCameraFrame frame) (at Library/PackageCache/com.unity.xr.arfoundation@5.1.0/Runtime/ARFoundation/ARCameraManager.cs:491)
UnityEngine.XR.ARFoundation.ARCameraManager.Update () (at Library/PackageCache/com.unity.xr.arfoundation@5.1.0/Runtime/ARFoundation/ARCameraManager.cs:351)
Appreciate any help. Thanks!
(sorry first post so not allowed to put more links for extra context)