Resolved ARFoundation - Switching between front and rear camera

Hello,

I'm trying to see if its possible to switch between front and rear cameras while an ARSession is running. Looking at the ARFoundation samples I can't seem to figure out how to explicitly select front or rear camera. Any help would be greatly appreciated!

Thank you

 

ARFoundation does not let you choose a camera explicitly. ARFoundation provides a platform abstraction to ARCore and ARKit. It is up to these SDKs to choose a hardware camera. Currently, ARCore only uses the rear-facing camera. ARKit will use the front-facing camera for face tracking, but otherwise use the rear-facing camera.

What are you trying to do? Why do you need to be able to select the camera?

 

Thank you for your response! I'm trying to create an AR experience that works with both front and rear cameras (where available). I want to switch between them just like you would with a default camera app.

I can get this to work by having two separate scenes (one for front cam, one for rear cam) and using SceneManager to switch between the two, but if a more elegant solution is possible that'd be awesome.

 

Any progress on this? Some clients are asking for AR features to work on both cameras. Would be great to be able to switch them at runtime; especially with ARKit 3's new set of features. Of course; the AR features depending on the camera and devices (as only A12 chips and newer can handle all of ARKit3's features)

 

You can swap by enabling and disabling different trackable managers of ARKit. For example - if you only turn on the face manager and turn everything else off - it automatically flips to the selfie cam. If you turn on plane tracking - it will automatically flip to the "world" camera.

So rather than explicitly saying which camera to use - you implicitly make the decision by telling ARKit what kinds of things you want to track and it figures out the rest.

 

Oh great. Thank you for that explanation. Indeed, it seems that AR Foundation is picking the camera based on the feature/manager chosen. One other issue seems to be the People Occlusion. The end goal I'm trying to do is have it act as a Segmentation filter on the selfie/front camera; cull out the background behind the user's face/shoulders. So far, switching managers to create such a segmentation filter seems to cause a runtime crash; my guess is because of forcing the hand of this flip. Was worried that maybe People Occlusion or even a more basic segmentation-like filter just wasn't doable on ARKit due to hardware limitations. However, I've been told that ARKit native builds indeed can handle such a thing on the front camera and that it might be a bug of ARFoundation currently (since People Occlusion and all all still so new/in beta).

I guess in short... hoping to be able to have the feature work on the front camera.

 

I would prefer explicit control of which camera is being used, front or back. For example now in ARKit 3.0 it supports simulaneous planar tracking using the rear camera, while getting data from the front camera at the same time, but it keeps automatically switching to showing the front camera if the face manager is enabled. That's not what I want.

 

Any update on this feature? Would like to be able to switch manually between front and back camera.

 

We are working on an update that will allow for more explicit selection of the camera.

 

is there a timeline for this feature?

 

AR Foundation packages version 4.0.0-preview.1 are now available and provide for more explicit selection of the camera.

 

Great, thank you!

 

Is this supported on iOS? Can't get the front cam image on an iPhone SE or iPhone X. No matter what mode I request, it keeps showing the rear camera.

Works on Android though.

 

For future readers, this is supported only on iphones with a true depth camera (X and up)

 

@todds_unity please add support for front camera on older ios devices, I need to be able to select difference cameras for different purposes, one of these purposes might simply be accessing the front camera with no additional requirements, I would like to be able to do this without needing an additional library. Thanks =)

 

i can't find how to selection of the camera function. i'm using Unity 2019.3.8f1 and AR Foundation4.0 preview. please explain how to select device camera.

 

See the ARCameraManager.requestedFacingDirection property.

 

Using ARFoundation 4 preview 3, which camera is being used keeps shifting for me between builds, sometimes facing the world, sometimes facing the user. I'm not changing it, it's set to "World" in the ARCameraManager.

 

Any thougths about choose between wide angle camera and default camera on devices like OnePlus 7 pro?

 

We need this feature too. Our use case is a social network based on AR, where people can place selfie videos in the world. Switching to the device camera for shooting these videos is too cumbersome and did not work well. So we found a way to record the AR camera instead. But we'd need to switch the camera on the fly. We wouldn't care if world tracking is restarted in this case. Yet we need an on-the-fly flip.

 

I tried to set this property to User at runtime, but the rear camera stays active. What might I miss here? I am using Unity 2019.3.15f1 on an iOS device.

 

This property will work if you disable current feature/manager script, such as AR Occlussion Manager script.

 

With AR Foundation 4, we exposed more details on configuring an AR session.

Not all features are compatible at the same time. In choosing a configuration, the maximum number of features decides which configuration will be active.

If you build a Unity project with "Development Build" enabled, you will see the set of features and which ones are supported versus not satisfied with the configuration.

For instance, in this example I am requesting front-facing camera, but I am also requesting "Environment Probes", "Auto-Focus", and "Environment Depth" which are all three rear-facing camera features on my iOS device. The log output contains:

However, if I disable both "Environment Probes" and "Environment Depth", then the configuration chooser will switch to the front-facing camera. That log output contains:

 

Hi Todd,

Thanks for the clarification. In my opinion it is unclear from the docs that the front facing configuration is only available on newer gen iphones with a TrueDepth camera. I think this is causing some confusion.

 

What is the recommended way of switching cameras at runtime? On ARCore (AR Foundation 4.0.2), I first tried switching using requestedFacingDirection. Then with having 2 cameras in the scene with only one of the camera game objects being active at a time. I also tried moving each camera object to a separate scene and only loading one scene at a time. In all cases, the first switch from world facing to user facing works, but as soon as I switch back, the app crashes with nothing left in the logs.

 

Hi Todd, we got the camera switch working, but ran into an issue on some iOS devices, i.e. the iPhone XR and 11 in particular. When we switch from rear to front and back again, and repeat this two times, the AR objects get "stuck" to the camera, i.e. they won't stay at their position in the world anymore. Seems as if world tracking gets disabled in this case.

I filed support ticket 1261310 for this, which also contains a project to illustrate the problem. In this project, you have to repeat the back and forth switch 2 times. In our actual app, the objects get stuck to the camera at the first time you switch from front to back.

Do you have any idea what might cause this behaviour? The very same code works reliably on an iPhone X and all my Android devices. But it fails on the more recent iOS devices, like the iPhone XR and 11. Any help would be highly appreciated.

 

Hi,

Is is possible to add a full dpcumentation on what extra is needed for migration of 3 to 4.1, as currently the camera is not operating given the information in the migration documentation.

Thank you in advance

Also the github demos are not working, are those supposed to be upgraded for the latest 4.1 version (in the latest preview branch) or are all pending, as nothing seem to work currently.

 

Hi Todds,
Can I ask about is that feasible to open both cameras simultaneously. My app requires eye laser features from the front camera and environment information and plane from the rear camera. It needs eye laser to interact with the element in the rear camera.

 

Hi,

I am switching the cameras by enabling/disabling face tracking and then restarting the AR components. It works well on an iPhone 11 but the same app is not changing between rear and front camera on an iPhone 8 and an iPhone 11 Pro Max.

Can you help me figuring out why is this happening?

 

How about the ability to use the "Back Telephoto Camera" of the iPhone X / XS with ARFoundation ?
I would like to switch between the "normal camera" to the "zoomed camera" at runtime.

Thanks a lot for your answer.

Nicolas.

 

Changing CameraManager.requestedFacingDirection causes crashes every now and then (after 10 - 20 switches).

Fixed it with stopping and starting the ARSession in a coroutine, something like this (pseudo):

ARSession.enabled = false;

WaitForEndOfFrame();

CameraManager.requestedFacingDirection = newFacingDirection;

WaitForEndOfFrame();

ARSession.enabled = true;

 

good practice @efge thank you.

do you have an iphoneX device? or an android?
is this feature supported for standard devices now, or probably not?

 

where do i find the AR Foundation configuration file?

 

Does AR Foundation support tracking other people's faces with the rear camera, or will it only track the user's face with the front camera regardless of which camera is being displayed?

 

Both ARCore and ARKit support face tracking only on the user-facing camera.

 

Thank you

 

i have built my app for ipad, and it uses the rear camera.
i only have face tracker active.
i use ar foundation 4.0.0.
on android phone it works as expected with the user facing camera.

any ideas why? could it be the iPad?

 

I’m having the same issue than shacharoz on an iPhone 12 mini. Tried with Unity 2019.4 LTS and with 2020.2, URP and default. I created the simplest project from scratch to get Unity to display the default AR face (as stated in many tutorials), and the camera that fires on execution is the rear one. So no face tracking at all, of course.

 

@ikermozos i have found that the ipad is not supported.
i found this text somewhere:
"on ARKit, we only support ARM 64, so you won't be able to target older 32-bit devices."
"Using Face Tracking requires compatible hardware (iPhone X or above)"

so my iPad is not supported.
but later I trued on an iPhone X and it worked as expected like on the android.

 

@shacharoz, thanks for the update! I’m trying with the latest iPhone, so it should be working. But at least I know that my issue is user error, so I’ll keep digging. I’ll start by taking a look at the demos and see if I can get those working, and take it from there. Cheers!

 

Just for the record, what solved my problem was to update AR Foundation and ARKit packages to 4.1.1 instead of 4.0.x version I was using. That made the 'Face Tracking' checkbox appear in the Build Settings, which was not there before. Checking that made everything work as it should.

 

Could you please told me, how can I get the following value at runtime

Requested Features: User Facing Camera, Rotation and Orientation, Environment Probes, Auto-Focus, Environment Depth
Supported Features: Rotation and Orientation, Environment Probes, Auto-Focus, Environment Depth
Requested features not satisfied: User Facing Camera

 

Are you sure it solved your problem? I updated the packages as you did and it still uses the rear camera. It even uses both. It uses the front for face tracking but the image renderered is from the rear camera but I need to see the face from the front camera

Edit : XCode console prints this

[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input...

[Subsystems] UnityARKit successfully registered Provider for ARKit-Input

[Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing...

[Subsystems] Failed to initialize subsystem ARKit-Meshing [error: 1]

2021-02-10 18:05:00.812423+0100 YAAARFace[526:51283] Unbalanced calls to begin/end appearance transitions for <UnityViewControllerStoryboard: 0x10541a400>.

UnloadTime: 2.044750 ms

2021-02-10 18:05:02.801408+0100 YAAARFace[526:51283] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x282fb7a80 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x2839ffd40 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Disabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=None collaboration=Disabled userFaceTracking=Enabled sceneReconstruction=None>

 

The trick to camera selection is to use a custom configuration selector, see this sample:
https://github.com/Unity-Technologi.../Assets/Scripts/CustomConfigurationChooser.cs

I can’t recall off hand, but it may need some adjustment to work correctly. The basic idea is to give preference to the camera type you are looking for.

 

ARKit doesn't support Environment Depth on User Facing Camera. You can check if all requested features can be supported simultaneously by using the XRSessionSubsystem.DetermineConfiguration(), here is an example:
https://forum.unity.com/threads/hum...ed-to-arfoundation-arkit.781406/#post-6723655

 

If a native implementation doesn't support two features simultaneously, you can't fix it by using a custom Configuration Chooser. With a custom configuration chooser, you can only decide what native configuration descriptor should be used for a set of requested features. More info here:
https://forum.unity.com/threads/hum...ed-to-arfoundation-arkit.781406/#post-6727495
https://docs.unity3d.com/Packages/c...1608749981#camera-and-tracking-mode-selection

 

Hi,

The switching of camera was working fine on ARFoundation 3.1.0 preview 7. When I upgraded it to ARFoundation 4.1.5, along with other packages, the front camera has stopped working.

In the AR Remote asset, the error that it throws when I try to switch to the front camera is:
"No active UnityEngine.XR.ARSubsystems.XRFaceSubsystem is available. Please ensure that a valid loader configuration exists.
UnityEngine.XR.ARFoundation.ARTrackableManager'4:OnEnable()"

I have the FaceTracking checkbox enabled in Player Settings > XR Plug in Management > AR Kit. None of the sample AR Foundation scenes are working with the front camera. It automatically opens the back camera.

Am I missing some migration step?

 

@renee_unity I think that's a bug with ARfoundation 4.1.5 but I might be wrong. Try deleting the XR Management Package cache and then closing/reopening Unity. See if that helps. It does for me when I get those kinds of errors.

 

@danUnity Thanks for the reply.

I tried deleting the XR Management Package cache but it did not work. I deleted all other AR Packages cache manually. I, also, cleaned the cache from Unity by Edit > Preferences > Gl Cache > Clean cache but the error is still the same and front camera does not work.

Could it be a Unity Editor version issue? I am using Unity 2019.4.20f1.

 

UPDATE:
After downgrading the ARFoundation and other packages from 4.1.5 to 4.1.3, the front camera is working again. [SOLVED]