Search Unity

  1. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

Resolved ARFoundation - Switching between front and rear camera

Discussion in 'AR' started by edee1337, Jan 30, 2019.

  1. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    34
    Hello,

    I'm trying to see if its possible to switch between front and rear cameras while an ARSession is running. Looking at the ARFoundation samples I can't seem to figure out how to explicitly select front or rear camera. Any help would be greatly appreciated!

    Thank you
     
  2. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    604
    ARFoundation does not let you choose a camera explicitly. ARFoundation provides a platform abstraction to ARCore and ARKit. It is up to these SDKs to choose a hardware camera. Currently, ARCore only uses the rear-facing camera. ARKit will use the front-facing camera for face tracking, but otherwise use the rear-facing camera.

    What are you trying to do? Why do you need to be able to select the camera?
     
    shacharoz likes this.
  3. edee1337

    edee1337

    Joined:
    Apr 10, 2013
    Posts:
    34
    Thank you for your response! I'm trying to create an AR experience that works with both front and rear cameras (where available). I want to switch between them just like you would with a default camera app.

    I can get this to work by having two separate scenes (one for front cam, one for rear cam) and using SceneManager to switch between the two, but if a more elegant solution is possible that'd be awesome.
     
  4. princetrunks

    princetrunks

    Joined:
    Mar 13, 2015
    Posts:
    3
    Any progress on this? Some clients are asking for AR features to work on both cameras. Would be great to be able to switch them at runtime; especially with ARKit 3's new set of features. Of course; the AR features depending on the camera and devices (as only A12 chips and newer can handle all of ARKit3's features)
     
    MaryamKamel likes this.
  5. distastee

    distastee

    Joined:
    Mar 25, 2014
    Posts:
    58
    You can swap by enabling and disabling different trackable managers of ARKit. For example - if you only turn on the face manager and turn everything else off - it automatically flips to the selfie cam. If you turn on plane tracking - it will automatically flip to the "world" camera.

    So rather than explicitly saying which camera to use - you implicitly make the decision by telling ARKit what kinds of things you want to track and it figures out the rest.
     
    DrSharky and princetrunks like this.
  6. princetrunks

    princetrunks

    Joined:
    Mar 13, 2015
    Posts:
    3
    Oh great. Thank you for that explanation. Indeed, it seems that AR Foundation is picking the camera based on the feature/manager chosen. One other issue seems to be the People Occlusion. The end goal I'm trying to do is have it act as a Segmentation filter on the selfie/front camera; cull out the background behind the user's face/shoulders. So far, switching managers to create such a segmentation filter seems to cause a runtime crash; my guess is because of forcing the hand of this flip. Was worried that maybe People Occlusion or even a more basic segmentation-like filter just wasn't doable on ARKit due to hardware limitations. However, I've been told that ARKit native builds indeed can handle such a thing on the front camera and that it might be a bug of ARFoundation currently (since People Occlusion and all all still so new/in beta).

    I guess in short... hoping to be able to have the feature work on the front camera.
     
    distastee likes this.
  7. edwon

    edwon

    Joined:
    Apr 24, 2011
    Posts:
    221
    I would prefer explicit control of which camera is being used, front or back. For example now in ARKit 3.0 it supports simulaneous planar tracking using the rear camera, while getting data from the front camera at the same time, but it keeps automatically switching to showing the front camera if the face manager is enabled. That's not what I want.
     
  8. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    57
    Any update on this feature? Would like to be able to switch manually between front and back camera.
     
  9. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    171
    We are working on an update that will allow for more explicit selection of the camera.
     
    nilsdr and mariosubspace like this.
  10. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    353
    is there a timeline for this feature?
     
    DrSharky likes this.
  11. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    171
    AR Foundation packages version 4.0.0-preview.1 are now available and provide for more explicit selection of the camera.
     
    Fenikkel likes this.
  12. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    353
    Great, thank you!
     
  13. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    353
    Is this supported on iOS? Can't get the front cam image on an iPhone SE or iPhone X. No matter what mode I request, it keeps showing the rear camera.

    Works on Android though.
     
    Jokonut likes this.
  14. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    353
    For future readers, this is supported only on iphones with a true depth camera (X and up)
     
    DrSharky likes this.
  15. xzodia

    xzodia

    Joined:
    Jan 24, 2013
    Posts:
    45
    @todds_unity please add support for front camera on older ios devices, I need to be able to select difference cameras for different purposes, one of these purposes might simply be accessing the front camera with no additional requirements, I would like to be able to do this without needing an additional library. Thanks =)
     
    skoteskote likes this.
  16. Kideokk

    Kideokk

    Joined:
    Jan 18, 2018
    Posts:
    31
    i can't find how to selection of the camera function. i'm using Unity 2019.3.8f1 and AR Foundation4.0 preview. please explain how to select device camera.
     
  17. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    171
  18. DreamPower

    DreamPower

    Joined:
    Apr 2, 2017
    Posts:
    102
    Using ARFoundation 4 preview 3, which camera is being used keeps shifting for me between builds, sometimes facing the world, sometimes facing the user. I'm not changing it, it's set to "World" in the ARCameraManager.
     
  19. DanOrk

    DanOrk

    Joined:
    Nov 27, 2016
    Posts:
    1
  20. waldgeist

    waldgeist

    Joined:
    May 6, 2017
    Posts:
    221
    We need this feature too. Our use case is a social network based on AR, where people can place selfie videos in the world. Switching to the device camera for shooting these videos is too cumbersome and did not work well. So we found a way to record the AR camera instead. But we'd need to switch the camera on the fly. We wouldn't care if world tracking is restarted in this case. Yet we need an on-the-fly flip.
     
  21. waldgeist

    waldgeist

    Joined:
    May 6, 2017
    Posts:
    221
    I tried to set this property to User at runtime, but the rear camera stays active. What might I miss here? I am using Unity 2019.3.15f1 on an iOS device.
     
  22. CKong91

    CKong91

    Joined:
    Sep 11, 2015
    Posts:
    2
    This property will work if you disable current feature/manager script, such as AR Occlussion Manager script.
     
  23. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    171
    With AR Foundation 4, we exposed more details on configuring an AR session.

    Not all features are compatible at the same time. In choosing a configuration, the maximum number of features decides which configuration will be active.

    If you build a Unity project with "Development Build" enabled, you will see the set of features and which ones are supported versus not satisfied with the configuration.

    For instance, in this example I am requesting front-facing camera, but I am also requesting "Environment Probes", "Auto-Focus", and "Environment Depth" which are all three rear-facing camera features on my iOS device. The log output contains:
    However, if I disable both "Environment Probes" and "Environment Depth", then the configuration chooser will switch to the front-facing camera. That log output contains:
     
    shacharoz likes this.
  24. nilsdr

    nilsdr

    Joined:
    Oct 24, 2017
    Posts:
    353
    Hi Todd,

    Thanks for the clarification. In my opinion it is unclear from the docs that the front facing configuration is only available on newer gen iphones with a TrueDepth camera. I think this is causing some confusion.
     
    shacharoz likes this.
  25. Thomas-Mountainborn

    Thomas-Mountainborn

    Joined:
    Jun 11, 2015
    Posts:
    419
    What is the recommended way of switching cameras at runtime? On ARCore (AR Foundation 4.0.2), I first tried switching using requestedFacingDirection. Then with having 2 cameras in the scene with only one of the camera game objects being active at a time. I also tried moving each camera object to a separate scene and only loading one scene at a time. In all cases, the first switch from world facing to user facing works, but as soon as I switch back, the app crashes with nothing left in the logs.
     
    Last edited: Jun 30, 2020
  26. waldgeist

    waldgeist

    Joined:
    May 6, 2017
    Posts:
    221
    Hi Todd, we got the camera switch working, but ran into an issue on some iOS devices, i.e. the iPhone XR and 11 in particular. When we switch from rear to front and back again, and repeat this two times, the AR objects get "stuck" to the camera, i.e. they won't stay at their position in the world anymore. Seems as if world tracking gets disabled in this case.

    I filed support ticket 1261310 for this, which also contains a project to illustrate the problem. In this project, you have to repeat the back and forth switch 2 times. In our actual app, the objects get stuck to the camera at the first time you switch from front to back.

    Do you have any idea what might cause this behaviour? The very same code works reliably on an iPhone X and all my Android devices. But it fails on the more recent iOS devices, like the iPhone XR and 11. Any help would be highly appreciated.
     
  27. nasos_333

    nasos_333

    Joined:
    Feb 13, 2013
    Posts:
    9,962
    Hi,

    Is is possible to add a full dpcumentation on what extra is needed for migration of 3 to 4.1, as currently the camera is not operating given the information in the migration documentation.

    Thank you in advance

    Also the github demos are not working, are those supposed to be upgraded for the latest 4.1 version (in the latest preview branch) or are all pending, as nothing seem to work currently.
     
  28. alphay

    alphay

    Joined:
    Mar 9, 2018
    Posts:
    1
    Hi Todds,
    Can I ask about is that feasible to open both cameras simultaneously. My app requires eye laser features from the front camera and environment information and plane from the rear camera. It needs eye laser to interact with the element in the rear camera.
     
  29. tahafarooq

    tahafarooq

    Joined:
    Sep 18, 2015
    Posts:
    7
    Hi,

    I am switching the cameras by enabling/disabling face tracking and then restarting the AR components. It works well on an iPhone 11 but the same app is not changing between rear and front camera on an iPhone 8 and an iPhone 11 Pro Max.

    Can you help me figuring out why is this happening?
     
  30. x2stone

    x2stone

    Joined:
    Dec 17, 2014
    Posts:
    19
    How about the ability to use the "Back Telephoto Camera" of the iPhone X / XS with ARFoundation ?
    I would like to switch between the "normal camera" to the "zoomed camera" at runtime.

    Thanks a lot for your answer.

    Nicolas.
     
  31. efge

    efge

    Joined:
    Dec 13, 2007
    Posts:
    60
    Changing CameraManager.requestedFacingDirection causes crashes every now and then (after 10 - 20 switches).

    Fixed it with stopping and starting the ARSession in a coroutine, something like this (pseudo):

    ARSession.enabled = false;
    WaitForEndOfFrame();
    CameraManager.requestedFacingDirection = newFacingDirection;
    WaitForEndOfFrame();
    ARSession.enabled = true;
     
    shacharoz likes this.
  32. shacharoz

    shacharoz

    Joined:
    Jul 11, 2013
    Posts:
    65
    good practice @efge thank you.

    do you have an iphoneX device? or an android?
    is this feature supported for standard devices now, or probably not?
     
  33. shacharoz

    shacharoz

    Joined:
    Jul 11, 2013
    Posts:
    65
    where do i find the AR Foundation configuration file?

     
  34. MikePanoff

    MikePanoff

    Joined:
    Apr 29, 2015
    Posts:
    20
    Does AR Foundation support tracking other people's faces with the rear camera, or will it only track the user's face with the front camera regardless of which camera is being displayed?
     
  35. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    707
    Both ARCore and ARKit support face tracking only on the user-facing camera.
     
  36. MikePanoff

    MikePanoff

    Joined:
    Apr 29, 2015
    Posts:
    20
    Thank you
     
  37. shacharoz

    shacharoz

    Joined:
    Jul 11, 2013
    Posts:
    65
    i have built my app for ipad, and it uses the rear camera.
    i only have face tracker active.
    i use ar foundation 4.0.0.
    on android phone it works as expected with the user facing camera.

    any ideas why? could it be the iPad?
     
  38. ikermozos

    ikermozos

    Joined:
    Dec 5, 2019
    Posts:
    8
    I’m having the same issue than shacharoz on an iPhone 12 mini. Tried with Unity 2019.4 LTS and with 2020.2, URP and default. I created the simplest project from scratch to get Unity to display the default AR face (as stated in many tutorials), and the camera that fires on execution is the rear one. So no face tracking at all, of course.
     
  39. shacharoz

    shacharoz

    Joined:
    Jul 11, 2013
    Posts:
    65
    @ikermozos i have found that the ipad is not supported.
    i found this text somewhere:
    "on ARKit, we only support ARM 64, so you won't be able to target older 32-bit devices."
    "Using Face Tracking requires compatible hardware (iPhone X or above)"

    so my iPad is not supported.
    but later I trued on an iPhone X and it worked as expected like on the android.
     
  40. ikermozos

    ikermozos

    Joined:
    Dec 5, 2019
    Posts:
    8
    @shacharoz, thanks for the update! I’m trying with the latest iPhone, so it should be working. But at least I know that my issue is user error, so I’ll keep digging. I’ll start by taking a look at the demos and see if I can get those working, and take it from there. Cheers!
     
  41. ikermozos

    ikermozos

    Joined:
    Dec 5, 2019
    Posts:
    8
    Just for the record, what solved my problem was to update AR Foundation and ARKit packages to 4.1.1 instead of 4.0.x version I was using. That made the 'Face Tracking' checkbox appear in the Build Settings, which was not there before. Checking that made everything work as it should.
     
    shacharoz likes this.
  42. dol_habibur

    dol_habibur

    Joined:
    Apr 23, 2019
    Posts:
    1
    Could you please told me, how can I get the following value at runtime

    Requested Features: User Facing Camera, Rotation and Orientation, Environment Probes, Auto-Focus, Environment Depth
    Supported Features: Rotation and Orientation, Environment Probes, Auto-Focus, Environment Depth
    Requested features not satisfied: User Facing Camera
     
  43. GeniusKoala

    GeniusKoala

    Joined:
    Oct 13, 2017
    Posts:
    71

    Are you sure it solved your problem? I updated the packages as you did and it still uses the rear camera. It even uses both. It uses the front for face tracking but the image renderered is from the rear camera but I need to see the face from the front camera :(

    Edit : XCode console prints this

    [Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input...

    [Subsystems] UnityARKit successfully registered Provider for ARKit-Input

    [Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing...

    [Subsystems] Failed to initialize subsystem ARKit-Meshing [error: 1]

    2021-02-10 18:05:00.812423+0100 YAAARFace[526:51283] Unbalanced calls to begin/end appearance transitions for <UnityViewControllerStoryboard: 0x10541a400>.

    UnloadTime: 2.044750 ms

    2021-02-10 18:05:02.801408+0100 YAAARFace[526:51283] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x282fb7a80 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x2839ffd40 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Disabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=None collaboration=Disabled userFaceTracking=Enabled sceneReconstruction=None>

     
    Last edited: Feb 10, 2021
  44. xzodia

    xzodia

    Joined:
    Jan 24, 2013
    Posts:
    45
  45. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    707
    ARKit doesn't support Environment Depth on User Facing Camera. You can check if all requested features can be supported simultaneously by using the XRSessionSubsystem.DetermineConfiguration(), here is an example:
    https://forum.unity.com/threads/hum...ed-to-arfoundation-arkit.781406/#post-6723655
     
  46. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    707
    If a native implementation doesn't support two features simultaneously, you can't fix it by using a custom Configuration Chooser. With a custom configuration chooser, you can only decide what native configuration descriptor should be used for a set of requested features. More info here:
    https://forum.unity.com/threads/hum...ed-to-arfoundation-arkit.781406/#post-6727495
    https://docs.unity3d.com/Packages/c...1608749981#camera-and-tracking-mode-selection
     
  47. renee_unity

    renee_unity

    Joined:
    Feb 11, 2021
    Posts:
    3
    Hi,

    The switching of camera was working fine on ARFoundation 3.1.0 preview 7. When I upgraded it to ARFoundation 4.1.5, along with other packages, the front camera has stopped working.

    In the AR Remote asset, the error that it throws when I try to switch to the front camera is:
    "No active UnityEngine.XR.ARSubsystems.XRFaceSubsystem is available. Please ensure that a valid loader configuration exists.
    UnityEngine.XR.ARFoundation.ARTrackableManager'4:OnEnable()"

    I have the FaceTracking checkbox enabled in Player Settings > XR Plug in Management > AR Kit. None of the sample AR Foundation scenes are working with the front camera. It automatically opens the back camera.

    Am I missing some migration step?
     
  48. danUnity

    danUnity

    Joined:
    Apr 28, 2015
    Posts:
    216
    @renee_unity I think that's a bug with ARfoundation 4.1.5 but I might be wrong. Try deleting the XR Management Package cache and then closing/reopening Unity. See if that helps. It does for me when I get those kinds of errors.
     
    KirillKuzyk likes this.
  49. renee_unity

    renee_unity

    Joined:
    Feb 11, 2021
    Posts:
    3
    @danUnity Thanks for the reply.

    I tried deleting the XR Management Package cache but it did not work. I deleted all other AR Packages cache manually. I, also, cleaned the cache from Unity by Edit > Preferences > Gl Cache > Clean cache but the error is still the same and front camera does not work.

    Could it be a Unity Editor version issue? I am using Unity 2019.4.20f1.
     
  50. renee_unity

    renee_unity

    Joined:
    Feb 11, 2021
    Posts:
    3
    UPDATE:
    After downgrading the ARFoundation and other packages from 4.1.5 to 4.1.3, the front camera is working again. [SOLVED]
     
    Last edited: Mar 8, 2021
    shacharoz likes this.
unityunity