Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

AR Fallback Tech ( when unsupported or tracking lost )

Discussion in 'AR' started by chantey, Sep 2, 2020.

  1. chantey

    chantey

    Joined:
    Mar 5, 2017
    Posts:
    49
    Hi All

    Working with the ARFoundation platform, understandibly sometimes the SLAM algorithm loses positional and rotation tracking when facing featureless surfaces.

    Using the existing Input.gyro.attitude we seem to be able to detect the absolute rotation of a device consistently and with high precision.

    My question is this: Should the ARFoundation platform have an option to fallback to IMU processing when SLAM fails or isn't supported? While it doesn't solve positional tracking, it would certainly smooth out the experience, especially with interpolation between the systems.

    Before hacking a solution together I wanted to run this by the community to hear thoughts and/or existing solutions.
     
  2. chantey

    chantey

    Joined:
    Mar 5, 2017
    Posts:
    49
    For anyone interested the solution turned out to be surprisingly simple and has some major benefits:

    • 6DOF pointing to true north
    • Backwards compatibility with older phones
    • Smooth rotation when tracking is lost
    • Easy testing using Unity Remote (3dof)
    All that is needed is two extra scripts: one for getting the device IMU rotation and the other for aggregating the IMU and AR inputs: rotation from IMU and position from SLAM.

    PS Getting the camera background to render in the second Unity camera is a different story, maybe one for another post.

    IMU + Camera version - Unity Remote


    ARFoundation version - 6DOF with IMU



    Code for Device Attitude.
    Note this must be parented by a transform with a rotation of x=90

    Code (CSharp):
    1.     public class DeviceAttitude : MonoBehaviour
    2.     {
    3.         [Header("parent rotation must = (90,0,0)")]
    4.         public bool _;
    5.  
    6.         void Start()
    7.         {
    8.             if (SystemInfo.supportsGyroscope)
    9.             {
    10.                 Input.gyro.enabled = true;
    11.                 Input.gyro.updateInterval = 0.0167f;// set the update interval to it's highest value (60 Hz)
    12.                 Debug.Log($"DeviceAttitude - Gyro Initialized");
    13.             }
    14.             else
    15.                 Debug.Log($"DeviceAttitude - Gyro Not Supported");
    16.         }
    17.         void Update()
    18.         {
    19.             var qrh = Input.gyro.attitude;// right handed to left handed
    20.             var qlh = new Quaternion(qrh.x, qrh.y, -qrh.z, -qrh.w);
    21.             transform.localRotation = qlh;
    22.         }
    23.     }
    Code for Pose Aggregator:
    Code (CSharp):
    1.  
    2. [ExecuteAlways]
    3. public class DevicePoseAggregator : MonoBehaviour
    4. {
    5.  
    6.     public Transform poseAr;
    7.     public Transform poseImu;
    8.     public Transform originAgg;
    9.     public Transform poseAgg;
    10.  
    11.     void Update()
    12.     {
    13.         var qAgg = poseImu.rotation * Quaternion.Inverse(poseAr.localRotation);
    14.         originAgg.rotation = qAgg;
    15.         var posAgg = originAgg.TransformPoint(poseAr.localPosition);
    16.         poseAgg.position = posAgg;
    17.         poseAgg.rotation = poseImu.rotation;
    18.     }
    19. }
    20.  
     
    bsrarb likes this.