Search Unity

  1. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

Help Wanted ARFoundation/ARKit tracking quality: drifting objects

Discussion in 'AR' started by pawi-cmrg, Sep 7, 2021.

  1. pawi-cmrg


    Sep 3, 2021
    In our current project, we are overlapping real objects with their virtual representation. The objects are pretty small (2 by 15cm). We not only overlay the objects for visual representation, but also for measuring distances between the objects and try to reach an accuracy within millimetres.

    While this works quite well in most cases, we observe that sometimes the objects start to drift off when moving the device. The virtual objects are offset of the real objects in the same direction as the movement of the device, also going back to the original position when moving the device back. Until now, we did not find a way to detect if the tracking is stable or not.

    We are using Unity 2020.3, ARFoundation 4.1.7, ARKit XR Plugin 4.1.7 running on iOS 14.7.1 (iPad Pro 11’ 2. and 3. Generation). Strangely, we also find that some devices work better than others, see

    To nail this down we created a simple ARFoundation sample project. Using a Raycast we find a position (e.g. plane or feature point) to anchor a simple test object. In this simple sample project, we feel the stability of the tracking (moving the device and expecting the objects to be at a fixed position) to be working well for some objects, while observing the same ‘drifts’ we see in our main project.

    What we did try so far:
    • making sure the AR session is properly initialised (moving/orbiting the device around the scene)
    • make sure the environment is good regarding lighting, avoiding glass or other reflecting materials, etc)
    • adding ARAnchor to the objects
      • using gameobject.AddComponent<ARAnchor>()
      • using arAnchorManager.AddAnchor(position)
      • parenting the objects to ARSessionOrigin
    • testing on different devices
    • checking for drops in framerate: constant at 60fps
    Generally, we feel that adding ARAnchors to the object does not help a lot, or only in extreme situations (e.g. when moving the device completely out to of the scene, shaking it etc.). Then we notice that the trackingState of the anchors are .NONE when the anchors are created using the gameobject.AddComponent<ARAnchor>() method, but .TRACKING when created using the ARAnchorManager instance’s AddAnchor() method. No difference between tracked and non-tracked objects/anchors, though.

    The demo video (link below) shows the placed objects to slightly drift with the device movement. Red spheres (anchored) are at the same position as the slightly smaller grey cubes (no anchors).

    Code (CSharp):
    2. void Update()
    3. {
    4.     if (Input.touchCount > 0)
    5.     {
    6.         Touch touch = Input.GetTouch(0);
    7.         if (touch.phase == TouchPhase.Began)
    8.         {
    9.             if (arRaycastManager.Raycast(touch.position, _raycastHits, TrackableType.All))
    10.             {
    12.                 //adding a sphere at the first raycast hit and adding an ARAnchor to it
    13.                 GameObject placedObject = GameObject.CreatePrimitive(PrimitiveType.Sphere);
    14.                 placedObject.transform.localScale = Scale;
    15.                 placedObject.transform.position = _raycastHits[0].pose.position;
    16.                 placedObject.transform.SetParent(aRSessionOrigin.transform);
    18.                 ARAnchor anchor = placedObject.GetComponentInParent<ARAnchor>();
    19.                 if (anchor == null)
    20.                 {
    21.                     placedObject.AddComponent<ARAnchor>();
    22.                     Debug.Log($"New anchor added. ");
    23.                 }
    24.                 _placedObjects.Add(placedObject);
    26.                 //adding a slightly smaller cube without an anchor at the same position
    27.                 GameObject placedObject2 = GameObject.CreatePrimitive(PrimitiveType.Cube);
    28.                 placedObject2.transform.localScale = Scale * 0.99f;
    29.                 var cameraTransform =;
    30.                 placedObject2.transform.position = _raycastHits[0].pose.position;
    31.                 _placedObjects.Add(placedObject2);
    32.             }
    33.         }
    34.     }
    36.     UpdateTrackingStates();
    37. }
    We are aware of the fact that current tracking technology in these devices is limited and intended to be used to place chairs, tables, pictures on the wall etc. We are pushing this to the limit. However, we saw that it works for our use case sometimes, but not all the time. We would like to have an indicator of the tracking quality to detect the drifting. The trackingState of trackable objects does not help here, unfortunately.

    Are we missing something?