Search Unity

iOS ARFoundation issue with ARPlane TrackingState

Discussion in 'Handheld AR' started by WolfBeardedLion, Jul 9, 2018.

  1. WolfBeardedLion

    WolfBeardedLion

    Joined:
    Apr 5, 2013
    Posts:
    27
    This code works as expected on Android, but when running this on an iOS device the function never returns true despite AR Planes visible on the screen.

    Any ideas why this would not work on iOS?

    Code (CSharp):
    1.     private void Update()
    2.     {
    3.         AreAnyPlanesBeingDetected();
    4.     }
    5.  
    6.     private bool AreAnyPlanesBeingDetected()
    7.     {
    8.         augmentedReality_ARPlaneManager.GetAllPlanes(augmentedReality_ARPlaneList);
    9.         for (int i = 0; i < augmentedReality_ARPlaneList.Count; i++)
    10.         {
    11.             if (augmentedReality_ARPlaneList[i].trackingState == TrackingState.Tracking)
    12.             {
    13.                 Debug.Log("A plane has been detected.");
    14.                 return true;
    15.             }
    16.         }
    17.  
    18.         Debug.Log("No planes are being detected.");
    19.         return false;
    20.     }
     
    Last edited: Jul 9, 2018
  2. WolfBeardedLion

    WolfBeardedLion

    Joined:
    Apr 5, 2013
    Posts:
    27
    Also, I am using the function below by subscribing to the ARSubsystemManager.planeAdded event. It works well in Android, but it doesn't work in an iOS build.

    Code (CSharp):
    1.     private void AugmentedReality_OnPlaneAdded(PlaneAddedEventArgs eventArgs)
    2.     {
    3.         ARPlane plane = augmentedReality_ARPlaneManager.TryGetPlane(eventArgs.Plane.Id);
    4.         ...
    5.     }
    Any help with these two issues would be very much appreciated; thank you.
     
  3. WolfBeardedLion

    WolfBeardedLion

    Joined:
    Apr 5, 2013
    Posts:
    27
    To fix this issue, whenever an OnPlaneAdded event is fired I start a coroutine that leads off with a "yield return new WaitForEndOfFrame();" before executing the TryGetPlane function call from the ARPlaneManager.

    It seems the ARPlaneManager needs a frame to register the new plane before it is accessible via the TryGetPlane call.
     
  4. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    547
    The behavior your're seeing is because the ARPlaneManager subscribes to the ARSubsystemManager to get its plane updates, and your method is being called first, before the ARPlaneManager has updated the planes it manages. The intention is to subscribe to the ARPlaneManager's planeAdded/Updated/Removed events, rather than the ARSubsystemManager's.
     
  5. WolfBeardedLion

    WolfBeardedLion

    Joined:
    Apr 5, 2013
    Posts:
    27
    Thank you for the reply, Tim. It's very much appreciated.

    First off, I am very impressed with AR Foundation; I think it will be one of the more popular tools Unity offers in the next year or so. I do a lot of AR work inside of Unity and I am really looking forward to seeing how this package evolves.

    I was able to figure out the issue you commented on, but the tip on using the planeAdded event from ARPlaneManager is very useful. Thank you; not sure how I overlooked that.

    However, I am still unsure as to why the first piece of code I posted here does not work on iOS but it does on Android. It seems the trackingState of an AR Plane is never set to "Tracking", which causes my function, AreAnyPlanesBeingDetected, to always returns false even when I can see on the screen I have detected plane.

    I had to replace the code with a similar function that only checks the child count of the ARPlaneManager, and then it returns true if it is more than 0. This works fine for now, but I'd prefer to actually check if the ARPlane is being tracked like I can on Android.

    Any thoughts on why this happening?
     
  6. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    547
    Thanks for the positive feedback! We're planning to add lots of features before the end of the year, including support for all the features of ARCore 1.3 and ARKit 2.0.

    Looks like a bug. I'll make sure it gets fixed for the next preview release.
     
  7. WolfBeardedLion

    WolfBeardedLion

    Joined:
    Apr 5, 2013
    Posts:
    27
    That's really great to hear! I look forward to utilizing everything you can give me. I'll continue to report whatever strange behavior I come across; hopefully I can be of some help to you all.

    Thank you for your help.
     
  8. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    547
    I looked more into this, and it's because there is no equivalent in ARKit -- only ARCore has per-plane tracking status. The possible values are "Unknown", "Tracking", and "Unavailable". ARCore will probably either be "Tracking" or "Unavailable" (meaning its not tracking). On iOS, ARPlane.trackingState should be returning "Unknown" (since individual planes don't have tracking states).

    You could try
    Code (CSharp):
    1. if (augmentedReality_ARPlaneList[i].trackingState != TrackingState.Unavailable)
    Would it make more sense if, on ARKit, the planes' trackingState simply returned the same value as the session tracking state?
     
  9. WolfBeardedLion

    WolfBeardedLion

    Joined:
    Apr 5, 2013
    Posts:
    27
    That's interesting; I had assumed ARKit was capable of that.

    If ARKit is always going to return the same result for each plane, then it would most likely be best to keep things very transparent for developers. You may want to add a note to where you have TrackingState documented that speaks about how ARKit does not support this feature, and that it will always return x result. This makes things very clear on why, and what, is going on.

    I wouldn't try to force something in there that isn't consistent with the ARCore feature. Apple may update ARKit to support this kind of detailed tracking in the future, and by leaving it like it is now will allow you to add that in without breaking code and forcing developers to relearn this area of the package.

    Maybe offer a potential workaround within the notes, so developers do not feel blocked and/or discouraged. I am sure there are a few different approaches we could take, but right now what I am doing is checking if the List of ARPlanes from ARPlaneManager.GetAllPlanes has a count greater than 0, and then I know I have at least one plane to work with. I was hoping to find a way to remove the ARPlane from the ARPlaneManager's list when the ARPlane becomes no longer active on the screen. I haven't been able to do this as of yet, but it would allow my UI to change back to the "Scanning for Surfaces" state when there are no longer any planes visible on the screen, like it's currently doing on Android now.

    I gave your code suggestion a try, and I was able to recreate the behavior I expect from my current workaround. However when I lose tracking of the planes that were on the screen, iOS still doesn't know that we have lost tracking of each one so my UI stays in a "Plane Detected" state.
     
  10. jprice67

    jprice67

    Joined:
    Apr 26, 2018
    Posts:
    1
    On iOS, if the session tracking state is currently "Tracking", and I cover the camera with my hand, it stays 'Tracking'.
    On Android, if the session tracking state is currently "Tracking", and I cover the camera with my hand, it changes to 'Unavailable'

    Currently, on iOS, there doesn't really seem to be a way to recover from losing tracking in a unified way.
     
  11. Jelmersb

    Jelmersb

    Joined:
    Jul 12, 2016
    Posts:
    66
    ARKit has the same, but how do you access it with ARFoundation?
    https://developer.apple.com/documentation/arkit/artrackingstate?language=objc
     
  12. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    547
    Jelmersb likes this.
  13. Jelmersb

    Jelmersb

    Joined:
    Jul 12, 2016
    Posts:
    66
    Are there any plans to include TrackingStateReason in ARFoundation? (excess motion / too dark etc)
     
  14. Aidan-Wolf

    Aidan-Wolf

    Joined:
    Jan 6, 2014
    Posts:
    50
    Bump "Are there any plans to include TrackingStateReason in ARFoundation? (excess motion / too dark etc)"

    @tdmowrer