iOS ARFoundation issue with ARPlane TrackingState

This code works as expected on Android, but when running this on an iOS device the function never returns true despite AR Planes visible on the screen.

Any ideas why this would not work on iOS?

Code (CSharp):

1.     private void Update()
2.     {
3.         AreAnyPlanesBeingDetected();
4.     }
5.  
6.     private bool AreAnyPlanesBeingDetected()
7.     {
8.         augmentedReality_ARPlaneManager.GetAllPlanes(augmentedReality_ARPlaneList);
9.         for (int i = 0; i < augmentedReality_ARPlaneList.Count; i++)
10.         {
11.             if (augmentedReality_ARPlaneList[i].trackingState == TrackingState.Tracking)
12.             {
13.                 Debug.Log("A plane has been detected.");
14.                 return true;
15.             }
16.         }
17.  
18.         Debug.Log("No planes are being detected.");
19.         return false;
20.     }

 

Also, I am using the function below by subscribing to the ARSubsystemManager.planeAdded event. It works well in Android, but it doesn't work in an iOS build.

Code (CSharp):

1.     private void AugmentedReality_OnPlaneAdded(PlaneAddedEventArgs eventArgs)
2.     {
3.         ARPlane plane = augmentedReality_ARPlaneManager.TryGetPlane(eventArgs.Plane.Id);
4.         ...
5.     }

Any help with these two issues would be very much appreciated; thank you.

 

To fix this issue, whenever an OnPlaneAdded event is fired I start a coroutine that leads off with a "yield return new WaitForEndOfFrame();" before executing the TryGetPlane function call from the ARPlaneManager.

It seems the ARPlaneManager needs a frame to register the new plane before it is accessible via the TryGetPlane call.

 

The behavior your're seeing is because the ARPlaneManager subscribes to the ARSubsystemManager to get its plane updates, and your method is being called first, before the ARPlaneManager has updated the planes it manages. The intention is to subscribe to the ARPlaneManager's planeAdded/Updated/Removed events, rather than the ARSubsystemManager's.

 

Thank you for the reply, Tim. It's very much appreciated.

First off, I am very impressed with AR Foundation; I think it will be one of the more popular tools Unity offers in the next year or so. I do a lot of AR work inside of Unity and I am really looking forward to seeing how this package evolves.

I was able to figure out the issue you commented on, but the tip on using the planeAdded event from ARPlaneManager is very useful. Thank you; not sure how I overlooked that.

However, I am still unsure as to why the first piece of code I posted here does not work on iOS but it does on Android. It seems the trackingState of an AR Plane is never set to "Tracking", which causes my function, AreAnyPlanesBeingDetected, to always returns false even when I can see on the screen I have detected plane.

I had to replace the code with a similar function that only checks the child count of the ARPlaneManager, and then it returns true if it is more than 0. This works fine for now, but I'd prefer to actually check if the ARPlane is being tracked like I can on Android.

Any thoughts on why this happening?

 

Thanks for the positive feedback! We're planning to add lots of features before the end of the year, including support for all the features of ARCore 1.3 and ARKit 2.0.

Looks like a bug. I'll make sure it gets fixed for the next preview release.

 

That's really great to hear! I look forward to utilizing everything you can give me. I'll continue to report whatever strange behavior I come across; hopefully I can be of some help to you all.

Thank you for your help.

 

I looked more into this, and it's because there is no equivalent in ARKit -- only ARCore has per-plane tracking status. The possible values are "Unknown", "Tracking", and "Unavailable". ARCore will probably either be "Tracking" or "Unavailable" (meaning its not tracking). On iOS, ARPlane.trackingState should be returning "Unknown" (since individual planes don't have tracking states).

You could try

Code (CSharp):

1. if (augmentedReality_ARPlaneList[i].trackingState != TrackingState.Unavailable)

Would it make more sense if, on ARKit, the planes' trackingState simply returned the same value as the session tracking state?

 

That's interesting; I had assumed ARKit was capable of that.

If ARKit is always going to return the same result for each plane, then it would most likely be best to keep things very transparent for developers. You may want to add a note to where you have TrackingState documented that speaks about how ARKit does not support this feature, and that it will always return x result. This makes things very clear on why, and what, is going on.

I wouldn't try to force something in there that isn't consistent with the ARCore feature. Apple may update ARKit to support this kind of detailed tracking in the future, and by leaving it like it is now will allow you to add that in without breaking code and forcing developers to relearn this area of the package.

Maybe offer a potential workaround within the notes, so developers do not feel blocked and/or discouraged. I am sure there are a few different approaches we could take, but right now what I am doing is checking if the List of ARPlanes from ARPlaneManager.GetAllPlanes has a count greater than 0, and then I know I have at least one plane to work with. I was hoping to find a way to remove the ARPlane from the ARPlaneManager's list when the ARPlane becomes no longer active on the screen. I haven't been able to do this as of yet, but it would allow my UI to change back to the "Scanning for Surfaces" state when there are no longer any planes visible on the screen, like it's currently doing on Android now.

I gave your code suggestion a try, and I was able to recreate the behavior I expect from my current workaround. However when I lose tracking of the planes that were on the screen, iOS still doesn't know that we have lost tracking of each one so my UI stays in a "Plane Detected" state.

 

On iOS, if the session tracking state is currently "Tracking", and I cover the camera with my hand, it stays 'Tracking'.
On Android, if the session tracking state is currently "Tracking", and I cover the camera with my hand, it changes to 'Unavailable'

Currently, on iOS, there doesn't really seem to be a way to recover from losing tracking in a unified way.

 

ARKit has the same, but how do you access it with ARFoundation?
https://developer.apple.com/documentation/arkit/artrackingstate?language=objc

 

This is a bug in the ARKit package (we report ARTrackingStateLimited as TrackingState.Tracking). This will change in the next release of the ARKit package to TrackingState.Unavailable.

 

Are there any plans to include TrackingStateReason in ARFoundation? (excess motion / too dark etc)

 

Bump "Are there any plans to include TrackingStateReason in ARFoundation? (excess motion / too dark etc)"

@tdmowrer