Plane detection issues with ARFoundation 2.1.0 (mostly on iPhone)

Hi

We are working in the development of AR App for iOS and Android using AR Foundation 2.1.0. I the past we were doing more or less the same scene just using ARKit, so we are able to compare between the two technologies.

We are having some problems in the plane detection mostly in iOS, we changed the frame rate to 60 fps and it improved but is not working so good. As I said, if we compare this with just using ARKit the differences are very important.

In Android it works more or less good, is not the same than ARCore but the result is acceptable.

The versions that we are using are:
- Unity 2019.1.1f1
- AR Foundation 2.1.0
- ARCore XR Plugin 2.1.0
- ARKit XR Plugin 2.1.0
- XR Legacy Input Helpers 2.0.2

The first steps are so simple, we only do the plane detection to place a character that we have modeled.

Could someone give us any advice or any trick to improve it?

Thanks in advance

 

What exactly is the problem? For example, frame drops, poor tracking, incorrectly detecting planes?

Also, what device(s) and iOS version(s) are you testing on?

 

Thanks for your answer, the main problem is that it takes a lot of time to detect the planes. Sometimes there are surfaces that it doesn't detect while with ARKit works perfectly

We are testing on iPhone X, iPhone 7 plus and iPad and we are using the latest versions of iOS. With the iPhone X we are using iOS 12.3.1 and before this update it worked in the same way.

 

That sounds fine. Can you reproduce this behavior in the SimpleAR scene from the samples repo? If not, would you be willing to share your project?

 

Yes, I was trying the samples of plane detection and I have found the same problems. If you compare with ARKit the difference is very big.

 

That's difficult for me to reproduce. Here are two videos of the same area, one using ARFoundaiton, the other the UntiyARKit Plugin on BitBucket. Both produce the same result as far as I can tell.

If the difference is more obvious in your case, could you post similar videos?

 

Hi!

I have problems detecting any Planes with the same settings, using ARCore XR Plugin 2.1.0 and android device. I only get FeaturePoint hits when not using any filters, but no Planes. Here is my code:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;
//using UnityEngine.Experimental.XR;
using System;

public class ARTapToPlaceObject : MonoBehaviour
{
public GameObject placementIndicator;
public GameObject objectToPlace;
private ARRaycastManager arRaycastManager;
private Pose placementPose;
public GameObject info;
private Vector2 startPos;
private Vector2 direction;
private bool placePoseIsValid = false;
void Start()
{
arRaycastManager = FindObjectOfType<ARRaycastManager>();
}

void Update()
{
UpdatePlacementPose();
UpdatePlacementIndicator();

if(placePoseIsValid && placementIndicator.activeSelf && Input.touchCount > 0 ) {

Touch touch = Input.GetTouch(0);

switch (touch.phase)
{
//When a touch has first been detected, change the message and record the starting position
case TouchPhase.Began:
// Record initial touch position.
startPos = touch.position;
break;

//Determine if the touch is a moving touch
case TouchPhase.Moved:
// Determine direction by comparing the current touch position with the initial one
direction = touch.position - startPos;
placementIndicator.transform.Rotate(0,direction.x/5,0,Space.Self);
break;

case TouchPhase.Ended:
// Report that the touch has ended when it ends
direction = touch.position - startPos;
if(direction == Vector2.zero) {
PlaceObject();
}
break;
}
}
}

private void PlaceObject() {
Instantiate(objectToPlace, placementIndicator.transform.position, placementIndicator.transform.rotation);
}
private void UpdatePlacementIndicator()
{
if(placePoseIsValid)
{
placementIndicator.SetActive(true);
placementIndicator.transform.SetPositionAndRotation(placementPose.position, placementIndicator.transform.rotation);
}
else
{
info.GetComponent<UnityEngine.UI.Text>().text = "No hit";
placementIndicator.SetActive(false);
}
}
private void UpdatePlacementPose()
{
var screenCenter = Camera.current.ViewportToScreenPoint(new Vector3(0.5f,0.5f));
var hitsRaycastManager = new List<ARRaycastHit>();
arRaycastManager.Raycast(screenCenter, hitsRaycastManager, TrackableType.All);
placePoseIsValid = hitsRaycastManager.Count > 0;

if(placePoseIsValid)
{
info.GetComponent<UnityEngine.UI.Text>().text = System.Enum.GetName(typeof(TrackableType),hitsRaycastManager[0].hitType);
placementPose = hitsRaycastManager[0].pose;
}
}
}

 

I am also having difficulty detecting planes with AR Core. I intermittently get raycast hits from the Raycast Manager if I use TrackableType.All, but zero hits with .Planes. Maybe I'm doing something wrong?

2019.2.4f1
Pixel 2
AR Foundation 2.0.2
ARCore XR Plugin 2.0.2

https://imgur.com/a/Roci25f

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.XR.ARFoundation;
5. using UnityEngine.XR;
6. using UnityEngine.XR.ARSubsystems;
7. using System;
8.  
9. public class ARTapToPlaceObject : MonoBehaviour
10. {
11.     public GameObject objectToPlace;
12.     public GameObject placementIndicator;
13.     private Camera myCamera;
14.     private ARSessionOrigin arOrigin;
15.     private ARRaycastManager raycastManager;
16.     private Pose placementPose;
17.     private bool placementPoseIsValid = false;
18.  
19.     void Start()
20.     {
21.         arOrigin = FindObjectOfType<ARSessionOrigin>();
22.         raycastManager = FindObjectOfType<ARRaycastManager>();
23.     }
24.  
25.     void Update()
26.     {
27.         UpdatePlacementPose();
28.         UpdatePlacementIndicator();
29.  
30.         if (placementPoseIsValid && Input.touchCount > 0 && Input.GetTouch(0).phase == TouchPhase.Began)
31.         {
32.             PlaceObject();
33.         }
34.     }
35.  
36.     private void PlaceObject()
37.     {
38.         Instantiate(objectToPlace, placementPose.position, placementPose.rotation);
39.     }
40.  
41.     private void UpdatePlacementIndicator()
42.     {
43.         if (placementPoseIsValid)
44.         {
45.             placementIndicator.SetActive(true);
46.             placementIndicator.transform.SetPositionAndRotation(placementPose.position, placementPose.rotation);
47.         }
48.         else
49.         {
50.             placementIndicator.SetActive(false);
51.         }
52.     }
53.  
54.     private void UpdatePlacementPose()
55.     {
56.         var screenCenter = Camera.current.ViewportToScreenPoint(new Vector3(0.5f, 0.5f));
57.         var hits = new List<ARRaycastHit>();
58.         raycastManager.Raycast(screenCenter, hits, TrackableType.Planes);
59.         Debug.Log("hits: " + hits.Count);
60.         placementPoseIsValid = hits.Count > 0;
61.  
62.         if (placementPoseIsValid)
63.         {
64.             placementPose = hits[0].pose;
65.             var cameraForward = Camera.main.transform.forward;
66.             var cameraBearing = new Vector3(cameraForward.x, 0, cameraForward.z).normalized;
67.             placementPose.rotation = Quaternion.LookRotation(cameraBearing);
68.         }
69.     }
70.  
71. }
72.  

 

No, the code is from that video on youtube, you're doing everything right, as far as I can tell.

Just a guess - try setting the camera through inspector, not by using Camera.current

 

Hi, I'm having similar issues with ARFoundation 3.0.0 preview3 on iPhone X iOS 13.12, built on Unity 2019.2.4f1

On android, the plane detection works fine. We're using the samples from this video

. My programmer made it work in 3.0.0.

On iPhone, the plane detection jags about for 2-3 seconds until it finds the surface, BUT, if I scan a table for example which is a "higher altitude" so to say than the floor, my marker won't snap back to the floor when I point the camera back towards the floor. It remains in the same height level, but if I lower my camera in a world position lower than the table, it will find the floor again.
It's almost like the plane detection virtually builds infinitely wide planes when finding another level of height, so I can't move my marker back to the floor.

 

yes, it works like that for me, too, always has I think

 

but it's not supposed to work like this. It's clearly shown in the video.

 

I tried setting it to only detect tables using the new classification feature, but now it doesn't work at all((

 

That sucks. Any dev who can respond to this problem? :/ it’s quite urgent for me, as it’s used in a job for a client

 

TrackableType.Planes includes ALL varieties of plane, including TrackableType.PlaneWithinInfinity. It sounds like you probably want to use something like TrackableType.PlaneWithinPolygon, which only uses the "2D convex shape associated with a plane's boundary points."

 

How would I do this? Do I change something in the ARPlaneManager script or is there a setting?
It's not the Detection Mode right? Which has the Everything, Horizontal and Vertical alternatives.

 

it's when you're doing an ARRaycast form ARSessionOrigin

 

Hi we solved it

My programmer had made his own script apparently so we did it from there.
Thank you!

 

Also having very slow plane detection since switching to AR Foundation v3.0.1. Was ways better in v1, what i used before. Can just talk about Android, but it's extremly slower now.

@illusiveworks How did you solve it? Did you also had to adapt the ARRaycastManager or anything else? Could you maybe share the script or at least the imporant parts of it? Would be very helpful.

 

Hi,

i could extremly improve the tracking speed now by simply disabling the SetTargetFramerate component on the ARSession GameObject (tested on Android with a Pixel 2).

Maybe it helps for somebody else, too.

 

Hi!

Could you please share a code?

Thanks.

 

Thanks. Changing TrackableType.Planes to TrackableType.PlaneWithinPolygon worked