Search Unity

  1. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

ARKit Remote Touch Inpu

Discussion in 'AR' started by nesta219, Oct 15, 2017.

  1. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    Hi there folks,

    I've been doing a lot of searching on how to get the ARKit remote connection (which is by and large great) to register touch inputs in the editor. The following code works perfectly when running on the actual ipad I'm using, using easytouch as my touch/gesture library:

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using UnityEngine.XR.iOS;
    5. using HedgehogTeam.EasyTouch;
    6.  
    7. namespace Nesta.DigitalBlack
    8. {
    9.     public class InputHelper : MonoBehaviour
    10.     {
    11.  
    12.         public static ARHitTestResult _arTouchHitTestResult;
    13.         public static ARHitTestResult _arCenterScreenHitTestResult;
    14.  
    15.         public static Vector3 arHitTestPosition;
    16.  
    17.         public static string AR_PLANE_LAYER = "ARKitPlane";
    18.  
    19.         public static bool TryARTouchHit() {
    20.        
    21.             if (EasyTouch.current != null) {
    22.                 if(EasyTouch.current.type == EasyTouch.EvtType.On_TouchDown) {
    23.                     var viewPosition = Camera.main.ScreenToViewportPoint (EasyTouch.current.position);
    24.  
    25.                     ARPoint point = new ARPoint {
    26.                         x = viewPosition.x,
    27.                         y = viewPosition.y
    28.                     };
    29.  
    30.                     // prioritize reults types
    31.                     ARHitTestResultType[] resultTypes = {
    32.                         ARHitTestResultType.ARHitTestResultTypeExistingPlaneUsingExtent,
    33.                         // if you want to use infinite planes use this:
    34.                         ARHitTestResultType.ARHitTestResultTypeExistingPlane,
    35.                         ARHitTestResultType.ARHitTestResultTypeHorizontalPlane,
    36.                         ARHitTestResultType.ARHitTestResultTypeFeaturePoint
    37.                     };
    38.  
    39.                     foreach (ARHitTestResultType resultType in resultTypes) {
    40.                         List<ARHitTestResult> hitResults = UnityARSessionNativeInterface.GetARSessionNativeInterface ().HitTest (point, resultType);
    41.                         if (hitResults.Count > 0) {
    42.                             foreach (ARHitTestResult hitResult in hitResults) {
    43.                                 _arTouchHitTestResult = hitResult;
    44.                                 arHitTestPosition = UnityARMatrixOps.GetPosition (InputHelper._arTouchHitTestResult.worldTransform);
    45.                                 return true;
    46.                             }
    47.                         }
    48.                     }
    49.  
    50.                 }
    51.             }
    52.  
    53.             return false;
    54.         }
    55.     }
    56. }
    However, when I use this in the editor with ARKit remote, it won't register any hits. The bizarre thing is that the output from easy touch is registering the same values in the editor as on the device, so I know the touch input data itself is good. Is there some kind of issue that is preventing UnityARSessionNativeInterface.GetARSessionNativeInterface ().HitTest(point, resultType) from working in the editor?
     
  2. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    It's worth noting that I tried the techniques in "EditorHitTest.cs" to no avail.
     
  3. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    The EditorHitTest example does work in its own scene but the same technique applied to my script is no bueno... very frustrating.
     
  4. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    HitTest only works on device - you should simulate hit test with what EditorHitTest does - it casts a ray and collides against the generated planes. Are you using the GeneratePlanes GO in the scene?
     
  5. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    That's just the thing! I tried adapting the EditorHitTest code for my game and no dice... maybe I need to actually draw the debug rays to see what's going on.
     
  6. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Check that the collider on your debugPlane prefab has a Layer set that is the same as the collisionLayerMask on EditorHitTest or change one or both to make them the same
     
  7. ifeltdave

    ifeltdave

    Joined:
    Aug 28, 2012
    Posts:
    16
    This thread helped me to get it to work.. it isn't immediately clear that the collisionLayerMask is required, but this did solve it for me!

    One additional question I had that remains unsolved... is it correct that the HitTest can't come from my mobile device's screen, and only by interacting with the editor? I can't seem to get taps on the screen to register, which makes sense since the camera feed doesn't display there either. Maybe there is a way around this?
     
  8. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    Ok, so I've come back with a screen shot and a snippet to hopefully figure out what's going on here. This is the code I'm running in my update loop:

    Code (CSharp):
    1. //in-editor test
    2.             if (Input.GetMouseButtonDown (0)) {
    3.                 Ray ray = Camera.main.ScreenPointToRay (Input.mousePosition);
    4.                 RaycastHit hit;
    5.  
    6.                 if (Physics.Raycast (ray, out hit, 200f,  LayerMask.NameToLayer("ARKitPlane"))) {
    7.                     Debug.Log("Hit ARPlane");
    8.                     Debug.DrawRay(ray.origin, hit.point, Color.green, 2f);
    9.                     return true;
    10.                 } else {
    11.                     Debug.Log ("Hit nothing");
    12.                     Debug.DrawRay(ray.origin, ray.GetPoint(100), Color.red, 2f);
    13.                 }
    14.             }
    And yet when I try and hit my planes, I get a red ray even though it's passing right through it, per the attached image. The layer masks match as well so I know that's not it. Any thoughts?
     

    Attached Files:

    replysenad likes this.
  9. ifeltdave

    ifeltdave

    Joined:
    Aug 28, 2012
    Posts:
    16
    Hey nesta, have you tried adding a rigidbody and collider to the object you're testing ray hits against? I can't tell if the object in your screenshot is the one you're trying to test interaction with but I think those components are needed for a raycast to return a hit.
     
  10. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    There is a collider on the child object but not a rigid body. I didn't thinking a rigid body was necessary for Ray casting?
     
  11. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    I just added a rigidbody and still no dice. I uploaded a pic of the inspector view of the child object with collider, mesh, and rigidbodies on it.

    This is the entire C# file... I am calling my method statically instead of having it be an instance of a MonoBehaviour... think that could be it?
     

    Attached Files:

  12. nesta219

    nesta219

    Joined:
    Nov 3, 2015
    Posts:
    14
    Finally I have found some success! So ultimately, I had to create a public LayerMask variable and set its value in the editor... for some reason, using LayerMask.NameToLayer("ARKitPlane") didn't work for the layer parameter in the raycast call. Any ideas what's up with that?
     
  13. joshpetersonhush

    joshpetersonhush

    Joined:
    Nov 9, 2017
    Posts:
    12
    I'm new to Unity and I can't get this to work. Taking UnityARKitScene as an example, what I did was:

    1. Opened up the debugPlanePrefab
    2. Made a new layer 'foo' and set the prefab's layer to it
    3. Click Apply
    4. Applied EditorHitTest to HitCube
    5. Set CollisionLayerMask on that script on HitCube to 'foo' in the inspector
    Am I missing anything?

    This is super key for AR workflow. Would be amazing to get some more documentation here, a youtube video just setting this up would be awesome. Seems like there are two cases maybe - when you want to make hits on arkit planes, and when you want to make hits on game objects. Do each of those need to be handled differently?
     
  14. joshpetersonhush

    joshpetersonhush

    Joined:
    Nov 9, 2017
    Posts:
    12
    Okay I guess in hindsight this is obvious, but I didn't really understand what the UnityARKitScene did, so: you have to wait until GeneratePlanes draws a plane onscreen before you can click on it to simulate a hit.

    Then I started getting hits, but I would get this error on hit:

    UnassignedReferenceException: The variable m_HitTransform of EditorHitTest has not been assigned.
    You probably need to assign the m_HitTransform variable of the EditorHitTest script in the inspector.

    I set Hit Transform in the inspector to ARCameraManager and now I'm getting values logged in the console.
     
    Last edited: Nov 14, 2017
unityunity