Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

AR Scale in ARCore Preview 2 w/o the experimental AR interface [Tutorial]

Discussion in 'AR/VR (XR) Discussion' started by kbabilinski, Jan 10, 2018.

  1. kbabilinski

    kbabilinski

    Joined:
    Jul 12, 2012
    Posts:
    14

    Blender File by : fokuspace
    Introduction
    Based on his Unite Austin talk, Rick Johnson wrote a blog post describing the concept of scale in AR.

    Link: https://blogs.unity3d.com/2017/11/16/dealing-with-scale-in-ar/

    However, you might ask, how do I solve the scaling issue without using the AR interface. Fear not, I tinkered around with the new SDK and learned how to achieve a similar result.


    Tutorial

    Importing the SDK
    We’re going to start with a new project and a fresh import of the ARCore preview 2 SDK. To learn how to setup the ARCore sdk in your project follow the official documentation here
    link: https://developers.google.com/ar/develop/unity/getting-started
    GitHub Releases: https://github.com/google-ar/arcore-android-sdk/releases

    *Important: Preview 2 only runs on devices that have the ARCore Preview 2 apk installed, and requires Unity 2017.3 or higher


    Handling scaling
    By default the “AR Core Device” prefab is setup to support scaling. The component responsible for this is the “Tracked Pose Driver.cs” The component is attached to the child, “First Person Camera”, notice how “Use Relative Transform” is checked.





    While the description in the documentation might not be obvious, the property handles the scale transformations to the camera, relative to the parent.

    The documentation for the tracked pose driver: https://docs.unity3d.com/ScriptReference/SpatialTracking.TrackedPoseDriver.html


    Scaling the AR Components
    Because the scale is handled relatively, the structure of the hierarchy is important. To efficiently scale the AR camera, you must scale all the AR components in the Hello AR Scene. Scaling all the components insures that all the information captured by the AR SDK is being presented accurately, around the same origin.

    Example of hierarchy that is ready to scale.



    *Scaling only the camera component will result in the point cloud appearing in a different scale than the camera rig.

    *Not all of these components are required, but I like to group my SDK components under a single parent.


    Apply the Scale
    Now that our hierarchy is setup correctly, make sure that all the child objects are located at the origin (not including the light or example controller) 0,0,0. Relative to the parent object. Then you can scale the parent object. I scaled mine to be 20x the regular scale.

    *Note scaling the parent object will not scale the “Session.Raycast” used in the Example Controller

    Positioning the Camera Rig Relative the environment.
    Rick Johnson also described positioning the camera rig so that it appears that the terrain is placed on top of an anchor point, when in fact the camera rig is being offset. To achieve this we have to do two things,

    1. Scale the Sessions.Raycast by the parent transform scale so that the hit point is accurate to our camera scale.

    2. Offset the parent transform by the inverse of the rayhit position.

    I’ve modified the Example Controller code, I won’t go into much detail as I’ve tried to name the variables properly so that the code is easy to follow. Feel free to leave comments.

    Code (CSharp):
    1.    // Raycast against the location the player touched to search for planes.
    2.         TrackableHit hit;
    3.         TrackableHitFlags raycastFilter = TrackableHitFlags.PlaneWithinBounds | TrackableHitFlags.PlaneWithinPolygon;
    4.      
    5.         if (Session.Raycast(touch.position.x, touch.position.y, raycastFilter, out hit))
    6.         {
    7.             //Store the parent transform
    8.             Transform rootTransform = m_ARCamera.transform.parent;
    9.             //Store the hitPosition for scaling.
    10.             Vector3 hitPosition = hit.Pose.position;
    11.             if (rootTransform != null)
    12.             {
    13.                 //Scale the hitPosition by the scale of the root transform
    14.                 hitPosition.Scale(rootTransform.transform.localScale);
    15.                 //Position the root transform by negative the hitPosition
    16.                 rootTransform.localPosition = hitPosition * -1;
    17.             }
    18.               //I added a conditional property so that I can load a scene
    19.             if (objectPlace == false)
    20.             {
    21.                 //I load a scene , for this example the origin of the scene must be at 0,0,0
    22.                 SceneManager.LoadSceneAsync(scene, LoadSceneMode.Additive);
    23.                 objectPlace = true;
    24.             }
    25.  
    26.         }
     
    Last edited: Jan 13, 2018
  2. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    792
    This is great work, @Bravo101, but I'd like to correct a few misconceptions in your post.

    The dealing with scale blog was based on @timmunity and my talk. If you look in that link, our talk also dealt with cross-platform AR across ARKit and ARCore, and provided some experimental code: https://github.com/Unity-Technologies/experimental-ARInterface

    That code has recently been updated to work with ARCore Preview2 , and the scaling works just fine.
     
  3. kbabilinski

    kbabilinski

    Joined:
    Jul 12, 2012
    Posts:
    14
    This is embarrassing :eek::oops:I knew that the AR Interface existed and I thought I cloned it onto my computer. When I was trying to incorporate the ar interface in my project I swore I was looking at the correct thing. But, I just looked at my copy and I found out that I was using the Mapbox project.

    I'll update the post:D

     
    Last edited: Jan 13, 2018
  4. kbabilinski

    kbabilinski

    Joined:
    Jul 12, 2012
    Posts:
    14
    Updated the title to be more clear, incase anyone wants to scale the scene without using the interface.

     
  5. nerkderk

    nerkderk

    Joined:
    Oct 24, 2017
    Posts:
    14
    Thanks for this, @Bravo101 ! Exactly what I am looking for while I try to figure out how to use the ARInterface. I am not sure I am understanding the changes to the code entirely, here is how I changed the Session.Raycast code that places my object.

    Code (CSharp):
    1.  
    2.  
    3. if (Session.Raycast(touch.position.x, touch.position.y, raycastFilter, out hit))
    4. {
    5.     //Store the parent transform
    6.     Transform rootTransform = FirstPersonCamera.transform.parent;
    7.     // Store the hitPosition for scaling.
    8.     Vector3 hitPosition = hit.Pose.position;
    9.     if (rootTransform != null)
    10.     {
    11.     //Scale the hitPosition by the scale of the root transform
    12.     hitPosition.Scale(rootTransform.transform.localScale);
    13.     //Position the root transform by negative the hitPosition
    14.     rootTransform.localPosition = hitPosition * -1;
    15.     }
    16.  
    17.     var placedObject = Instantiate(objectToPlace, hitPosition, hit.Pose.rotation);
    18.                     // Create an anchor to allow ARCore to track the hitpoint as understanding of the physical world evolves.
    19.     var anchor = hit.Trackable.CreateAnchor(hit.Pose);
    20.  
    21.     // Make object a child of the anchor.
    22.     placedObject.transform.parent = anchor.transform;
    23. }
    24.  
    When I build & run, the object gets placed but not exactly where I touched. Additionally, the script on my placed object that handles touches on the object using Physics.Raycast isn't working properly anymore...when I touch the object nothing is moving as it is supposed to. I suppose maybe those Physics.Raycast/touches have to be scaled as well? I'm a bit confused here.
     
  6. kbabilinski

    kbabilinski

    Joined:
    Jul 12, 2012
    Posts:
    14
    @nerkderk I'm glad my post helped :) I have transitioned to the Unity AR Interface after I learned that I was using the wrong repo, so my understanding of the SDK is still limited. But I can try to help.

    I'm not sure what's the exact issue you're experiencing but I can share some thoughts. One idea is that you're placing your object in the incorrect position. We use the hitPosition as an offset for the parent transform. That's why we do

    Code (CSharp):
    1. rootTransform.localPosition = hitposition *-1;
    Here is a bad diagram of what's happening :


    *We're moving the camera back so it appears that the object was scaled.

    So I would suggest changing

    Code (CSharp):
    1. var placedObject = Instantiate(objectToPlace, hitPosition, hit.Pose.rotation);
    to the default hit position .

    Code (CSharp):
    1. [code=CSharp]var placedObject = Instantiate(hit.Pose.position, hit.Pose.rotation);
    [/code]


    Let me know if that works. Stay sharp ;) [Bad code joke]
     
  7. nerkderk

    nerkderk

    Joined:
    Oct 24, 2017
    Posts:
    14
    @Bravo101 Ah yea, that seems to have helped greatly! Thank you!

    I guess I should probably switch over to using the ARInterface, but I am having a hard time getting started with it. For example, I haven't even been able to find the proper code for instantiating an object at a touch point, to see how it differs from ARCore. It seems like they don't have an example of such a project...or documentation on how to use it. Although I am most likely just missing something. Do you have any tips to get me started / point me in the right direction of how to get acquainted with the ARInterface?
     
  8. Section-One

    Section-One

    Joined:
    Sep 8, 2014
    Posts:
    80
    @jimmya

    Does the experimental library's scale option works correctly after it was updated to ARcore 1?

    I opened an issue about scale on github but it hasn't received a response so I don't know if its broken or if I'm doing something wrong..