Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Multi-platform Handheld AR

Discussion in 'AR' started by jimmya, Jun 15, 2018.

  1. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    During GDC 2018, we talked about providing a multi-platform AR solution so that we could democratize AR development:


    Today we announced the first step in that endeavor - ARFoundation: https://blogs.unity3d.com/2018/06/15/multi-platform-handheld-ar-in-2018-part-1/

    Here is a summary:

    ARFoundation currently exposes these features:
    • Planar surface detection
    • Depth data represented as point clouds
    • Performant pass-thru rendering
    • Reference points to aid in anchoring virtual objects to the physical world
    • Estimates for average color temperature and brightness
    • Tracking device position and orientation in physical space
    • Utilities for scaling content properly in AR
    • Raycasting against plane and depth data
    Get started by installing 2018.1 with iOS and/or Android support, getting the sample projects from arfoundation-samples (they include ARFoundation and its required platform specific parts in their manifest), and deploying to your iOS and/or Android device.

    Read up on the documentation, and come to this forum for any questions or comments. We're also working on more tools that work with ARFoundation, and integrating more recent features of the AR platforms.

    So give this a try for your AR projects and give us feedback here! Happy developing!
     
    Last edited: Jun 16, 2018
  2. ZeKJ

    ZeKJ

    Joined:
    Jul 16, 2016
    Posts:
    4
    I cannot seem to find ScaledContent, is it coming soon? :) Thanks
     
  3. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    Support for scale is still there, but we decided to remove the explicit "scale" field from the ARSessionOrigin, since all it did was uniformly set the transform of its GameObject. There's no example in the sample repo yet, but all you have to do is set the scale of the transform of the GameObject with the ARSessionOrigin, and it will do the same thing as you may have seen in ARInterface.

    Cheers,
    Tim
     
  4. kexar66

    kexar66

    Joined:
    Feb 27, 2013
    Posts:
    38
    Good Job! It seems I cannot iterate in editor (like in AREditorInterface), am I missing something?
     
  5. Saicopate

    Saicopate

    Joined:
    Sep 25, 2017
    Posts:
    71
    hey, first of all it's good to have this package finally! :)

    A question: How is camera permission managed for Android? Is it still on ARCore side? Previously I was able to subscribe to AndroidPermissionsManager event form GoogleARCore to get notified about user permission. But now ARCore API is not exposed.. Is there any way?
     
  6. ZeKJ

    ZeKJ

    Joined:
    Jul 16, 2016
    Posts:
    4
    Thanks Tim

    Does it still only "transform" the camera? We changed the "Place On plane" script, to find our gameobject by tag and move it.

    When i´m testing, it seems i only move part of my scene. And that dosent make sense if the camera is the one moving? Is it becourse i need to anchor the full scene?

    Hope this makes sense, thanks!
     
  7. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    Support for in-Editor remoting and simulation is coming soon (it will be in another package). Stay tuned!
     
    kexar66 likes this.
  8. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    If the app has been configured as AR Optional (instead of AR Required, see https://docs.unity3d.com/Packages/com.unity.xr.arcore@1.0/manual/index.html#build-settings ), then camera permissions are requested when you try to start an AR Session (in ARFoundation, this happens when the ARSession component is enabled). There is no callback (maybe there should be?), but you can query the current state of camera permissions with
    Code (CSharp):
    1. ARSubsystemManager.cameraSubsystem.IsPermissionGranted()
     
  9. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    The "PlaceOnPlane" sample component places itself on the plane, that is, it moves the content. This is to show how to place very simple content, like a cube, onto the plane using a raycast.

    If you want to move the entire camera rig (and all trackables) to make complex content (e.g., terrain) appear on a plane (but without actually moving the content itself), then you should look at https://docs.unity3d.com/Packages/c...ContentAppearAt_Transform_Vector3_Quaternion_

    This is conceptually similar to this method in ARInterface: https://github.com/Unity-Technologi.../UnityARInterface/Scripts/ARController.cs#L92
     
  10. ZeKJ

    ZeKJ

    Joined:
    Jul 16, 2016
    Posts:
    4
    Thanks alot! Are you still planning on sharing your sample projects, from March 19 talk? :)
     
  11. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    We'll be adding samples to the existing github repo over time. It may not be exactly those samples, but something to show similar concepts, yes.
     
    ZeKJ likes this.
  12. frankCIQ

    frankCIQ

    Joined:
    Apr 6, 2018
    Posts:
    5
    I swapped out the Unity ARKit 1.5 plugin with the new AR Foundation and builds worked great until I went to create an archive build in Xcode for publishing to Test Flight. Then I keep getting stuck on this error:

    ld: bitcode bundle could not be generated because '/Users/[UserName]/[ProjectName]/[BuildName]/Libraries/UnityARKit.a(RootViewSize.o)' was built without full bitcode. All object files and libraries for bitcode must be generated from Xcode Archive or Install build for architecture arm64

    Any ideas? Thanks!
     
  13. Seto

    Seto

    Joined:
    Oct 10, 2010
    Posts:
    230
    What about other ar features? Will them be supported? Like face tracking, worldmap, reference image, reference objects of ARKit? Like cloud anchors , augmented image of ARCore?
     
  14. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Yes the idea is that these features will be implemented into ARFoundation in a multiplatform manner soon.
     
  15. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    Hmm, looks like we need to build the ARKit plugin with
    -fembed-bitcode
    to allow that. We don't do that currently; I'll look into it for the next release.
     
    UFODriverr, rocket5tim and frankCIQ like this.
  16. frankCIQ

    frankCIQ

    Joined:
    Apr 6, 2018
    Posts:
    5
    Appreciate it!
     
  17. rocket5tim

    rocket5tim

    Joined:
    May 19, 2009
    Posts:
    233
    I'm really happy to see things are moving along with AR Framework! I'm doing some preliminary investigation before I replace the Experimental ARInterface in my project with AR Framework.

    Does the Raycasting stuff work at all in the editor? With ARInterface I could aim AR Root around in the scene to raycast against planes in the ARGameObject layer. But the AR Default Plane in AR Framework doesn't work the same way so I'm having a hard time figuring out how I can do simple raycast tests from the AR Camera in the editor.
     
  18. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    Not currently. In-Editor simulation is coming (via another package).
     
    Saicopate and rocket5tim like this.
  19. Saicopate

    Saicopate

    Joined:
    Sep 25, 2017
    Posts:
    71
    Thanks Tim. I ended up using a separate plugin /UniAndroidPermission/ because I need to request camera permissions independently for a non AR-scene.

    BTW. What's up with ARAnchors? Are they not needed anymore or not currently supported?

    BTW2. I filed a bug report - > quitting ARscene causes Android app to hang and throw a leakage warning in Android Monitor, easly reproductuble with sample scene + added non-AR scene (all attached in bug report).
     
  20. rocket5tim

    rocket5tim

    Joined:
    May 19, 2009
    Posts:
    233
    What's the best way to remove ARPlane's after they've been created and stop generating new ones?

    Here's what I've tried so far:

    Code (CSharp):
    1. public class RemovePlanes : MonoBehaviour
    2. {
    3.     [SerializeField]
    4.     private ARPlaneManager m_ARPlaneManager;
    5.  
    6.     private List<ARPlane> m_planes = new List<ARPlane>();
    7.  
    8.     public void RemoveAllPlanes()
    9.     {
    10.         if (m_ARPlaneManager.planeCount == 0)
    11.         {
    12.             Debug.Log("** no planes to remove");
    13.             return;
    14.         }
    15.  
    16.         // get all the planes in the scene
    17.         m_ARPlaneManager.GetAllPlanes(m_planes);
    18.  
    19.         // and disable their game objects
    20.         for (int i = 0; i < m_planes.Count; i++)
    21.         {
    22.             Debug.Log("Removed "+m_planes[i].name);
    23.             m_planes[i].gameObject.SetActive(false);
    24.         }
    25.  
    26.         // disable the plane manager so we stop generating new planes
    27.         m_ARPlaneManager.enabled = false;
    28.     }
    29. }
    The planes are disabled, plane generation stops and physics objects that were placed on top of the AR Planes fall through. So far so good.

    But then in another script I'm aligning an object to the AR Planes with a Raycast. After removing the planes with the above script, my "hit object" in the following script still moves around on the plane as if it's still totally there.

    Code (CSharp):
    1.     void Update ()
    2.     {
    3.         var hits = s_RaycastHits;
    4.         Ray ray = new Ray(m_camera.position, m_camera.forward);
    5.  
    6.         if(m_SessionOrigin.Raycast(ray, hits, TrackableType.PlaneWithinPolygon))
    7.         {
    8.             m_hitObject.position = hits[0].pose.position;      
    9.         }
    10.     }
    11.  
    12.     List<ARRaycastHit> s_RaycastHits = new List<ARRaycastHit>();
    So it seems that the physical Plane has been removed, but there is still some aspect of the plane there that the Raycast is hitting. This is probably ok, but it makes me think that I'm not removing the Planes correctly.
     
    Last edited: Jun 21, 2018
  21. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    You have to distinguish between the AR platform hit testing and Unity's Physics hit testing. The GameObjects that you create in Unity that correspond to the planes that are detected will react to Unity Physics raycast. The actual planes that were detected in the AR platform will still be there even if you delete the GameObjects. To delete the existing planes on the AR platform, you will have to restart the session (or reset it with some options to remove planes).
     
  22. rocket5tim

    rocket5tim

    Joined:
    May 19, 2009
    Posts:
    233
    Thanks fro the reply, Jimmy. Switching to the following for the raycasting did the trick

    Code (CSharp):
    1.         RaycastHit hit;    
    2.         if (Physics.Raycast(ray, out hit, 100f, m_layerMask))
    3.         {
    4.             m_hitObject.position = hit.point;
    5.             Debug.Log("*** LAYER: "+hit.transform.gameObject.layer);
    6.         }
    One thing I noticed, is that in ARPlaneManager's AddPlane function, the Plane's layer is being set like this:
    go.layer = gameObject.layer;
    so if you want to set your planes to be in a specific layer you need to change the layer that the ARPlaneManager component is on to that layer. Easy enough, but not super obvious w/out looking in the code.

    What's the best way to restart the session and/or reset with options? Would you simply disable and then enable the ARSession (sounds iffy)?
     
    jimmya likes this.
  23. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    disable and enable map to pause and resume, respectively. To restart the session, that is, destroy all trackables and reset world tracking, you can destroy and re-create the ARSession component.
     
  24. Saicopate

    Saicopate

    Joined:
    Sep 25, 2017
    Posts:
    71
    Rocket5tim no need to switch to physics raycasting, try disabling collider components on planes from m_planes List :)
     
  25. tdmowrer

    tdmowrer

    Unity Technologies

    Joined:
    Apr 21, 2017
    Posts:
    577
    ARAnchor may refer to platform-specific implementations. The equivalent concept in Unity is a "Trackable". Anything that is a trackable, e.g., a plane, gets a TrackableId. See this glossary.

    Thanks for the bug report; I'll take a look. Do you happen to know the bug ID?
     
  26. Saicopate

    Saicopate

    Joined:
    Sep 25, 2017
    Posts:
    71
    Sure, case ID: 1051049. Thx.
     
  27. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    colliders are the components of GameObjects that respond to Unity Physics Raycasts. So in this case disabling them will have no effect on the ARRaycast.
     
  28. JokerZ

    JokerZ

    Joined:
    May 16, 2014
    Posts:
    4
    Can i build android application for devices with api lower than 24 and just start ARSession if it available on current device? Or maybe build another version without arcore support. Currently Unity told me that ARCore min version 24 required even if in XR Settings ARCore support is disabled.
     
  29. Saicopate

    Saicopate

    Joined:
    Sep 25, 2017
    Posts:
    71
    Good to know.

    EDIT: You are right - colliders do not respond here. What I did was to put additional if statement before ARRaycast that turns to false after I place an object on the plane.
     
    Last edited: Jun 22, 2018
  30. pixoglcresce

    pixoglcresce

    Joined:
    Jan 24, 2018
    Posts:
    5
    I can't seem to find the method: https://docs.unity3d.com/2018.1/Documentation/ScriptReference/Experimental.XR.XRCameraSubsystem.html
    Were you maybe referring to another class that might provide it...?

    On a related note (permissions): when using the ARCore plugin for Unity, additional permissions (for example, Location) are not automatically requested anymore and requests need to be performed manually (see: https://github.com/google-ar/arcore-unity-sdk/issues/14).
    It seems to be the same when using ARFoundation: I'm trying it out in a project which requires access to fine location and the request is not popping out automatically. When using the ARCore plugin, it's possible to leverage on the AndroidPermissionManager class to perform the check manually, is there any equivalent provided with ARFoundation?

    Thanks!

    EDIT: Upon further inspection, I found the class UnityEngine.XR.ARCore.ARCorePermissionManager, which seems to leverage on the same .aar library as the ARCore plugin.
     
    Last edited: Jun 22, 2018
  31. DarrenDuck

    DarrenDuck

    Joined:
    Mar 7, 2016
    Posts:
    3
    Hi Guys,

    We are making an Android app which will target Android 4.4 KitKat. We also want to include an optional AR game in it for people who have Android 7.0 and an AR capable device.

    Is it possible to use AR foundation for this optional AR game and target KitKat but also only have one version of the apk on the Play Store?

    Hope that makes sense.
    Great work, Thanks!
     
  32. rocket5tim

    rocket5tim

    Joined:
    May 19, 2009
    Posts:
    233
    Thanks Tim, but just to clarify what do you mean by "disable and enable map.."? What's "map"?
     
  33. Balours

    Balours

    Joined:
    Nov 27, 2013
    Posts:
    52
    Hi, thanks for the beginning this package.
    I'm trying to implement the scale content feature as it's almost impossible to work without it in AR ...
    I tried to simply scale the AR Session Origin transform but obviously everything move away from the camera as soon as the transform's scale is changed.
    I tried to implement the example from the AR Interface, but the surface detection is not working correctly anymore everything as an offset coming from I don't know where and the camera movement doesn't seems quite right regarding the AR content. If anyone as examples or clues on how to achieve this it would be very helpful.
    Thanks a lot
     
  34. rocket5tim

    rocket5tim

    Joined:
    May 19, 2009
    Posts:
    233
    You basically have to hold your objects in place while scaling to keep them from shooting off. One way is to snap the object's position to the hit.point on one of the ARPlane's by raycasting from the AR camera. Here's an example that should help get you started:

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.Experimental.XR;
    3. using UnityEngine.XR.ARFoundation;
    4. using UnityEngine.UI;
    5.  
    6. public class ARScaleExample : MonoBehaviour
    7. {
    8.  
    9.     [SerializeField]
    10.     private Transform m_camera;
    11.  
    12.     [SerializeField]
    13.     private Transform m_myObject;
    14.  
    15.     [SerializeField]
    16.     private Slider m_scaleSlider;
    17.  
    18.     [SerializeField]
    19.     private Transform m_ARSessionTrans;
    20.  
    21.     [SerializeField]
    22.     private LayerMask m_layerMask;
    23.     void Update()
    24.     {
    25.         Ray ray = new Ray(m_camera.position, m_camera.forward);    
    26.         RaycastHit hit;
    27.         if (Physics.Raycast(ray, out hit, 100f, m_layerMask))
    28.         {
    29.             m_myObject.position = hit.point;
    30.         }
    31.     }
    32.  
    33.     public void OnSliderChanged()
    34.     {    
    35.         var scale = m_scaleSlider.value;
    36.         m_ARSessionTrans.localScale = new Vector3(scale, scale, scale);
    37.     }
    38. }
     
  35. Kiupe

    Kiupe

    Joined:
    Feb 1, 2013
    Posts:
    528
    Hello,

    Is this project compatible with your newly announced MARS project or will it be a replacement ?

    This project seems to let user not choose between ARCore and ARKit, but will it bring marker detection like Vuforia or is it compatible with Vuforia ?

    Thanks
     
  36. CJRobertson932

    CJRobertson932

    Joined:
    Jul 11, 2016
    Posts:
    5
    Would this be the same idea as setting the world origin so EVERYTHING moves? Meaning if I place a bunch of objects in the world while in editor (so not anchored to the world using AR) and attempt to move the world origin, would all of the objects I placed before and after AR use move accordingly?
     
    Last edited: Jun 25, 2018
  37. TijsVdV

    TijsVdV

    Joined:
    Mar 7, 2017
    Posts:
    21
    How do you delete the tracked point clouds? Disabeling the trackable parent doesnt seem to work.
     
  38. O_Monteiro

    O_Monteiro

    Joined:
    Jul 7, 2015
    Posts:
    2
    Hi there!
    First of all: Much appreciation for this package!
    I tried the sample on iOS and it worked flawlessly however when I built it for android it simply shows no planes whatsoever (built with Unity2018.1.6f1 and running on Google Pixel 2). Anyone having this issue?

    How should we go about importing this in a ongoing project? I just imported the 3 packages (AR Foundation, ARCore XR Plugin and ARKit XR Plugin) using Window->Package Manager. Everything seemed fine, but building for iOS throws an error on XCode. Project can't find UnityARkit.m on the Libraries folder. Building the sample actually places such file there but I can't have the same behaviour with another project (manually copying the file does the trick but I'm afraid something breaks because of this).

    Upon performing certain actions on my ongoing project with this package imported I get an all black screen and no errors at all.How should I try and debug this? Anyone experienced such issues?

    Thanks!
     
  39. sam598

    sam598

    Joined:
    Sep 21, 2014
    Posts:
    59
    The sample scene will run for a few second on iOS, and will then stop tracking. Ground plane and point clouds become "stuck" to the screen, while the camera updates in the background. This happens without fail in multiple environments.

    No errors but I am receiving these messages in XCode.

    Code (CSharp):
    1. 2018-06-25 11:16:14.292269-0700 arexample[378:13681] [Technique] World tracking performance is being affected by resource constraints [0]
    2. 2018-06-25 11:16:15.283777-0700 arexample[378:13681] [Technique] World tracking performance is being affected by resource constraints [0]
    3. 2018-06-25 11:16:16.286994-0700 arexample[378:13681] [Technique] World tracking performance is being affected by resource constraints [0]
    4. 2018-06-25 11:16:33.899677-0700 arexample[378:13681] [Technique] World tracking performance is being affected by resource constraints [1]
    5. 2018-06-25 11:16:34.874913-0700 arexample[378:13681] [Technique] World tracking performance is being affected by resource constraints [1]
    6. 2018-06-25 11:16:41.156274-0700 arexample[378:13681] [Technique] World tracking performance is being affected by resource constraints [1]
    The device is a iPhone 7+ running iOS 12 Beta 1. This does not happen with other applications built with the standalone ARKit plugin. It also does not happen on iOS devices running iOS 11.4. The frame rate of the of the AR Foundation scene also appears to be slower than the standalone ARKit plugin. Because of the new package manager workflow it's hard to tell what settings specific to ARKit could be causing this issue.

    Please let me know if there is more information I can provide. Excited to get started using AR Foundation.
     
  40. sam598

    sam598

    Joined:
    Sep 21, 2014
    Posts:
    59
  41. Kiupe

    Kiupe

    Joined:
    Feb 1, 2013
    Posts:
    528
    Hello guys,

    I'm trying to compile to Android the arfoundation-samples project but I'm experiencing an issue. It seems that some UnityARCore classes are not found

    Screen Shot 2018-06-26 at 11.14.58.png

    I did check, and the Temp/StagingArea/Andrdoid-librairies/UnityARCore/classes.jar indeed does not exist. I tried to delete the tmp folder and re-open the project but the error is still there.

    Any idea why ?

    Thanks

    [EDIT] I removed and re-installed all AR packages and the build succeeded. Now I can run the application on my phone (OnePlusOne 5T) and the screen keeps black. I was wondering if the camera access is correctly granted. Should I do something by myself to grant permission ?
     
    Last edited: Jun 26, 2018
    DreamEnder likes this.
  42. rob_ice

    rob_ice

    Joined:
    Nov 11, 2016
    Posts:
    112
    @jimmya @timmunity Any word on having the image detection features of both ARKit/ARCore included here?
     
  43. IonutReachRobotics

    IonutReachRobotics

    Joined:
    Sep 14, 2017
    Posts:
    1
    Hi guys,

    I'm having a bit of a hard time getting the background image of the camera in our AR app. From what I could gather, all that I need to actually do is make a call to
    ARSubsystemManager.cameraSubsystem.GetTextures(cameraTextures);

    I do get a texture back that's the proper size and everything but it's completely black.
    I'm currently developing on Android (Samsung S7 and S8) so that's the only platform I tested on.

    In the scene I have an ARSession, an ARSessionOrigin with an AR Camera in it (Target Texture: None, Tracked Pose Driver, AR Camera Background).
    The AR Background works fine but is there any specific limitation on the cameraSubsystem that I'm not aware of?

    I guess I could set a render target on the camera and show that as a background, but GetTextures approach plugs right into our existing project without any major rework and I would love to keep it that way for the time being.
    Thanks! :D
     
  44. DreamEnder

    DreamEnder

    Joined:
    Apr 12, 2011
    Posts:
    190
    ARSessionOrigin - Setting surface detection orientation
    How do you go about setting which surfaces to detect? I only want to detect horizontal surfaces (floors).
     
    Saicopate likes this.
  45. rob_ice

    rob_ice

    Joined:
    Nov 11, 2016
    Posts:
    112
    Cross-platform multiplayer session connecting for this would be fantastic #dreamfeatures
     
  46. sam598

    sam598

    Joined:
    Sep 21, 2014
    Posts:
    59
    So AR Foundation runs at 30 fps on iOS, while the standalone UnityARKit plugin runs at 60 fps.

    These are the default scenes with no assets in them. What would be causing the framerate to drop like this?

    With AR Foundation UnityARKit.a is a static library so there is no way to see if this framerate is being hard coded into the plugin.

    Tested on both an iPhone 6s and 7+.
     
  47. kennyd-itp

    kennyd-itp

    Joined:
    Jan 13, 2014
    Posts:
    30
    Unity defaults to 30fps on mobile (I guess for battery savings). You're right that UnityARKit plugin sets it to 60fps but ARFoundation doesn't - not sure if that's an oversight or a deliberate decision. So you'll need to set it yourself:
    Code (CSharp):
    1. Application.targetFrameRate = 60;
     
  48. sam598

    sam598

    Joined:
    Sep 21, 2014
    Posts:
    59
    Yup, that was it.

    This also seems to solve the issue of ARKit crashing in iOS 12. Maybe 30fps is causing some kind of bottleneck on the GPU that ARKit needs in order to run properly.
     
  49. Kiupe

    Kiupe

    Joined:
    Feb 1, 2013
    Posts:
    528
    Does anyone have informations about image/marker detections like within Vuforia ?

    Thanks
     
  50. DreamEnder

    DreamEnder

    Joined:
    Apr 12, 2011
    Posts:
    190
    I'm not sure if it'll help but you can add this line to your Android Manifest:

    <uses-permission android:name="android.permission.CAMERA"/>
     
unityunity