Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official AR Foundation v5.0

Discussion in 'AR' started by tropicdragon_unity, Aug 24, 2022.

  1. tropicdragon_unity

    tropicdragon_unity

    Unity Technologies

    Joined:
    Feb 21, 2020
    Posts:
    1
    What’s New in AR Foundation 5.0

    Version 5.0 of AR Foundation is here and comes with several changes and new features. AR Foundation 5.0 works with Unity 2021LTS, Unity 2022.1 and is available in beta with Unity 2022.2.

    Changes
    For more information on migrating your project to 5.0, see the AR Foundation migration documentation.

    New Features
    • XR Simulation allows you to run your AR app in Play mode by simulating an AR device, no longer requiring you to rebuild your app to test changes.
    • AR Debug Menu is a new Prefab you can use to view available configurations on your device and visualize AR subsystem data such as planes and point cloud positions.
    Requirements
    • AR Foundation 5.0 requires Unity 2021LTS and later.

    XR Simulation

    AR Foundation can simulate an AR device, allowing the testing of AR enabled scenes in the Editor. In play mode you can navigate around a variety of simulated environments detecting planes, tracked images, etc. as if you are using a real device. You can also create your own environments to use in the simulation.

    The following AR Foundation features are supported:



    Enable simulation

    Go to Project Settings > XR Plug-in Management > Windows, Mac, Linux and enable XR Simulation. When you enter Play Mode in the editor with a scene that contains an ARSession, a simulated environment is loaded. Click the right mouse button in the Game view to look around and press the W, A, S, and D keys to navigate around the environment.

    As you navigate around the environment, AR Foundation discovers features of the simulated environment as if you were navigating around a real space. However, there are a few differences compared to using a real device:
    • Planes: Planes are only discovered on axis aligned surfaces.
    • Meshes: Meshes are all discovered immediately on entering play mode.
    • Tracked images: Any tagged image in the simulation environment can be tracked regardless of what images are in your reference library. This allows testing without having to modify either your reference image library or the environment.
    For more in-depth information on how to set up and work with AR Simulation, please refer to the AR Foundation Simulation documentation. You can also watch a tutorial from AR Content Creator and Founder of LearnXR, Dilmer Valecillos, on YouTube.

    Unity Mars and Simulation

    Unity Mars includes a simulation feature that can work with AR Foundation. Unity Mars’ Simulation System supports the same features as AR Foundation and more. We recommend that Unity Mars developers continue with Unity Mars simulation. Note that both Simulations features cannot be used at the same time.

    XR Origin

    ARSessionOrigin is now deprecated and is replaced by XROrigin. The XR Origin represents the center of worldspace in an XR scene. The purpose of the XR Origin is to transform objects and trackable features to their final position, orientation, and scale in the Unity scene. It specifies an Origin, a Camera Floor Offset Object, and a Camera.


    For more information on migrating ARSessionOrigin to XROrigin, see the AR Foundation migration documentation.

    AR Debug Menu

    You can add the AR Foundation Debug Menu to an AR scene to visualize tracking information and available configurations. When that scene is active on a target device, you can use the menu to view the following information:
    • Stats Info
      • FPS
      • Tracking Mode
    • Session Configurations
    • Debug Options
      • XR Origin
      • Planes
      • Anchors
      • Point Clouds
    The AR Debug menu works with any scene that contains an ARSession, an XROrigin, and a Camera.

    The Debug menu is a Prefab object containing the ARDebugMenu script and a set of GameObjects and components that provide the UI. See Setting up the menu for instructions to add the Debug menu to a scene.

    Setting up the menu


    To create an out-of-the-box fully configured debug menu, right-click anywhere in the Hierarchy window and select XR > AR Debug Menu.



    The AR Debug menu appears as a toolbar that is anchored either to the left or bottom side of the screen.

    Stats Info

    The Stats Info tab provides general information about a session, including current FPS and TrackingMode.



    Session Configurations

    The Session Configurations tab displays all the available session Configurations for a specific platform and highlights the currently active configuration.



    Debug Options

    The Debug Options tab allows the user to toggle visualizers for various ARTrackables in the scene.

     
  2. mikeyrafier98

    mikeyrafier98

    Joined:
    May 19, 2022
    Posts:
    37
    I have tested AR Foundation 5.0 before while still using stable ver 4.2.
    It's quite interesting to have many improvement, especially now it can simulate in Editor before compiling to real device.
    Although to migrate the project, need to careful watch the changes, such as the XR Origin.
    Overall, very nice.

    I would like to ask two questions.
    1. As being said it needs at least Unity 2021 LTS, does it also affect the minimum requirement for real/mobile device?
    For example, for at least Android 14, but since AR Foundation 5.0, it will required ....

    2. AR camera now has camera Y offset, which represent of user height.
    I read the script documentation that this feature will be useful for TrackingOriginMode = floor
    Does it affect if we use standard, or I think NotSpecified origin mode?
    I still confuse about this one...
     
    Last edited: Aug 29, 2022
  3. Deleted User

    Deleted User

    Guest

    Last edited by a moderator: Aug 29, 2022
  4. CiaranWills

    CiaranWills

    Unity Technologies

    Joined:
    Apr 24, 2020
    Posts:
    198
    ARCore itself doesn't support meshing the way ARKit does, hence why it's not supported. However depending on what you are trying to achieve you might find the depth image useful.
     
    KyryloKuzyk likes this.
  5. Deleted User

    Deleted User

    Guest

    @CiaranWills Thanks for the reply. But as the ARCore Depth API seems to enable (at least some sort of) meshing, I was wondering if AR Foundation would support meshing through AR Mesh Manager at some point. Or is this technically impossible?
     
  6. Stents

    Stents

    Joined:
    Jan 23, 2014
    Posts:
    18
    Am I missing something? In Unity 2021.3.8, Version 4.2.3 is still the only available option via the package manager. Also the docs still only list 5.0.0-pre.13 as the latest release. Is this just an updated post about the old pre release? Or is there actually a new full release of ARFoundation 5.0 coming out and it is just not listed in the package manager yet?
     
    rianders likes this.
  7. ankur-unity

    ankur-unity

    Unity Technologies

    Joined:
    Aug 12, 2021
    Posts:
    34
    AR Foundation 4.2.3 is still the verified version for Unity 2021.3 (LTS). However, you can manually update the entry
    "com.unity.xr.arfoundation": "4.2.3"
    in the
    Packages/manifest.json
    file in your Unity project to point to the "5.0.0-pre.13" version of the package. You might need to update the versions for
    com.unity.xr.arcore
    and
    com.unity.xr.arkit
    packages as well.

    AR Foundation 5.0 full-release is on the way and will be available in the package manager soon. Right now, 5.0.0-pre.13 is the latest publicly available version of the package.
     
    JumpingGuy likes this.
  8. Stents

    Stents

    Joined:
    Jan 23, 2014
    Posts:
    18
    I see, thanks for the clarification @ankur-unity !

    I found it a bit confusing that there was this new post and no mention that it was still referring to pre.13, which has been out for a while now.

    I would like to also pass on my dislike for the change of hiding all the non verified versions completely. I know many other people have expressed this as well, but I do not see the point in hiding these things and making it harder for developers to use other versions.

    Appreciate the work on the new features that are coming with AR Simulation etc though, very much looking forward to the full release!
     
    Christin2015 likes this.
  9. ankur-unity

    ankur-unity

    Unity Technologies

    Joined:
    Aug 12, 2021
    Posts:
    34
    @Stents - Fair point. We could have been clear that at the time of writing this post, pre.13 is available and 5.0.0 is in QA, soon to be released. And thank you for all your feedback. I will pass your feedback about the package manager's handling of non verified versions to the internal team at Unity.
     
  10. ankur-unity

    ankur-unity

    Unity Technologies

    Joined:
    Aug 12, 2021
    Posts:
    34
    I have good news, 5.0.2 is now available. AR Foundation samples have been updated to point to the 5.0.2 versions of the AR Foundation packages.
     
  11. seyfe

    seyfe

    Joined:
    May 10, 2019
    Posts:
    74
    Our application uses the
    ConvertAsync(..)
    method of an
    XRCpuImage
    for getting the camera image data. Running the simulation in editor, it is reported that camera-image conversion is not supported. Without this it the simulation is not that helpful - (when) can we expect this to be implemented?
     
  12. jalamparambil_unity

    jalamparambil_unity

    Joined:
    Dec 16, 2021
    Posts:
    4
    I have 2021.3.9f1 installed, and it does not let me choose ARFoundation 5.0.2 from the package manager, even if I enable pre-release packages.
     
    rianders likes this.
  13. CiaranWills

    CiaranWills

    Unity Technologies

    Joined:
    Apr 24, 2020
    Posts:
    198
    Yes, that's on our roadmap - so you can expect it but I can't give you a when right now.
     
  14. afurioso

    afurioso

    Joined:
    Jan 12, 2022
    Posts:
    88
    Hi, I have unity LTS 2021.3.4f1 but I can install AR foundation 5.0.2. What should I do?
     
    rwby2373 likes this.
  15. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
  16. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    I don't see AR Foundation 5 in the package manager. I have pre-release packages enabled and am on 2021.3.x, which is higher than the minimum according to the docs. What is happening?
     
  17. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @JelmerV Thanks for asking as this is a common pain point. See my message above for how to access AR Foundation 5.0.2 in Unity 2021.3. At the time of writing, we are in the release process for 5.0.3, which includes a documentation update addressing this topic.
     
  18. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    Ok, new reply (deleted the old one). I actually run into a bunch of problems:
    Unity 2021.3.11: I added ARFoundation5 via the custom package manager way, described in your post. The result is a hot red glowing console, see attached image.
    Unity 2022.1.22: I folllow the documentation that states "Thus, AR Foundation 5.0 appears in the Package Manager window in Unity 2022.x versions.". Well, NO ARFoundation 5 in my package manager.

    I've been using ARKit in Unity since 2017 and have been using ARKit plugin and later AR Foundation in many projects and it is pretty crazy that I am already using more than an hour (again) to try to get this to work. :confused:
     

    Attached Files:

  19. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    Hi @JelmerV

    For your 2021.3 issue, make sure that you have the same version numbers installed for the following packages:

    • com.unity.xr.arfoundation
    • com.unity.xr.arkit
    • com.unity.xr.arcore
    When upgrading to AR Foundation 5, you should remove the following packages as stated in step 4 of that new docs page:

    • com.unity.xr.arsubsystems // this assembly has been merged into AR Foundation in 5.0
    • com.unity.xr.arkit-face-tracking // this assembly has been merged into the ARKit package in 5.0
    For more information see our Migration Guide.

    Thanks for pointing out the error regarding our documentation of Unity 2022.1. You still need to manually add AR Foundation 5.0 to your manifest in that version. AR Foundation 5.0 is in "Released" status as of Unity 2022.2, so 2022.2 is the first Editor where it appears in the package manager, and we'll update our docs to correctly reflect that.
     
    JumpingGuy and John1515 like this.
  20. Gmora7

    Gmora7

    Joined:
    Apr 17, 2013
    Posts:
    12
    Has anybody been having "wobbling" issues with iOS 16? By wobble I mean the object placement is anchored in place, but when you move the phone around there is a subtle wobble to it and it doesn't have 100% stable placement.
    Unity 2021.3.9f1
    ARFoundation 5.1.0-pre2
    ARKit XR Plugin 5.1.0-pre2
     
  21. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Maybe you're experiencing this issue:
    https://issuetracker.unity3d.com/is...mutithreaded-rendering-when-running-on-ios-16
     
  22. Gmora7

    Gmora7

    Joined:
    Apr 17, 2013
    Posts:
    12
  23. Gmora7

    Gmora7

    Joined:
    Apr 17, 2013
    Posts:
    12


    Here is a visual example of the "wobble" showing the cubes not being fully stable on the ground plane, the cubes slide around.
    iOS 16.1.1
    Unity 2021.3.9f1
    "com.unity.xr.arcore": "5.1.0-pre.2",
    "com.unity.xr.arfoundation": "5.1.0-pre.2",
    "com.unity.xr.arkit": "5.1.0-pre.2",

    I also tried Unity 2023.1.0a19, same results...
     
    Last edited: Nov 19, 2022
    EthanFischer likes this.
  24. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @Gmora7 If you move your device with high acceleration then it can indeed be possible for objects to drift. That's primarily what I see when I watch this video.

    "100% stable placement" as you've described it is not even theoretically possible with any realtime visually based tracking system. There will always be factors like lighting conditions, feature point density, sensor noise, vibrations, etc, that will cause some margin of tracking error.

    Note also that Unity does not provide an implementation of device tracking-- this comes from provider implementations such as Google's ARCore and Apple's ARKit. Neither Apple nor Google release any official numbers about their tracking precision.

    As a recommendation if you aren't already doing so, tracking stability will always improve if your cube is anchored using a standalone Anchor as opposed to attaching the anchor to a plane (or simply parenting the cube as a child of the plane GameObject). You should always use Anchors when stability of object placement is your goal.
     
  25. Gmora7

    Gmora7

    Joined:
    Apr 17, 2013
    Posts:
    12
    My tests have been in the ARFoundations example project on github:
    /Assets/Scenes/Anchors/Anchors.unity

    I've heard from other engineers when running AR with iOS 16 Unity's main thread and render thread get out of sync. Is this a known issue with Unity?
     
  26. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    This is not true. There was a different iOS 16-related rendering bug in our ARKit plug-in package, but with the release of our latest patches we are unaware of any iOS 16-specific rendering issues.
     
  27. dandanthepizzaman666

    dandanthepizzaman666

    Joined:
    Jul 15, 2019
    Posts:
    6
    Hi, my goal is to achieve steady image tracking for up to 8 images. I understand ARFoundation on iOS uses ARKit and version 4 reliably tracks 4 or less images. I read that ARF5 will reliably track many more. I've been testing the sample project here https://github.com/Unity-Technologies/arfoundation-samples/tree/5.0#image-tracking and have added a few more good quality image markers to the reference library. e.g.

    upload_2022-11-28_18-6-24.png

    As far as I can tell, the image tracking is still flakey for more than 4 images. Have I misunderstood or is this still a limitation with ARF5? Testing on iPhone XR running iOS16.
    Thanks in advance!
     
  28. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @dandanthepizzaman666

    From our docs:

    The AR Foundation package contains interfaces for AR features, but doesn't implement any features itself.

    Both AR Foundation 4 and AR Foundation 5 use ARKit's image tracking implementation on iOS. To reason about image tracking on iOS, we must look at Apple's documentation.

    Apple's guidance states:

    Limit the number of reference images in use at one time. Image detection performance works best when ARKit looks for 100 or fewer distinct images in the real-world environment.

    We made no changes to the Apple ARKit XR Plug-in's Image Tracking implementation with the release of AR Foundation 5.

    Note that tracking instability can be caused by incorrect texture import settings for non-power of two (NPOT) textures. For more information, see our docs: https://docs.unity3d.com/Packages/c...#reference-images-and-texture-import-settings.

    If you are still experiencing issues with image tracking on iOS 16, it is possible that Apple made a change in iOS 16 that affected your app. In that case you can file a bug with Apple: https://developer.apple.com/bug-reporting/.
     
    Last edited: Nov 30, 2022
    KyryloKuzyk likes this.
  29. Kasperknop

    Kasperknop

    Joined:
    Mar 28, 2017
    Posts:
    4
    I am also noticing significant regression in tracking when upgrading to ARFoundation + ARCore 5.0.3 in 2021.3.9f1.

    ARFoundation + ARCore 4.2.3


    ARFoundation + ARCore 5.0.3


    Same regression is present when running various scenes from the AR Foundation samples.
     
    Last edited: Nov 30, 2022
    EthanFischer likes this.
  30. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    Hi @Kasperknop

    Thanks for these videos. I can't tell by looking if this is an issue with AR Foundation or ARCore but this does appear to be some sort of regression. Can you please file a bug and post the bug number here? https://unity3d.com/unity/qa/bug-reporting
     
    EthanFischer likes this.
  31. Kasperknop

    Kasperknop

    Joined:
    Mar 28, 2017
    Posts:
    4
    Bug number: IN-24555
     
    andyb-unity likes this.
  32. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @dandanthepizzaman666

    Looking at your reference image again, I question how you came to the determination that this was a "good quality" reference image? Xcode has a reference image quality assessment tool:

    Not all images are equally suited for using them as ARReferenceImages. Good reference images typically have high contrast, good texture, clear edges, and are printed on a flat, non-glossy surface. Low-texture and low-contrast images will likely not work so well. When you add reference images to an ARResourceGroup in an Xcode asset catalog, Xcode will show quality estimation warnings, which may provide additional insights about the specific issue with that particular image.

    In some cases, you can try using an image editing tool to tweak contrast, exposure, or emphasize edges. Depending on the subject, cropping may work as well. For example, a picture showing mostly a blue sky with a small airplane could be cropped to just the region showing the airplane. If you can't get detection to work at all for a particular image, you probably need to consider replacing it.

    In particular I've seen developers struggle with images on a solid-colored background. This violates the "clear edges" requirement described above. I would suggest trying some different images and/or verifying the quality in Xcode to ensure that your tracking issue is not caused by reference image quality.
     
    Last edited: Nov 30, 2022
  33. ankur-unity

    ankur-unity

    Unity Technologies

    Joined:
    Aug 12, 2021
    Posts:
    34
    @Kasperknop
    Which Android device, android OS version, and ARCore APK version did you use to test the two AR Foundation versions in your videos?
     
  34. Kasperknop

    Kasperknop

    Joined:
    Mar 28, 2017
    Posts:
    4
    Samsung Galaxy S22 Ultra, Android 13, GPS for AR version 1.34.222640293
     
  35. arnesso

    arnesso

    Joined:
    Mar 3, 2015
    Posts:
    96
    Is there any news when will be available XR simulation input in Simulator mode?
    Currently the input works in Game mode only with keyboard and Mouse buttons, but i'd like to extend this input for touch input. (whole simulation is in dll read only)
     
    Last edited: Dec 9, 2022
  36. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @arnesso We've also noticed this. We have an open investigation right now, but it's trickier than we expected and may require a change to the Editor. We don't have an ETA for a fix, but if you wish to file a bug you and other community members can track the progress.
     
  37. Thaina

    Thaina

    Joined:
    Jul 13, 2012
    Posts:
    1,157
    When could we have webxr compatible in foundation?
     
    makaka-org and newguy123 like this.
  38. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
  39. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    KyryloKuzyk and newguy123 like this.
  40. DiogoSB

    DiogoSB

    Joined:
    Aug 24, 2018
    Posts:
    2
    I have an issue with AR Foundation and ARCore not working in devices like S22, Galaxy Tab S8 and S21 FE. Does 5.0 brings a solution to it? Or anybody has a suggestions how to fix?
     
  41. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @DiogoSB You can find Google's list of ARCore supported devices here: https://developers.google.com/ar/devices

    For devices on that list, if you are having trouble getting your app to launch correctly, make sure that you've covered all the steps in our ARCore Project Configuration guide: https://docs.unity3d.com/Packages/com.unity.xr.arcore@5.0/manual/project-configuration-arcore.html

    As an alternate starting point, our Samples app should work out of the box on supported ARCore devices: https://github.com/Unity-Technologies/arfoundation-samples
     
    KyryloKuzyk likes this.
  42. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    Would be great if we can get the AR Foundation samples on GIT updated to URP by default instead of built-in

    Would save us time having to do it manually
     
  43. seyfe

    seyfe

    Joined:
    May 10, 2019
    Posts:
    74
    Any updates on this? Currently, this limitiation is still quite an annoying obstacle for us for certain development tasks.
     
  44. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @newguy123 This is on the road map. We're not ready to do this immediately but possibly sometime in 2023.
     
    Thaina and newguy123 like this.
  45. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @seyfe XR Simulation support for CpuImages will be in our next 5.1 pre-release build in January.
     
  46. seyfe

    seyfe

    Joined:
    May 10, 2019
    Posts:
    74
    That's great to hear! Thanks for your quick reply :)
     
  47. ihsnfkrii

    ihsnfkrii

    Joined:
    Jan 10, 2023
    Posts:
    1
    yayasan alfatihah adalah yayasan pendidikan qura untuk anak-anak yang kecanduan gadget
     
  48. Dangers_Dad

    Dangers_Dad

    Joined:
    Jan 8, 2023
    Posts:
    3
    I am working with AR Foundation bug having issues with AR tracked Images. I have a post to ask for help but wandered if it’s a known issue that Image names are always Null. If so is there a work around?

    I don’t really want to go to Vuforia but wasted hours trying to find solutions.
     
  49. andyb-unity

    andyb-unity

    Unity Technologies

    Joined:
    Feb 10, 2022
    Posts:
    1,018
    @Dangers_Dad Please limit your forum activity to one thread per issue. This thread is about our August 2022 announcement of AR Foundation 5-- I have responded to your other thread.
     
  50. Dan4x8

    Dan4x8

    Joined:
    Feb 27, 2015
    Posts:
    13
    I wanted to ask if there are any news on the issue @Kasperknop described as I'm unable to find the mentioned issue (Bug number in-24555) in the issue tracker and we're running in the same problem.