Search Unity

  1. All Pro and Enterprise subscribers: find helpful & inspiring creative, tech, and business know-how in the new Unity Success Hub. Sign in to stay up to date.
    Dismiss Notice
  2. Dismiss Notice

[RELEASED] ✨ AR Simulation ✨ — AR development made easy

Discussion in 'Assets and Asset Store' started by fherbst, Jul 1, 2020.

  1. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    761
    I'm very happy to present you AR Simulation, our in-editor XR plugin to speed up development of ARFoundation applications.
    AR Simulation is available on the AssetStore, and we have a trial version that you can try for 7 days.

    CoverImage.png

    Build AR apps with confidence.
    Iterate fast, right in Editor.
    Non-invasive, drop-in solution.


    AR Simulation allows you to work on and test AR apps right in the Editor. No need for numerous deployments, no need to build complicated mock structures...
    Just move around in the Editor, test your AR app, without having to change any code or structure. Iterate faster, test out more ideas, build better apps!

    Features

    ✔️ Plane Tracking
    ✔️ Touch Input (single touch for now)
    ✔️ Simulated Environments
    ✔️ Device Simulator
    ✔️ Point Clouds
    ✔️ Image Tracking
    ✔️ Light Estimation
    ✔️ Anchors
    ✔️ Meshing
    ✔️ Camera Background Rendering

    simple-explanation.gif

    Licensing

    Using AR Simulation requires you to buy a per-seat license —
    please buy seats for your team through Unity AssetStore.
    This is a single payment, not a monthly subscription.

    You can use it for 7 days for evaluation purposes only,
    without buying a license. We trust you. :oops:

    Verified to work with

    • Unity 2019.3 • 2019.4 • 2020.1 • 2020.2 •
    • ARFoundation 3.x • 4.x •
    • Device Simulator •
    • Built-In • URP •
    • "Old" Input System • "New" Input System • "Both"•

    ARSimulation is a custom XR backend, built on top of the XR plugin architecture.
    Because it's "just" another XR Plugin and we took great care to simulate important features, it works with your existing ARFoundation app, new/old input system, XR Interaction Toolkit — zero changes to your code or setup needed! ✨
    And if you need more control, there's a lot of knobs to turn.

    For support, please see the Manual and our dedicated Discord channel.
    Please report bugs on GitHub.
     
    Last edited: Jul 8, 2020
    marwi and ROBYER1 like this.
  2. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    761
    TLDR: This post explains more about our approach for AR Simulation, why we think an XR plugin is a great solution, and how this compares to other solutions like AR Remote and MARS. For even more details, please refer to the Readme.

    Long story short, MARS (Mixed and Augmented Reality Studio), AR Remote and AR Simulation serve different use cases and audiences and each have their own Pros and Cons.
    • MARS is a framework. This means that it wraps around all ARFoundation functionality – all your code and design will have to go through MARS. This also means that MARS has a lot of complexity just because it needs to replicate all ARFoundation features (and extend them in some places), a process that is not complete (yet?). Additionally, MARS introduces lots of additional concepts that are useful for specific types of applications but not others (e.g. constraints, functionality injection, simulated environments, ...). At some unknown point in the future MARS will get companion apps (?) that allow you to test remotely on a device, and record data for future editor sessions.

    • AR Remote is a XR plugin for Editor that allows you to stream data from your device, in real-time, to test your app without deploying. This is very useful to reduce iteration times, but still requires you to have a physical device to test your app, and you always start "from scratch" (track some planes, find images, ...) for testing. Since it's an XR plugin, no changes to your existing AR Foundation app are required – for desktop, the remote backend is used, on device, the actual device backend will be used.

    • AR Simulation is also a XR plugin for Editor, and allows you to test your app right in the editor, without the need for a physical device. It supports Device Simulator (for testing various resolutions and target devices), allows to set up simple or complex test scenarios (e.g. what does my app do with 3 tracked images and 2 tracked planes, and suddenly an image loses tracking), and allows for simulated environments, automated testing, and others.
    We think AR Simulation has a unique selling point in ease of use – we spent a lot of time integrating it seamlessly with AR Foundation and the Unity Editor – and speed of iteration (literally press play and fly around and interact).

    Some features are currently better suited towards AR Remote, such as simulating tracked face data, or tracked skeleton data, were we felt the overhead of building "simulation tools" for was bigger than remoting to an actual device with actual data. However, that might change if enough people think that would be useful for them!


    Some additional background:
    We were active MARS alpha testers, trying to use it for our own AR applications, and started developing our own solution in parallel. After a while, we stopped using MARS (besides of course testing and giving feedback to new releases), as we felt that for our use cases it was too heavy, too complex, required too many changes to existing projects and approaches, and didn't have the features we needed (e.g. in terms of interacting with AR worlds).

    With AR Simulation, our goal were fast iteration times in the Editor for a range of AR applications we and partner companies build. These usually consist of placing and interacting with objects from different angles. We just needed a way to "simulate" an AR device in the Editor, not a full-blown additional framework!

    Fortunately, Unity provides the ability to build exactly that using the XR plugin architecture: a custom XR provider that works in the Editor and Desktop builds.
    (Note: we suggested to the MARS teams to implement it as XR backend; still not sure why they didn't.)
     
    Last edited: Jul 30, 2020
  3. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,164
    Been looking for a tool like this for a while. MARS just seemed very heavy for getting going with an AR Foundation project to test and rapidly iterate in the editor.
     
  4. saxovr

    saxovr

    Joined:
    Dec 13, 2017
    Posts:
    25
    Sounds really interesting. Would you please elaborate on the above? (as in comparison to AR remote and Mars)
     
    fherbst and ROBYER1 like this.
  5. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,164
    I'm sure the asset devs will reply back here soon but the comparisons you need are here in the manual for this asset:

    https://github.com/needle-tools/ar-simulation/
     
    saxovr likes this.
  6. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    761
    Thanks @ROBYER1, well appreciated.

    I updated the post with an in-depth explanation of how we see the current AR environment with MARS, AR Simulation, and AR Remote:
    • Each has its own respective use cases, and we think AR Simulation is the superfast, supereasy, non-invasive, don't-want-to-mess around solution that today's AR designers and developers need.

    • MARS might be the one future AR devs need, but nobody knows yet!

    • AR Remote is better when for cases where simulation will be too far off of real world, such as face tracking and skeletal tracking. For other features such as planes, image tracking, point clouds, environments, testing with AR Simulation is faster.
    Please let me know if you'd like to know further details or clarification!
     
    Last edited: Jul 30, 2020
    saxovr likes this.
  7. trnq1ll0

    trnq1ll0

    Joined:
    Aug 30, 2019
    Posts:
    27
    I use your plugin in two projects in where I didn't use it before and have to say i love it's simplicity. So thank you very much for your very tough work!

    I don't need the examples, but I meet an error while they are present:

    Code (CSharp):
    1. BuildFailedException: ARKit requires dimensions for all images. Reference image at index 0 named 'imageTrackingExample' in library 'Assets/Samples/AR Simulation/1.0.2/Getting Started/Assets/ReferenceImageLibraryExample.asset' requires a non-zero width.
    2. UnityEditor.XR.ARKit.ARKitReferenceImageLibraryBuildProcessor+<ResourceGroups>d__0.MoveNext () (at Library/PackageCache/com.unity.xr.arkit@3.1.0-preview.4/Editor/ARKitReferenceImageLibraryBuildProcessor.cs:36)
    3. UnityEditor.XR.ARKit.ARKitReferenceImageLibraryBuildProcessor+Preprocessor.OnPreprocessBuild (UnityEditor.Build.Reporting.BuildReport report) (at Library/PackageCache/com.unity.xr.arkit@3.1.0-preview.4/Editor/ARKitReferenceImageLibraryBuildProcessor.cs:86)
    4. UnityEditor.Build.BuildPipelineInterfaces+<>c__DisplayClass15_0.<OnBuildPreProcess>b__1 (UnityEditor.Build.IPreprocessBuildWithReport bpp) (at <cf7d2c2e83f749e8943578126f9eff4b>:0)
    5. UnityEditor.Build.BuildPipelineInterfaces.InvokeCallbackInterfacesPair[T1,T2] (System.Collections.Generic.List`1[T] oneInterfaces, System.Action`1[T] invocationOne, System.Collections.Generic.List`1[T] twoInterfaces, System.Action`1[T] invocationTwo, System.Boolean exitOnFailure) (at <cf7d2c2e83f749e8943578126f9eff4b>:0)
    6. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
     
    ROBYER1 likes this.
  8. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130

    Hi! Thanks for the feedback! Happy to hear it's useful to you!

    Which ARSimulation version are you currently using? This bug should've been fixed with version 1.0.3-preview.1 released a couple of days ago. Sorry for the inconvenience :)
     
    trnq1ll0 likes this.
  9. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    761
    @trnq1ll0 to add to what marwi said above – you can update AR Simulation directly through Package Manager right now, no need to wait for a new AssetStore release.
     
    trnq1ll0 and marwi like this.
  10. GameArt

    GameArt

    Joined:
    Feb 24, 2013
    Posts:
    86
    This looks awesome fherbst! Exactly what I wanted out of a simulator. I've been building purpose built simulation scenes for my AR apps and it can be a pain. I'm looking forward to checking it out soon :)
    I noticed you mentioned you have meshing support- do you have a video clip or gif of what that looks like? Meshing is particularly painful to simulate right now in editor.
     
  11. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Hello @GameArt I'm glad you like the package. Let us know when you get to try it.
    About meshing: We have plane tracking/detection support, but the meshing manager API is not yet supported (although we look into it and would like to support it eventually).
     
  12. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Hello, I just wanted to let you know that we're currently working on adding proper meshing support. You can follow the progress here: https://github.com/needle-tools/ar-simulation/issues/37

    Please feel free to let us know if you have any feature requests regarding meshing (ideally on github) or if there is anything else missing for you!

    2020-09-17-162631.gif
     

    Attached Files:

  13. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    We recently published version 1.0.4-preview.1 adding support for ARMeshManager for windows and osx

    2020-09-18-162314.gif
     
  14. sskenth

    sskenth

    Joined:
    Sep 23, 2016
    Posts:
    50
    Hi

    I recently bought your asset from the store. I think it has some amazing potential and I'm looking forward to its progress.

    I left a review and mentioned a bug in there, I decided I'd leave a video to show you what I meant.
    You can see that here.
    The 2 issues are in the description and below.

    Please let me know if there's anything I can do to help debug this.

    Thank you.

    -Satwant

    _______YT DESCRIPTION_____
    There are two issues that you can spot in this video. The first is that the plant keeps getting placed even when I'm not tapping on the game window. I would guess it's reacting to the mouse down and placing at 0,0,0. You can see this at the 1min mark.

    The second issue is a little more subtle when I click back into the game window I get some slowdown and freezing. You can see this at the 1min and 20sec mark where I cannot move the game camera around and then it jumps.

    You recommended I download version 1.0.3-preview.1 Although its not shown in the video as I couldn't work out how to enable preview packages as it changed in 2020. I worked it out in the end thankfully. The issue still persists, it is also still there in version 1.0.4-preview.1

    Im using Unity 2020.1.3f and I have a MacOS Catalina 2019
     
    fherbst likes this.
  15. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Hello @sskenth Thanks for getting back at us!

    Thanks for the video, that's great. I'll try to reproduce it tomorrow when I'm back at the office and have access to osx. I assume issue one and two are related.

    Btw if you're using discord feel free to join here: https://discord.gg/U2NVvvd
    It is sometimes easier and faster to communicate
     
  16. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Hi @sskenth

    Do you think you could send me the project you're using in the video? Im trying to reproduce it but unfortunately I havent been able to so far.
    I'll also send you a PM with a change that could potentially fix it for you, would be great if you could try that as well.

    Edit: OK sent you a message
     
    Last edited: Sep 25, 2020
  17. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Hello everyone!

    We just added support for Object Tracking in version 1.0.4-preview.2
    The "Getting Started" samples contain an example scene.

    As always: please let us know if you run into issues or have any feature requests!

    And in case you didnt notice: ARSimulation is currently on SALE with 50% OFF in the AssetStore


    2020-09-27-183124.gif
     
  18. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Version 1.0.4 released fixing bugs with image tracking
     
  19. Burn0815

    Burn0815

    Joined:
    Jul 3, 2019
    Posts:
    3
    Great stuff you are doing! Keep up the good work!
     
    marwi likes this.
  20. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Thanks a lot! And thank you for your nice review!
     
  21. marwi

    marwi

    Joined:
    Aug 13, 2014
    Posts:
    130
    Version 1.0.5-preview.1 is out

    Fixed
    - Issue with creating SimulatedComponents in Editor even if XR Plugin was disabled
    - Issue on osx with input events leaking in scene

    Changed
    - Removed ``Attatching Anchors is not supported yet`` debug log
    - InputHelper.SimulateTouch restricted to GameView window

    Added
    - Movement option to allow using arrow keys for moving around (see ``ProjectSettings/AR Simulation/Input Settings``)
    - Movement option to allow disabling right mouse button for moving around (see ``ProjectSettings/AR Simulation/Input Settings``)
     
  22. karonte

    karonte

    Joined:
    Jun 2, 2013
    Posts:
    38
    Good evening. i'm trying the asset before buy it ( i download it from github). in my project I installed ar foundation, arcore, device simulator, ec... but when i press PLAY i see only purple...any ideas?
    Regards.
     
  23. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    761
    @karonte "I see only purple" is usually a good indicator of something with renderpipelines being wrongly configured, e.g. the scene you're loading uses URP and the graphics pipeline is set to built-in or vice versa.
    Can you try in an empty project first please? Thanks!
     
    karonte likes this.
  24. Cascho01

    Cascho01

    Joined:
    Mar 19, 2010
    Posts:
    1,219
unityunity