Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Mock AR Device for in-Editor Simulation

Discussion in 'AR' started by tdmowrer, Aug 24, 2018.

  1. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    We've been working on a new package that allows for in-Editor simulation of ARFoundation via a mock AR device. It's finally ready for you to try.

    I've added a sample scene to the ARFoundation Samples github repo. It's in a separate branch, since it relies on a new package that hasn't been tested extensively. There's a scene called "MockExample" that runs in the Editor. There's an API for supplying data to the mock device, such as planes and feature points.

    This is very early stuff, and we'll be updating it soon. It should allow for much faster iteration within the Editor, so I hope you find it useful!

    -Tim
     
  2. rxmarccall

    rxmarccall

    Joined:
    Oct 13, 2011
    Posts:
    353
    Is this new / experimental package available via the Package Manager? Or is it currently only contained in that specific branch in the sample repo?
     
  3. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    It is a package; the sample repo contains several scripts showing its use. It's not production quality yet, so you have to do some dances with the package manifest to obtain the package (the samples repo is already setup appropriately).
     
  4. rxmarccall

    rxmarccall

    Joined:
    Oct 13, 2011
    Posts:
    353
    Thank you, I was able to get the package in my project following the sample repo's manifest file as an example!
     
  5. Freaking-Pingo

    Freaking-Pingo

    Joined:
    Aug 1, 2012
    Posts:
    310
    I just gave the preview package a go, but I am facing some compile errors for iOS:

    There seems to have sneaked in a compile error, a mistakenly placed 'c', at then end of EditorRemoting.cs #if UNITY_IOS clause. (line 606).

    There is missing an
    Code (CSharp):
    1. using 'UnityEditor.Hardware'
    in order for the EditorRemoting.cs to compile the DevDeviceList
     
    Last edited: Aug 28, 2018
  6. Freaking-Pingo

    Freaking-Pingo

    Joined:
    Aug 1, 2012
    Posts:
    310
    So I am a bit unsure how this actually works. Is the mock just a remote connection to a mobile device using the Unity Remote app? Or is the mock some sort of build-in editor tool that enables you to 'fake' AR within editor?
     
  7. rxmarccall

    rxmarccall

    Joined:
    Oct 13, 2011
    Posts:
    353
    @Freaking-Pingo This is not a "remote" for testing. As your 2nd statement says, it's a in editor "mock" tool for testing AR inside of the Unity editor without having to build to device.
     
  8. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    Thanks for reporting! Will get it fixed asap.
     
  9. MaxXR

    MaxXR

    Joined:
    Jun 18, 2017
    Posts:
    67
    EDIT: ARRig script is called 'Scaler.cs' in the Mockup scene :)
    Looking forward to iOS build so can test :)

    Just encountered iOS built error also.

    Is it possible to get this demo scene because i'm unable to figure out how to rebuilt it from AR Foundation and the mock scene?
    Perhaps i'm stupid but it doesn't seem very intuitive. For eg, demo has ARRig script but sample scene nor Mockup doesn't.
    upload_2018-8-31_19-55-24.png
     
  10. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    The "ARRig" in that screenshot is now called "ARSessionOrigin" and all scenes in the sample repo should have this.

    The mock example also has a "scaler" script (all it does is set the transform's localScale). The only difference between the gdc demo and the mock example is a the content is a blue cube instead of a village scene.
     
  11. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    Also, the iOS build error hah been fixed but you'll need to update the package to get the fix (the repo still points to the older package). Also note that the mock doesn't run on iOS so this is only an error if you happen to set the current build platform to iOS.
     
  12. MaxXR

    MaxXR

    Joined:
    Jun 18, 2017
    Posts:
    67
    Got scaling working in editor (see vid below) but fails when built to device (see 2nd vid, am scaling with finger) Any ideas?

    Working in Unity using simulation mode (this is cool!)


    Not working when built to iPhone:


    Only edit I made was to the Scaler script - adding a method and then connecting the slider to it.
    Code (CSharp):
    1. public class Scaler : MonoBehaviour
    2.     {
    3.         [SerializeField]
    4.         float m_Scale = 1f;
    5.  
    6.         void Update()
    7.         {
    8.             transform.localScale = Vector3.one * m_Scale;
    9.         }
    10.  
    11.         public void AdjustScale(float newScale)
    12.         {
    13.             m_Scale = newScale;
    14.         }
    Are you able to please share your version which works or advise what's going wrong?
    Thanks
     
  13. tonOnWu

    tonOnWu

    Joined:
    Jun 23, 2017
    Posts:
    83
    Hi guys.

    I just tried to work with this, but I got this error when I switch to iOS Devices:

    /Users/tonymunoz/Library/Unity/cache/packages/staging-packages.unity.com/com.unity.xr.mock@0.0.1-preview.6/com.unity.xr.remoting/Editor/EditorRemoting.cs(615,13): error CS1002: ; expected

    I tried to find EditorRemoting but I couldn't.

    Any suggestions?
     

    Attached Files:

  14. rxmarccall

    rxmarccall

    Joined:
    Oct 13, 2011
    Posts:
    353
    @tdmowrer We've noticed some major memory leaks occurring in our app that uses ARFoundation with ARKit. As we've tried to track where this leak is occurring, I'm noticing that if I simply run the MockExample scene in the Unity Editor and leave it running, that the total allocated memory continuously grows and grows. I started at 0.66GB total allocated, and after about 5 minutes was up to 1.17GB.

    Perhaps there is a memory leak in ARFoundation??
     
  15. rxmarccall

    rxmarccall

    Joined:
    Oct 13, 2011
    Posts:
    353
    I had this error when my build platform was set as iOS, you need to have it set as Standalone.
     
    tonOnWu likes this.
  16. tonOnWu

    tonOnWu

    Joined:
    Jun 23, 2017
    Posts:
    83
    Thanks. It seems that this library has a long way to go.
     
    rxmarccall likes this.
  17. djayboy007007

    djayboy007007

    Joined:
    Mar 14, 2015
    Posts:
    5
  18. lukasz_pasek

    lukasz_pasek

    Unity Technologies

    Joined:
    Oct 16, 2017
    Posts:
    22
    @rxmarccall Could you let me know what version of Unity are you currently using ?
     
  19. DanieleFiorenti

    DanieleFiorenti

    Joined:
    Feb 23, 2017
    Posts:
    4
    Hi! I tried to open the ARFoundation Sample project in mock branch, when I open the project (either in Standalone or in Android) I get the following error:

    C:/Users/fiorenti/AppData/Local/Unity/cache/packages/staging-packages.unity.com/com.unity.xr.mock@0.0.1-preview.6/com.unity.xr.remoting/Runtime/RemoteScreenComponent.cs(12,16): error CS0246: The type or namespace name `AsyncGPUReadbackRequest' could not be found. Are you missing `UnityEngine.Experimental.Rendering' using directive?

    Do you have some idea to fix it?
    Thank you.

    Update: I get the same error also in iOS!

    Daniele
     
    Last edited: Oct 11, 2018
  20. lukasz_pasek

    lukasz_pasek

    Unity Technologies

    Joined:
    Oct 16, 2017
    Posts:
    22
  21. DanieleFiorenti

    DanieleFiorenti

    Joined:
    Feb 23, 2017
    Posts:
    4
    I tried with Unity 2018.1.9f2, should I use another version?
    Thank you for your answer!
     
  22. johnsietsma

    johnsietsma

    Unity Technologies

    Joined:
    Jul 1, 2016
    Posts:
    8
    I'm in 2018.3.0b3 and had similar compile issues on the mock branch of ARfoundation-samples. The package manager was missing from the windows menu. I reset the packages and readded ARKit, ARCore, SRP Core and LWRP to get it going again.
     
  23. DanieleFiorenti

    DanieleFiorenti

    Joined:
    Feb 23, 2017
    Posts:
    4
    I made it work with Unity 2018.2.11f1, thank you for your help.
     
  24. pacepiro

    pacepiro

    Joined:
    Jun 1, 2016
    Posts:
    4
    Any news about this?
     
  25. Mal_Duffin

    Mal_Duffin

    Joined:
    Jan 22, 2015
    Posts:
    71
    I just checked out the mock branch, loaded up the MockExample ( from the Simulation folder ), made sure Unity wasn't in iOS platform mode, and got it running.

    Are there any docs to explain what is actually happening, and how this can be used to test an existing scene?

    Cheers!
     
  26. Pavlko

    Pavlko

    Joined:
    Dec 23, 2012
    Posts:
    9
    How did you get the xr.mock package into your project?

    I tried copying the Assets/Simulation folder within the arfoundation-samples "mock" branch, also copied the Packages/manifest.json line containing the xr.mock dependency, but had an error in console:

    Assembly has reference to non-existent assembly 'Unity.XR.Mock' (Assets/ARFoundation-Samples-Simulation/Unity.XR.Mock.Example.asmdef)

    Also the Package Manager shows Xr.mock, with an error saying package cannot be found.

    I can see the xr.mock package in my user folder:
    \AppData\Local\Unity\cache\packages\staging-packages.unity.com\com.unity.xr.mock@0.0.1-preview.14

    I am on Unity 2018.2.13f1
     
  27. Pavlko

    Pavlko

    Joined:
    Dec 23, 2012
    Posts:
    9
  28. Sh0dan

    Sh0dan

    Joined:
    Apr 11, 2013
    Posts:
    1
    Hi,
    In order to fix the iOS issues (that still persist because of mismatching variable names in the Remoting subpackage) I've embedded the package by copying it to my project's Packages folder, but now my editor crashes whenever I try to access native properties/methods such as
    SessionApi.sessionState
    ,
    ARSubsystemManager.systemState
    is always Unsupported and all its subsystems are null.

    How can I get ARFoundation to correctly detect the mock subsystems in the package?
     
  29. Eli1234

    Eli1234

    Joined:
    Jul 13, 2017
    Posts:
    24
    I'm using 2018.3.0b5 and ran into compiler errors in EditorRemoting.cs in the Unity.XR.Mock package.

    FYI, I was able to work around them by digging into my /Library/PackageCache folder, finding the sources for the Unity.XR.Mock package, and modifying line 754 of EditorRemoting.cs. I changed the declaration of the CreateSubsystem function to the following:

    Code (CSharp):
    1. #if UNITY_2018_3_OR_NEWER
    2.         private static TSubsystem CreateSubsystem<TDescriptor, TSubsystem>(List<TDescriptor> descriptors, string id = null)
    3.             where TDescriptor : IntegratedSubsystemDescriptor<TSubsystem>
    4.             where TSubsystem : IntegratedSubsystem<TDescriptor>
    5. #else
    6.         private static TSubsystem CreateSubsystem<TDescriptor, TSubsystem>(List<TDescriptor> descriptors, string id = null)
    7.             where TDescriptor : SubsystemDescriptor<TSubsystem>
    8.             where TSubsystem : Subsystem<TDescriptor>
    9. #endif
    10.  
     
  30. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    515
    As I see there is a <remote> package available now. Is it correct that only 1 package can be used at one time? I thought mock is a simulation and remote is real device usage approaches
     
  31. tdmowrer

    tdmowrer

    Joined:
    Apr 21, 2017
    Posts:
    605
    Remoting relies on the mock - the remoting package feeds data to a mock device running in the Editor.

    The mock mentioned in this thread is only available in a special branch of the samples repo. The remoting package mentioned here uses an embedded version of the mock package, so I believe you are right that you cannot have both packages at the same time.

    Production of the mock was paused until we get full runtime feature parity with ARCore and ARKit, but we plan to continue development on the mock in the new year.
     
  32. czadam

    czadam

    Joined:
    Oct 14, 2012
    Posts:
    15
    Hi! Is there any update on this feature? I still cannot make the mock branch work on iOS, I tried with different unity version applying different fixes mentioned here and still nothing. Can someone give me a unity version, where the package can be compiled (and possibly which fix to apply)?
    Or even better is Unity wants to support this feature properly anytime soon?
     
    Mal_Duffin likes this.
  33. czadam

    czadam

    Joined:
    Oct 14, 2012
    Posts:
    15
    FYI. Since I only wanted to use the mock package and not the remoting, I ended up simply deleting the com.unity.xr.mock.remoting folder from the PackageCache
     
  34. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    324
    We do have feature parity with the ability to move a camera in space, and that would likely solve 9/10 developer use cases for testing.

    There must be a simple way you guys can add support for a camera shim, so we can quickly iterate in the editor?

    I built on an AR app last year, and ARInterface had this built in. It's pretty frustrating to come back, and see a new version which is missing core fundamental functionality, required for any real large scale development work.
     
    Last edited: May 9, 2019
  35. coder89

    coder89

    Joined:
    Oct 26, 2018
    Posts:
    29
    enhawk, Blarp and shawnblais like this.
  36. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    324
    Thanks! Sucks that the developers of ARFoundation seem to not care at all for realworld workflows, but I guess at least it's open source!
     
  37. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    324
    "Production of the mock was paused until we get full runtime feature parity with ARCore and ARKit,"

    Whoever made that decision is wrong. Pausing until we got "full parity" makes no sense when the core functionality of AR already had parity and has been for several years, not to mention there will likely never technically be 'full parity' between ARKit and ARCore.

    we plan to continue development on the mock in the new year.

    5 months and counting.
     
    enhawk, edhjones and Blarp like this.
  38. BidOnDev1

    BidOnDev1

    Joined:
    May 28, 2019
    Posts:
    3


    Does anyone try to construct a raycast using Raycast function from RaycastManager and have problems with intersecting raycasts with virtual AR planes in editor?
     
  39. coder89

    coder89

    Joined:
    Oct 26, 2018
    Posts:
    29
    As I mentioned in that post (and one issue on GitHub) I have implemented only XRSession-, XRPlane- and XRReferencePoint- Subsystem. Therefore Raycasting won't work when using ARRaycastManager - you need to mock XRRaycastManagerSubsystem to get it to work. I haven't had time to implement it and my project does not rely on it. Only ARSession/SessionOrigin, PlaneManager, ReferencePointManager APIs will work at the moment.

    I will happily accept any PR and update the package if someone implements that missing piece. Unfortunately, due to lack of time I won't be able to implement it myself in the coming weeks.
     
  40. BidOnDev1

    BidOnDev1

    Joined:
    May 28, 2019
    Posts:
    3
  41. anficyon

    anficyon

    Joined:
    Jan 30, 2018
    Posts:
    4
    the only solution that is work for me in Unity 2019.1.5.f1 was
    use this versions of arfoundations and arcore

    • AR Foundation preview.6 - 1.1.0
    • Arcore XR Plugin preview.24 - 1.0.0
    and the unity AR remote works!! :) :);)
     

    Attached Files:

  42. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    215
  43. enhawk

    enhawk

    Joined:
    Aug 22, 2013
    Posts:
    833
    This is really cool, thanks. Up until now I've been using my own solution based on the old AR samples (which was more than good enough) - With no remoting for ARFoundation, it's essential to have something like this to speed up development time dramatically.

    I hazard a guess that ARFoundation is pausing the mock while project MARS is taking shape, that also seems not be forth coming so your github repo is most appreciated!
     
  44. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    1,080
    whatever happened to this?
     
    Tarrag likes this.
  45. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    215
  46. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128