Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

AR Remote Tool - Alpha Preview

Discussion in 'AR' started by lukasz_pasek, Oct 24, 2018.

Thread Status:
Not open for further replies.
  1. BinaryBanana

    BinaryBanana

    Joined:
    Mar 17, 2014
    Posts:
    81
    If you see white with USB it's probably the same case as mine (see above). It went away when I switched to WiFi. To make WiFi work, your PC and phone has to be on the same network.
     
  2. BinaryBanana

    BinaryBanana

    Joined:
    Mar 17, 2014
    Posts:
    81
    If you need to get access to the camera image while in remote, you can use this hack:

    RemoteScreenComponent rsc = Camera.main.gameObject.GetComponent<RemoteScreenComponent>();
    if (rsc != null)
    {
    Texture2D texture = rsc.m_injectedBackground;
    }
    + make m_injectedBackground public in the script RemoteScreenComponent.

    I am loading texture in the RawImage.

    From what I see on Pixel 2, the resolution is 10% of the original, so instead of 2880x1440 I am getting 288x144 but as it's for debug only, it's more than I need. Works great and saved me a lot of trouble with deploying to device. :)
     
  3. jatreutlein

    jatreutlein

    Joined:
    Dec 14, 2018
    Posts:
    6
    Hi, where did you add this code (in what hierachy and in what script file)?
    Thanks
     
  4. enhawk

    enhawk

    Joined:
    Aug 22, 2013
    Posts:
    833
    @lukasz_pasek

    I'm getting the same problems as the guys above, changed to NET 4.0 to get everything running but Unity tells me the remote is connected even if the app isn't running on the phone.

    I get a green screen saying "waiting for connection" - but only if I launch the app and lock / unlock the phone. Otherwise it's a black screen. On both USB and WiFi

    The game screen shrinks to 10 pixels.

    On Unity 2018.2.20 / iPhoneX / iOS 12 / XCode 10.1
     
    Last edited: Jan 27, 2019
  5. soorya696

    soorya696

    Joined:
    Dec 13, 2018
    Posts:
    71
    Hey guys,
    I have found a YouTube tutorial to do this with out any errors check it out!
     
    Kiteva4 likes this.
  6. KrikeyAdmin

    KrikeyAdmin

    Joined:
    Feb 5, 2018
    Posts:
    29
    I watched through the video.... nothing helpful there for me unfortunately. Still getting a white screen on my iphone 6s. I did notice the "Remote Screen Component" on my Camera that shows up after bringing in the Remote AR package. It has some public fields that are empty, including a Render Texture. Is this perhaps part of my issue? I'm also on the LWRP if that makes any difference....
     

    Attached Files:

  7. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    I'm having white screen problem too.

    Let me describe the problem in detail, hope it will help to fix the issue.
    1. Sometimes, Editor receives the first frame from phone and updates camera location.
    2. Similar problem to @KMarc. 'RemoteScreenComponent' is created with None RenderTexture but populated CompisitionMaterial.
    3. I tried both WiFi and wired connection (using 127.0.0.1 address)
    4. I tried to toggle on/off 'Development build'
    Here is an error from xCode when device tries to start AR session:
    My setup: Unity 2018.3.2f1, xCode 9.4.1, iPhone X running iOS 11.4
    Maybe the problem is in xCode or iOS version, although iOS 11 should support ARKit 1.5.
     
    nir11 likes this.
  8. zerozerozero

    zerozerozero

    Joined:
    Mar 8, 2015
    Posts:
    3
    Thanks for this. Just a heads up - Unity 2018.3.1f1, - I installed the xrRemoting package as described and got an error:

    Library/PackageCache/com.unity.xr.remoting@0.0.1-preview.10/com.unity.xr.remoting/Editor/Providers/DirectConnectionProvider.cs(44,17): error CS0433: The type 'CancellationToken' exists in both 'System.Threading.Tasks.Net35, Version=1.0.0.12675, Culture=neutral, PublicKeyToken=null' and 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

    Looking at the code in DirectConnectionProvider.cs(44,17) the problem is the line
    Code (CSharp):
    1. private CancellationToken ct;
    Since the variable ct doesn't appear to be used anywhere I comment it out and the error is resolved.
     
    Alt_VFX likes this.
  9. 1thewitcher

    1thewitcher

    Joined:
    Jul 23, 2013
    Posts:
    10
    Unity 2018.3.4f1
    Got my RemotingApp from the first post.
    Trying to test planar surface detection.
    I did everything like instruction tells me. When the remote session started I see just one first frame on my device which was captured from my camera, tracking is working fine though, but planes are not generating.
     
  10. Deleted User

    Deleted User

    Guest

    Hello,

    @lukasz_pasek
    thanks a lot, this tool is an invaluable time saver!

    I'm having the same Problem as Bodin with Unity 2018.3 when trying to call Raycast on the session origin:
    NullReferenceException: Object reference not set to an instance of an object
    EditorRemoting.MockProxy.RaycastHandler [...]

    Did you ever solve the issue and how?
     
    Last edited by a moderator: Feb 12, 2019
  11. edwon

    edwon

    Joined:
    Apr 24, 2011
    Posts:
    266
    I have the exact same issue. One question for Unity, why the F isn't this working? Also why has nobody from Unity responded to this for weeks?
     
    novaVision likes this.
  12. Deleted User

    Deleted User

    Guest

    @lukasz_pasek I encountered another problem: When using the Package together with Odin it crashes the Unity Editor. The friendly guys from Sirenix told me that Odin probably is only triggering the crash but not responsible for it: The type that is crashing is one with a many generic parameters, it might be related to that. Also it seems like you're using the DLR via the dynamic keyword which could also be the issue.

    0x000000006FC416C2 (mscorlib) System.Object.Dynamic_Sirenix.OdinInspector.Editor.PersistentContextCache.ContextKey_string, string, string, OdinDrawerExtensions.DrawerStateSignature, string()
    0x000000006F757773 (Unity.XR.ARFoundation) UnityEngine.XR.ARFoundation.ARSubsystemManager.CreateSubsystems()
    0x000000006F755DAB (Unity.XR.ARFoundation) UnityEngine.XR.ARFoundation.ARSession.OnEnable()
     
  13. AbradolfLinkler

    AbradolfLinkler

    Joined:
    Feb 26, 2017
    Posts:
    26
    Doesn't work.
    After building and uploading the remote app I am able to see the "Connected" message in the remote tab - however I get a black screen on the remote app and that's it.

    Running on iOS 12.1.3 iphone 8 plus, Unity 2018.3.4f1
     
  14. snw

    snw

    Joined:
    Mar 13, 2014
    Posts:
    42
    Hi,

    ARRemote currently seems to be pretty broken and unusable. For me, it crashes immediately on iPad 2018 (see here).

    As this makes testing and prototyping a pain, it would be nice if @lukasz_pasek (or somebody else from the team) could give us an update on the development status of ARRemote.

    Thanks.
     
    grubertm and KrikeyAdmin like this.
  15. edhjones

    edhjones

    Joined:
    Oct 23, 2018
    Posts:
    6
    Hello.

    I'm building an app which combines AR stuff with GPS data.
    I have this AR Remoting app, and the Unity Remote 5 app working separately. But it would be super-helpful if I could get GPS and AR data into my editor-running app at the same time.

    Any plans to enhance the official Unity Remote app to support AR?
     
  16. sally-slade

    sally-slade

    Joined:
    Apr 26, 2016
    Posts:
    2
    Hey friends,

    I am having difficulty connecting the Editor's Remote tool to the RemoteApp I built and pushed to my iOS device. The iPhones (I've tried a 7+ and an X) both hang on their green "Waiting for Connection" screen. I've tried using a USB and the localhost address in the Editor tool, as well as using WiFi and the device's IP.

    The Editor tool continually says there was a successful connection, but the RemoteApp does not leave its green "Waiting" screen, nor does the device send data back to the Editor.

    Pinging the devices from the machine is always successful; the devices and the machine are on the same WiFi.

    Any tips?

    Thank you.
     
  17. thorikawa

    thorikawa

    Joined:
    Dec 3, 2013
    Posts:
    25
    I am wondering how is this tool distinguished from ARKitRemote included in Unity ARKit plugin? Is there complete different roadmap for this or maybe merged in the future? Actually, in my use case, this tool is better than ARKitRemote, because I would like to check the AR image on my phone. Current ARKitRemote shows AR image only on PC (Unity Editor), but this tool shows it on the phone as well as PC, which is a great feature.
     
  18. newguy123

    newguy123

    Joined:
    Aug 22, 2018
    Posts:
    1,248
    No post here form the devs so far this year answering any questions or giving updates

    This project dead in the water then?
     
  19. abstractronchris

    abstractronchris

    Joined:
    Nov 1, 2016
    Posts:
    5
    As far as I understand it, this tool is meant to span ARCore and ARKit by using the AR Foundation that Unity has been working on to asbtract away AR functionality cross-platform.
     
  20. abstractronchris

    abstractronchris

    Joined:
    Nov 1, 2016
    Posts:
    5
    wanted to say AR one more time.. AR! :D
     
    Srokaaa likes this.
  21. dimitris_baud

    dimitris_baud

    Joined:
    Jan 15, 2018
    Posts:
    30
    Just adding one more voice here. This is essential for AR development. It looked OK in the GDC Demo, but it doesn't work under any circumstances on 2018.3.x. Looking forward to an update.
     
    grubertm likes this.
  22. Berberian

    Berberian

    Joined:
    Apr 16, 2018
    Posts:
    1
    I've used the remoting feature previously. Good luck to you all. Don't give up.
     
  23. Aaron-Meyers

    Aaron-Meyers

    Joined:
    Dec 8, 2009
    Posts:
    305
    Unity and @lukasz_pasek ... how about giving us an update on this situation? It's been more than three months since we've heard from anyone at Unity on this thread. I'm about to start an AR project and it would be cool to use ARFoundation, but not without a functional remoting tool.
     
  24. dansopanso

    dansopanso

    Joined:
    Jun 14, 2018
    Posts:
    11
    I'm stuck as some of the others here... Can't connect via USB and and via Wifi the Remote Window says I'm connected but on my Phone it just says "Waiting for Connection". Working on a LG G6 with Android 8.0 and Unity 2018.3.8. But it's cool to see that this is in developement... really hope to see this working soon :) keep it up guys
     
  25. GameDevHQ_Support

    GameDevHQ_Support

    Joined:
    Jan 28, 2019
    Posts:
    2
    Unity 2018.3.6f1 - LWRP

    When connected via USB - White Screen
    When connected via wifi: "Waiting for Connection"

    Nothing noted above seems to fix this.
     
  26. Daan-Gijzen

    Daan-Gijzen

    Joined:
    Apr 6, 2017
    Posts:
    4
    I'm using Unity 2019.2.0a9 and I get the following error after adding the "com.unity.xr.remoting" package:

    Library/PackageCache/com.unity.xr.remoting@0.0.1-preview.10/com.unity.xr.mock/Runtime/XRMockCameraExtensions.cs(2,22): error CS0234:
    The type or namespace name 'ARExtensions' does not exist in the namespace 'UnityEngine.XR' (are you missing an assembly reference?)

    Is this related to the 2019 beta? or am I just dumb and do I oversee something obvious?

    (I get 3 errors but all of them are similar to this one)
     
    AvidiaLabs, karsnen and nixedd like this.
  27. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    215
    hey @Daan-Gijzen
    check out https://github.com/Unity-Technologies/arfoundation-samples/issues/108

    In march I heard ->>While it's true the samples are only 2018.3 compatible, there is a version of ARFoundation that will work with 2019.1+:
    • Version 1.1 of ARFoundation will work with 2019.1+
    • The ARCore and ARKit preview packages will work with 2019.1 but not 2019.2a8+. We are working on releasing versions that will.
    hope this helps
    cheers, Sergio
     
  28. BenjaminNinpo

    BenjaminNinpo

    Joined:
    Mar 18, 2019
    Posts:
    1
    Hello,

    I had white screen problems when connecting to my android device on the 2019.1.0f2, I have removed the LWRP and now AR Remote is working.
     
  29. yogi_kenobi

    yogi_kenobi

    Joined:
    May 24, 2017
    Posts:
    23
    I'm on 2019.1.0f2 and using "com.unity.xr.remoting": "0.0.1-preview.10". Trying to compile the RemotingApp on Xcode ends up on multiple (linking) errors like

    Undefined symbol: _EnumerateVideoFormats
    Undefined symbol: _session_ExtractReferenceObject
    Undefined symbol: _SessionSetWorldOrigin
    Undefined symbol: _CapturePixelData
    etc..

    Should I be able to get the app to compile or do I have unsupported configuration at this moment?

    Cheers,
    jokke
     
  30. michaelybecker

    michaelybecker

    Joined:
    Jun 18, 2014
    Posts:
    19
    So, is the remote tool dead in the water? Any update would be appreciated, I wish Unity folks would share timelines and ETAs more transparently...

    Has anyone been able to get *any* sim tool working w/AR[whatever - core, kit, foundation] and LWRP?
     
  31. KnewK

    KnewK

    Joined:
    Sep 2, 2012
    Posts:
    19
    Hi,

    Thanks to all for the posts and advice on this thread. Definitely helped in getting a test scene running with AR Remote. I am running remote with Unity 2018.3.12 and iPhone 6s. AR Remote is a great addition to AR Foundation:
    • Fantastic to see the scene planes and camera movement in the editor.
    • The time saved in turnaround will be huge.
    Some of the things I would like to investigate/improve on:
    • White screen on iPhone (mentioned in other posts) not too much of an issue as I am looking at the editor while waving the iPhone around.
    • Having to rewrite/emulate ray casts and planes hits/detection for touches and projected guide objects.
    • Keen to see mobile touches, ray casts and trackable planes in the editor.
    • The frame rate running in the editor (2-4 fps) is poor compared to running on the iPhone (30+ fps). Also mentioned in other posts.
     
  32. aaaavvvv

    aaaavvvv

    Joined:
    Jan 3, 2017
    Posts:
    7
    Hi - any updates with getting the remote working with LWRP?
     
  33. dimitroff

    dimitroff

    Joined:
    Apr 3, 2013
    Posts:
    131
    How do we load the Remote scene from the package, when Unity is giving an error that a Read-Only scene could not be loaded? Unity version is 2019.1.0f2

    EDIT: Anyway, I was able to copy the package from the Cache directory to my Assets folder and load the scene. But for some reason I could not connect to the Remote app, no matter how I try. I am using iPhone XS, the Editor says I am connected, but the remote app is showing "Waiting for connection..". Any ideas?
     
    Last edited: May 1, 2019
    karsnen and Wazabit like this.
  34. roborovski

    roborovski

    Joined:
    Mar 8, 2016
    Posts:
    1
    @ KnewK what did you do to be able to mock your AR raycasts? I'm thankfully able to get the remote to work with 2019.1, but am running into the same " NullReferenceException: Object reference not set to an instance of an object" error that Bodin and others have run into as well.
     
  35. sonalr

    sonalr

    Joined:
    Jun 1, 2017
    Posts:
    1
    Code (CSharp):
    1.     // Update is called once per frame
    2.     void Update()
    3.     {
    4. #if UNITY_EDITOR
    5.     if (Input.GetMouseButtonDown(0))
    6.     {
    7.    
    8.         Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
    9.         int layerMask = 1 << LayerMask.NameToLayer("ARGameObject");
    10.  
    11.         RaycastHit hit;
    12.  
    13.         if (Physics.Raycast(ray, out hit, 1000f, layerMask))
    14.         {
    15.             if (spawnedObject == null)
    16.             {
    17.                 spawnedObject = Instantiate(m_PlacedPrefab, hit.point, Quaternion.identity);
    18.             }
    19.             else
    20.             {
    21.                 spawnedObject.transform.position = hit.point;
    22.             }            
    23.         }
    24.      
    25.      
    26.     }
    27. #else
    28.         if (Input.touchCount == 0)
    29.             return;
    30.         var touch = Input.GetTouch(0);
    31.  
    32.         if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
    33.         {
    34.             // Raycast hits are sorted by distance, so the first one will be closest hit.
    35.             var hitPose = s_Hits[0].pose;
    36.  
    37.             if (spawnedObject == null)
    38.             {
    39.                 spawnedObject = Instantiate(m_PlacedPrefab, hitPose.position, hitPose.rotation);
    40.             }
    41.             else
    42.             {
    43.                 spawnedObject.transform.position = hitPose.position;
    44.             }
    45.             arPlaneManager.enabled = false;
    46.         }
    47. #endif
    48.     }
    @roborovski This is my workaround for mocking raycast touch hit on the remote. I used my code from previous version of arkit and arcore versions. Add the ARPlane to the "ARGameObject" Layer and this should work.
     
    Last edited: May 6, 2019
    Tarrag and KnewK like this.
  36. Wazabit

    Wazabit

    Joined:
    Mar 10, 2017
    Posts:
    1
    same problem here. any help would be very appreciated. thanks in advance.
     
  37. nandodrw

    nandodrw

    Joined:
    Sep 12, 2017
    Posts:
    1
    This is a great tool! thanks for it.
    Hopefully you guys will be able to integrate this tool with Unity Remote to access to other phone's hardware.
     
  38. KnewK

    KnewK

    Joined:
    Sep 2, 2012
    Posts:
    19
    I drag the scene from the Packages folder into a Scene folder under Assets. Open the RemoteApp scene from Assets/Scene, open Build Setting and Add Open Scene.

    Although just rebuilt the RemoteApp to the iPhone and doesn't want to connect now. Hmmmmm.
     
  39. jddivine

    jddivine

    Joined:
    May 8, 2019
    Posts:
    3
    Is this still the package to download as of May, 2019? I can't find any information to get ARKit remote onto my iphone and using it.
     
  40. jddivine

    jddivine

    Joined:
    May 8, 2019
    Posts:
    3
    Does anyone know how to solve the errors for error CS0234: the type or namespace does not exist?
     
    novaVision likes this.
  41. Unlink

    Unlink

    Joined:
    Jul 23, 2013
    Posts:
    2
    My situation of white screen is solved by that selecting Device type to "Generic XR Remote" of Tracked Pose Driver component in AR Camera GameObject.
     
  42. shawww

    shawww

    Joined:
    Sep 30, 2014
    Posts:
    43
    @jddivine If you take the assets out of the package, bring them into your project directly and delete the .asmdef file, that should fix your namespace issues until they've been resolved by Unity.
     
  43. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    215
    I built ARRemote 0.0.1-preview.10 on 2019.1.0.f2 with lwrp 5.6.1 in it to be consistent with the scene i'd run. I then run a scene in 2019.1.0.f2 ARFoundation lwrp. It connects over usb, wifi doesn't detect it. I get white screen on the phone, and I could work with a white screen but if I move the phone around it doesn't detect any planes I can see in the hierarchy/editor or react to touches. Changing device type to Generic XR Remote didn't work for me. @KnewK you detect planes with white screen?
     
  44. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    515
    Same question here.... have no clue
     
  45. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    515
    How did you solve mock assembly reference issue mentioned here AR Remote Tool - Alpha Preview ?
     
    Last edited: May 21, 2019
  46. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    215
    Hey @novaVision , I tried following the link but it's broken, what issue? In my case console didn't report any errors or any assembly issues.
     
  47. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    515
    Here is full error message:
    Code (CSharp):
    1. Library/PackageCache/com.unity.xr.remoting@0.0.1-preview.10/com.unity.xr.mock/Runtime/XRMockCameraExtensions.cs(2,22): error CS0234: The type or namespace name 'ARExtensions' does not exist in the namespace 'UnityEngine.XR' (are you missing an assembly reference?)
    I get it just after installing remote package adding
    Code (CSharp):
    1. "com.unity.xr.remoting": "0.0.1-preview.10"
    to manifest.json. Everything worked fine on last ARFoundation 1.1 and Unity 2018.3
    Tried to reinstall package/s, remove cached library, but had no luck
     
  48. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    215
    Hey @novaVision , I didn't get this error so hard for me to tell. Just in case it helps this is what I did: I created a new project and used the instructions to build the AR Remote project https://forum.unity.com/threads/ar-remote-tool-alpha-preview.573229/
    Here are my packages in this project:
    upload_2019-5-21_12-47-47.png

    I then added "com.unity.xr.remoting": "0.0.1-preview.10" in a lwrp project i'm working on, here are the packages:
    upload_2019-5-21_12-51-11.png

    I can connect the remote app to this project i'm working on via USB and I don't get any error. But it's of no use because I get a white screen (which would be ok to debug) but the mobile (iphone i6) doesn't detect any planes, nothing changes in the hierarchy.

    Hope this helps
     
  49. novaVision

    novaVision

    Joined:
    Nov 9, 2014
    Posts:
    515
    oh.. we are on a different page. My issue related to new ARFoundation 2.1.0 package. Seems the only way to wait until Unity update remote preview package
     
  50. aaaavvvv

    aaaavvvv

    Joined:
    Jan 3, 2017
    Posts:
    7
    I am also having this issue :eek: @lukasz_pasek do you know when a update for Remote will be released that's compatible with ARFoundation 2.1.0 ?
     
    karsnen, chantey and linojon like this.
Thread Status:
Not open for further replies.