AR Remote Tool - Alpha Preview

If you see white with USB it's probably the same case as mine (see above). It went away when I switched to WiFi. To make WiFi work, your PC and phone has to be on the same network.

 

If you need to get access to the camera image while in remote, you can use this hack:

RemoteScreenComponent rsc = Camera.main.gameObject.GetComponent<RemoteScreenComponent>();
if (rsc != null)
{
Texture2D texture = rsc.m_injectedBackground;
}
+ make m_injectedBackground public in the script RemoteScreenComponent.

I am loading texture in the RawImage.

From what I see on Pixel 2, the resolution is 10% of the original, so instead of 2880x1440 I am getting 288x144 but as it's for debug only, it's more than I need. Works great and saved me a lot of trouble with deploying to device.

 

Hi, where did you add this code (in what hierachy and in what script file)?
Thanks

 

@lukasz_pasek

I'm getting the same problems as the guys above, changed to NET 4.0 to get everything running but Unity tells me the remote is connected even if the app isn't running on the phone.

I get a green screen saying "waiting for connection" - but only if I launch the app and lock / unlock the phone. Otherwise it's a black screen. On both USB and WiFi

The game screen shrinks to 10 pixels.

On Unity 2018.2.20 / iPhoneX / iOS 12 / XCode 10.1

 

Hey guys,
I have found a YouTube tutorial to do this with out any errors check it out!

 

I watched through the video.... nothing helpful there for me unfortunately. Still getting a white screen on my iphone 6s. I did notice the "Remote Screen Component" on my Camera that shows up after bringing in the Remote AR package. It has some public fields that are empty, including a Render Texture. Is this perhaps part of my issue? I'm also on the LWRP if that makes any difference....

 

I'm having white screen problem too.

Let me describe the problem in detail, hope it will help to fix the issue.

1. Sometimes, Editor receives the first frame from phone and updates camera location.
2. Similar problem to @KMarc. 'RemoteScreenComponent' is created with None RenderTexture but populated CompisitionMaterial.
3. I tried both WiFi and wired connection (using 127.0.0.1 address)
4. I tried to toggle on/off 'Development build'

Here is an error from xCode when device tries to start AR session:

My setup: Unity 2018.3.2f1, xCode 9.4.1, iPhone X running iOS 11.4
Maybe the problem is in xCode or iOS version, although iOS 11 should support ARKit 1.5.

 

Thanks for this. Just a heads up - Unity 2018.3.1f1, - I installed the xrRemoting package as described and got an error:

Library/PackageCache/com.unity.xr.remoting@0.0.1-preview.10/com.unity.xr.remoting/Editor/Providers/DirectConnectionProvider.cs(44,17): error CS0433: The type 'CancellationToken' exists in both 'System.Threading.Tasks.Net35, Version=1.0.0.12675, Culture=neutral, PublicKeyToken=null' and 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

Looking at the code in DirectConnectionProvider.cs(44,17) the problem is the line

Code (CSharp):

1. private CancellationToken ct;

Since the variable ct doesn't appear to be used anywhere I comment it out and the error is resolved.

 

Unity 2018.3.4f1
Got my RemotingApp from the first post.
Trying to test planar surface detection.
I did everything like instruction tells me. When the remote session started I see just one first frame on my device which was captured from my camera, tracking is working fine though, but planes are not generating.

 

Hello,

@lukasz_pasek
thanks a lot, this tool is an invaluable time saver!

I'm having the same Problem as Bodin with Unity 2018.3 when trying to call Raycast on the session origin:
NullReferenceException: Object reference not set to an instance of an object
EditorRemoting.MockProxy.RaycastHandler [...]

Did you ever solve the issue and how?

 

I have the exact same issue. One question for Unity, why the F isn't this working? Also why has nobody from Unity responded to this for weeks?

 

@lukasz_pasek I encountered another problem: When using the Package together with Odin it crashes the Unity Editor. The friendly guys from Sirenix told me that Odin probably is only triggering the crash but not responsible for it: The type that is crashing is one with a many generic parameters, it might be related to that. Also it seems like you're using the DLR via the dynamic keyword which could also be the issue.

0x000000006FC416C2 (mscorlib) System.Object.Dynamic_Sirenix.OdinInspector.Editor.PersistentContextCache.ContextKey_string, string, string, OdinDrawerExtensions.DrawerStateSignature, string()
0x000000006F757773 (Unity.XR.ARFoundation) UnityEngine.XR.ARFoundation.ARSubsystemManager.CreateSubsystems()
0x000000006F755DAB (Unity.XR.ARFoundation) UnityEngine.XR.ARFoundation.ARSession.OnEnable()

 

Doesn't work.
After building and uploading the remote app I am able to see the "Connected" message in the remote tab - however I get a black screen on the remote app and that's it.

Running on iOS 12.1.3 iphone 8 plus, Unity 2018.3.4f1

 

Hi,

ARRemote currently seems to be pretty broken and unusable. For me, it crashes immediately on iPad 2018 (see here).

As this makes testing and prototyping a pain, it would be nice if @lukasz_pasek (or somebody else from the team) could give us an update on the development status of ARRemote.

Thanks.

 

Hello.

I'm building an app which combines AR stuff with GPS data.
I have this AR Remoting app, and the Unity Remote 5 app working separately. But it would be super-helpful if I could get GPS and AR data into my editor-running app at the same time.

Any plans to enhance the official Unity Remote app to support AR?

 

Hey friends,

I am having difficulty connecting the Editor's Remote tool to the RemoteApp I built and pushed to my iOS device. The iPhones (I've tried a 7+ and an X) both hang on their green "Waiting for Connection" screen. I've tried using a USB and the localhost address in the Editor tool, as well as using WiFi and the device's IP.

The Editor tool continually says there was a successful connection, but the RemoteApp does not leave its green "Waiting" screen, nor does the device send data back to the Editor.

Pinging the devices from the machine is always successful; the devices and the machine are on the same WiFi.

Any tips?

Thank you.

 

I am wondering how is this tool distinguished from ARKitRemote included in Unity ARKit plugin? Is there complete different roadmap for this or maybe merged in the future? Actually, in my use case, this tool is better than ARKitRemote, because I would like to check the AR image on my phone. Current ARKitRemote shows AR image only on PC (Unity Editor), but this tool shows it on the phone as well as PC, which is a great feature.

 

No post here form the devs so far this year answering any questions or giving updates

This project dead in the water then?

 

As far as I understand it, this tool is meant to span ARCore and ARKit by using the AR Foundation that Unity has been working on to asbtract away AR functionality cross-platform.

 

wanted to say AR one more time.. AR!

 

Just adding one more voice here. This is essential for AR development. It looked OK in the GDC Demo, but it doesn't work under any circumstances on 2018.3.x. Looking forward to an update.

 

I've used the remoting feature previously. Good luck to you all. Don't give up.

 

Unity and @lukasz_pasek ... how about giving us an update on this situation? It's been more than three months since we've heard from anyone at Unity on this thread. I'm about to start an AR project and it would be cool to use ARFoundation, but not without a functional remoting tool.

 

I'm stuck as some of the others here... Can't connect via USB and and via Wifi the Remote Window says I'm connected but on my Phone it just says "Waiting for Connection". Working on a LG G6 with Android 8.0 and Unity 2018.3.8. But it's cool to see that this is in developement... really hope to see this working soon keep it up guys

 

Unity 2018.3.6f1 - LWRP

When connected via USB - White Screen
When connected via wifi: "Waiting for Connection"

Nothing noted above seems to fix this.

 

I'm using Unity 2019.2.0a9 and I get the following error after adding the "com.unity.xr.remoting" package:

Library/PackageCache/com.unity.xr.remoting@0.0.1-preview.10/com.unity.xr.mock/Runtime/XRMockCameraExtensions.cs(2,22): error CS0234:
The type or namespace name 'ARExtensions' does not exist in the namespace 'UnityEngine.XR' (are you missing an assembly reference?)

Is this related to the 2019 beta? or am I just dumb and do I oversee something obvious?

(I get 3 errors but all of them are similar to this one)

 

hey @Daan-Gijzen
check out https://github.com/Unity-Technologies/arfoundation-samples/issues/108

In march I heard ->>While it's true the samples are only 2018.3 compatible, there is a version of ARFoundation that will work with 2019.1+:

• Version 1.1 of ARFoundation will work with 2019.1+
• The ARCore and ARKit preview packages will work with 2019.1 but not 2019.2a8+. We are working on releasing versions that will.

hope this helps
cheers, Sergio

 

Hello,

I had white screen problems when connecting to my android device on the 2019.1.0f2, I have removed the LWRP and now AR Remote is working.

 

I'm on 2019.1.0f2 and using "com.unity.xr.remoting": "0.0.1-preview.10". Trying to compile the RemotingApp on Xcode ends up on multiple (linking) errors like

Undefined symbol: _EnumerateVideoFormats
Undefined symbol: _session_ExtractReferenceObject
Undefined symbol: _SessionSetWorldOrigin
Undefined symbol: _CapturePixelData
etc..

Should I be able to get the app to compile or do I have unsupported configuration at this moment?

Cheers,
jokke

 

So, is the remote tool dead in the water? Any update would be appreciated, I wish Unity folks would share timelines and ETAs more transparently...

Has anyone been able to get *any* sim tool working w/AR[whatever - core, kit, foundation] and LWRP?

 

Hi,

Thanks to all for the posts and advice on this thread. Definitely helped in getting a test scene running with AR Remote. I am running remote with Unity 2018.3.12 and iPhone 6s. AR Remote is a great addition to AR Foundation:

• Fantastic to see the scene planes and camera movement in the editor.
• The time saved in turnaround will be huge.

Some of the things I would like to investigate/improve on:

• White screen on iPhone (mentioned in other posts) not too much of an issue as I am looking at the editor while waving the iPhone around.
• Having to rewrite/emulate ray casts and planes hits/detection for touches and projected guide objects.
• Keen to see mobile touches, ray casts and trackable planes in the editor.
• The frame rate running in the editor (2-4 fps) is poor compared to running on the iPhone (30+ fps). Also mentioned in other posts.

 

Hi - any updates with getting the remote working with LWRP?

 

How do we load the Remote scene from the package, when Unity is giving an error that a Read-Only scene could not be loaded? Unity version is 2019.1.0f2

EDIT: Anyway, I was able to copy the package from the Cache directory to my Assets folder and load the scene. But for some reason I could not connect to the Remote app, no matter how I try. I am using iPhone XS, the Editor says I am connected, but the remote app is showing "Waiting for connection..". Any ideas?

 

@ KnewK what did you do to be able to mock your AR raycasts? I'm thankfully able to get the remote to work with 2019.1, but am running into the same " NullReferenceException: Object reference not set to an instance of an object" error that Bodin and others have run into as well.

 

Code (CSharp):

1.     // Update is called once per frame
2.     void Update()
3.     {
4. #if UNITY_EDITOR
5.     if (Input.GetMouseButtonDown(0))
6.     {
7.    
8.         Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
9.         int layerMask = 1 << LayerMask.NameToLayer("ARGameObject");
10.  
11.         RaycastHit hit;
12.  
13.         if (Physics.Raycast(ray, out hit, 1000f, layerMask))
14.         {
15.             if (spawnedObject == null)
16.             {
17.                 spawnedObject = Instantiate(m_PlacedPrefab, hit.point, Quaternion.identity);
18.             }
19.             else
20.             {
21.                 spawnedObject.transform.position = hit.point;
22.             }            
23.         }
24.      
25.      
26.     }
27. #else
28.         if (Input.touchCount == 0)
29.             return;
30.         var touch = Input.GetTouch(0);
31.  
32.         if (m_SessionOrigin.Raycast(touch.position, s_Hits, TrackableType.PlaneWithinPolygon))
33.         {
34.             // Raycast hits are sorted by distance, so the first one will be closest hit.
35.             var hitPose = s_Hits[0].pose;
36.  
37.             if (spawnedObject == null)
38.             {
39.                 spawnedObject = Instantiate(m_PlacedPrefab, hitPose.position, hitPose.rotation);
40.             }
41.             else
42.             {
43.                 spawnedObject.transform.position = hitPose.position;
44.             }
45.             arPlaneManager.enabled = false;
46.         }
47. #endif
48.     }

@roborovski This is my workaround for mocking raycast touch hit on the remote. I used my code from previous version of arkit and arcore versions. Add the ARPlane to the "ARGameObject" Layer and this should work.

 

same problem here. any help would be very appreciated. thanks in advance.

 

This is a great tool! thanks for it.
Hopefully you guys will be able to integrate this tool with Unity Remote to access to other phone's hardware.

 

I drag the scene from the Packages folder into a Scene folder under Assets. Open the RemoteApp scene from Assets/Scene, open Build Setting and Add Open Scene.

Although just rebuilt the RemoteApp to the iPhone and doesn't want to connect now. Hmmmmm.

 

Is this still the package to download as of May, 2019? I can't find any information to get ARKit remote onto my iphone and using it.

 

Does anyone know how to solve the errors for error CS0234: the type or namespace does not exist?

 

My situation of white screen is solved by that selecting Device type to "Generic XR Remote" of Tracked Pose Driver component in AR Camera GameObject.

 

@jddivine If you take the assets out of the package, bring them into your project directly and delete the .asmdef file, that should fix your namespace issues until they've been resolved by Unity.

 

I built ARRemote 0.0.1-preview.10 on 2019.1.0.f2 with lwrp 5.6.1 in it to be consistent with the scene i'd run. I then run a scene in 2019.1.0.f2 ARFoundation lwrp. It connects over usb, wifi doesn't detect it. I get white screen on the phone, and I could work with a white screen but if I move the phone around it doesn't detect any planes I can see in the hierarchy/editor or react to touches. Changing device type to Generic XR Remote didn't work for me. @KnewK you detect planes with white screen?

 

Same question here.... have no clue

 

How did you solve mock assembly reference issue mentioned here AR Remote Tool - Alpha Preview ?

 

Hey @novaVision , I tried following the link but it's broken, what issue? In my case console didn't report any errors or any assembly issues.

 

Here is full error message:

Code (CSharp):

1. Library/PackageCache/com.unity.xr.remoting@0.0.1-preview.10/com.unity.xr.mock/Runtime/XRMockCameraExtensions.cs(2,22): error CS0234: The type or namespace name 'ARExtensions' does not exist in the namespace 'UnityEngine.XR' (are you missing an assembly reference?)

I get it just after installing remote package adding

Code (CSharp):

1. "com.unity.xr.remoting": "0.0.1-preview.10"

to manifest.json. Everything worked fine on last ARFoundation 1.1 and Unity 2018.3
Tried to reinstall package/s, remove cached library, but had no luck

 

Hey @novaVision , I didn't get this error so hard for me to tell. Just in case it helps this is what I did: I created a new project and used the instructions to build the AR Remote project https://forum.unity.com/threads/ar-remote-tool-alpha-preview.573229/
Here are my packages in this project:


I then added "com.unity.xr.remoting": "0.0.1-preview.10" in a lwrp project i'm working on, here are the packages:


I can connect the remote app to this project i'm working on via USB and I don't get any error. But it's of no use because I get a white screen (which would be ok to debug) but the mobile (iphone i6) doesn't detect any planes, nothing changes in the hierarchy.

Hope this helps

 

oh.. we are on a different page. My issue related to new ARFoundation 2.1.0 package. Seems the only way to wait until Unity update remote preview package

 

I am also having this issue @lukasz_pasek do you know when a update for Remote will be released that's compatible with ARFoundation 2.1.0 ?