AR Foundation Remote | Test and debug your AR project in the Editor

Unfortunately, only one ARPoseDriver is updated, the one that is sitting on ARCamera:
https://forum.unity.com/threads/ar-...ject-in-the-editor.898433/page-8#post-6727696

You can implement your custom pose driver by using the ARFoundationRemote.Runtime.InputTracking.
GetNodeStates() and selecting the node with type XRNode.CenterEye.
https://forum.unity.com/threads/ar-...ject-in-the-editor.898433/page-7#post-6561910

 

Thanks for the stack trace! The issue occurs if 'Keep Texture at Runtime' setting is disabled in an image reference library.

3.0.1 is only the minimum version dependency. The plugin works across all versions of AR Foundation starting from 3.0.1 and ending with 4.1.1.

 

Hello.

I am using AR Foundation Remote (4.8.7) with image-tracking. It works, but sometimes i get this exception (see below) in the Companion app. Where "imagename" is the name of the image in the ReferenceImageLibrary.

I did not figure out when this happens and how to make it work again after this happens - reinstalling the companion app and restarting unity did not help.

Any idea how to make it work again?

imagename image already added

UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)

UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:LogError(Object)

ARFoundationRemote.Runtime.<addImage>d__16:MoveNext()

ARFoundationRemote.Runtime.<setNewImageLibraryCor>d__11:MoveNext()

ARFoundationRemote.Runtime.<countCoroutinesNumber>d__8:MoveNext()

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

UnityEngine.MonoBehaviour:StartCoroutineManaged2(IEnumerator)

UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)

ARFoundationRemote.Runtime.DontDestroyOnLoadSingleton:addCoroutine(IEnumerator, String)

ARFoundationRemote.Runtime.DontDestroyOnLoadSingleton:AddCoroutine(IEnumerator, String)

ARFoundationRemote.Runtime.ImageTrackingSubsystemSender:trySetNewImageLibrary(EditorToPlayerMessage)

ARFoundationRemote.Runtime.ImageTrackingSubsystemSender:EditorMessageReceived(EditorToPlayerMessage)

ARFoundationRemote.Runtime.Sender:editorMessageReceived(EditorToPlayerMessage)

System.Action`1:Invoke(T)

ARFoundationRemote.Runtime.TelepathyConnection`2:Update()

thanks

 

Could you please send me your reference image library in PM or via email? You can export it by pressing the right mouse button on it, then checking the 'Include dependencies' option.

Could you please also tell me your setup?
1. Unity version. Windows or macOS?
2. Packages/manifest.json file.
3. Your AR device and its OS version.

 

This is almost golden but when I run it in the beginning it's fine then all of a sudden the camera turns pink

AR Foundation Editor Remote 4.8.11
Unity 2019.4.18f1
URP 7.5.2
AR Foundation 4.0.9
ARKit XR Plugin 4.0.9
MacOS 11.1
iPhone 12 Pro Max iOS 14.2.1

 

I'm currently investigating the issue with URP, the plugin sometimes doesn't work with some URP setups.
Could you please send me a reproducible project so I can fix the issue quicker?

 

I cannot connect to the ARFoundation Remote.

I get error: please don't call blocking methods while AR Companion is not connected. I also get another error regarding a ARFoundationRemote Prefab. I've attached the Debug.Log error messages below.

Your app currently doesn't work at all for me. It just shows a black screen and doesn't connect to my remote. I tried reinstalling, updating, enabling XR Plugin, ensuring correct IP Address, etc.

How can I fix this?

AR Foundation Editor Remote 4.8.11
Unity 2019.4.0f1
AR Foundation 3.0.1
ARCore XR Plugin 3.0.1
Photon PUN 2
Firebase Unity 7.0.2
Windows 10
Pixel 2

 

1. Do example scenes work for you?
2. Could you please make a small reproducible project for me so I can fix the bug quicker?

 

I get the same error about the image already being added and no overlay is shown in the ar companion app (building the app works fine). we work with a dynamic image library generated via webrequests. the ReferenceImageLibrary at start (set in Unity editor) only contains 1 dummy image of which specify size and keep texture at runtime are activated as otherwise it didn't seem to work.
I run Unity 2020.1.17f1 with AR Foundation 4.1.1 on macOS and the AR device is iPhone X with iOS 14.3

Also not clear to me is if and how I should add the ReferenceImageLibrary in the ObjectTrackingLibraries of your tool?
Thank you for your help!

 

Do you have a tested list? I would like to know what I should be using for this to work correctly.

 

There are two ways to test Image Tracking using the plugin:

1. Dynamic: doesn't require any setup. Just use any ReferenceImageLibrary or add new images via ScheduleAddImageJob(). The plugin then sends the images via a network to the AR Companion.
2. Embedded: add your image libraries to 'Settings/Embedded Image Libraries' and make a new build of the AR Companion app.

To use Object Tracking, please add all your object reference libraries to 'Assets/Plugins/ARFoundationRemoteInstaller/Resources/ObjectTrackingLibraries' and make a new build of the companion app. There is no way to send object ref libraries via the network, so they should be embedded inside the companion app.

 

Thanks for sending me the reproducible project.
I hope I can fix the issue at the beginning of the next week.

 

Hi I'm using AR Foundation 4.1.3, ARcore XR plugin 4.1.3, Android
and using AR Plane Manager
I got this error in the ARcompanion app. I can't see the captured video in the unity editor's game scene. only augmented objects appear in the game scene.
occurred error img link

Exception: AssertionException: Assertion Failure. Values are not equal
Expected: R8G8B8A8_UNorm == R8G8B8A8_SRGB.......
...

please help me

 

Thanks for explaining. It was my fault, I tried to add the ImageLibrary to the Object Tracking Libraries instead of the Embedded Image Libraries in the Settings File. Unfortunately it doesn't make a difference whether I add the library there or not, still get the error

Code (CSharp):

1. Error: test-jpg image already added // test-jpg is the file name of the image
2. ARFoundationRemote.Runtime.<addImage>d__16.MoveNext()

any hints what the reason for that may be? I get the error about 10 seconds after all image trigger files are loaded

 

Please update to the latest version of the plugin, this issue was fixed in version 4.8.7.

 

I'm currently investigating the 'image already added' issue, and I hope I can fix it in few days.

 

I also asked this! Alas, it seems only one instance of AR Pose Driver (the one attached to ARCamera) is updated by the plugin. So if you want multiple objects to track the device pose you'll have to nest within that or copy the transform (& camera if applicable) properties by script.

 

I tried to update the plugin several times today, but somehow it always installs the old version again. i proceeded as described in the FAQ (deinstall, remove folder, import again). the weird thing: in the package manager under 'my assets' the correct version (4.8.11) shows. but then after import, under 'in project' it always shows the old version (4.7.11). any hints how to solve this?

 

This is the bug with Package Manager:
https://issuetracker.unity3d.com/is...91.1491653266.1610358292-869630540.1608749981

The workaround is to clear the Asset Store cache folder OR wait for 1 minute:
https://forum.unity.com/threads/the....1610358292-869630540.1608749981#post-6755947

In my experience, opening the asset store web page, then pressing the 'Open in Unity' button also helps.

 

Thanks for your detailed answer. Unfortunately both workarounds did not work for me. so I'll wait till the fix is released or till you release another update

 

@halinc Huge thanks for sending me the reproducible project! I was able to fix the 'image already added' bug.
@i-n-g-o Please check your PM for an unreleased version with a fix.

 

runs fine in 2019, but when I try to install it in 2020 (same machine) it says AR Foundation Editor Remote: please re-import the plugin or buy the additional license if you're trying to install the plugin on different development machine.

 

The plugin consists of two folders: installer and source code. This separation is required for the XR Plugin system to see the plugin. Let me assume that you've copied only the installer folder to your Unity 2020 project. Please re-import the plugin from the Asset Store to fix the issue.

 

Good evening All. i cannot compile the app for my ar device...

Failed to generated ARCore reference image library 'ReferenceImageLibrary'
UnityEngine.GUIUtilityrocessEvent(Int32, IntPtr)

I don't know why?! my first try was ok. Unity 2019.4.15f1 with ArFoundation 3.1. Now I updated my project to ArFoundation 4.1( arcore 4.10, ec...) but i cannot compile. And the old compile seems not to work because i cannot see the camere in my unity project.
Thanks.

 

Does your project compile without the plugin? Or the compilation fails only after the plugin installation?

 

It compiles. But after the upgrade of the plugin no. But I need to test real-time for a quick development .
To investigate, I created a NEW unity project, installed ARfoundation 4.1.3 +Arcore 4.1.3 + arkit 4.1.3 + ArRemote 4.8.11 but compile is always "Failed to generated ARCore reference image library 'ReferenceImageLibrary'
UnityEditor.BuildPipeline:BuildPlayer(String[], String, BuildTarget, BuildOptions)".

My option is to use the old "installed" arcompanion app but I need to upgrade my ar foundation because I need a better light estimation. please help. i'm in deadline....

 

Please delete the example image reference library located at 'Assets/Plugins/ARFoundationRemoteInstaller/Assets/Images/ReferenceImageLibrary.asset' and try again.
Could you please also send me all error logs in PM or via email?

 

FYI for anyone having trouble connecting with an iPhone via USB, go to System Prefs > Sharing and make sure Internet Sharing > iPhone USB is checked. After that in your Network settings, select iPhone USB and you should see "iPhone USB is currently active and has the IP address 172.20.10.4." and now that IP should show up in the AR Companion app.

 

@KirillKuzyk
Thank you for building the ARRemote, so far It helped me much while developing mobile targets in the editor.
My project also has a WebGL target now, which I also test in the editor.

Is there a way to disable ARFoundationRemote when my editor target is set to WebGL?
Otherwise I always get warnings and errors at startup, because of the missing XR Plugins.

 

Yes, please disable the plugin in the XR Plugin Management window.


Please also ensure you're using the latest version of the plugin (4.8.22).

 

@KirillKuzyk does the remote fully support unloading and loading scenes? I'm having issues when transitioning between scenes, with the remote going black or not scanning new planes. I know that ARF 4.x changes the way the ARSession is managed on iOS (pausing rather than stopping the underlying ARKit session when the ARSession object is destroyed), and I'm wondering if it's related. Is this an expected limitation or am I more likely hitting a bug or a problem with my own configuration?

Just updated to ARF 4.1.3 and remote 4.8.22 to verify this is a current problem but also had it with ARF 4.1.1 and a previous remote version.

 

Yes, scene reloads should be supported. I tested scene reloads with the AR Foundation Samples repository, but I may have missed some corner cases.
Could you please create a small reproducible project for me so I can find and fix the bug?

 

Currently occurring in a big, complex project and it's possible that it's my config issue. I'll see if it's possible to reproduce in a simpler project.

 

@KirillKuzyk do you know if if works with the Unity Recorder to save face mesh performances? I'm looking for a solution to record ARCore performance data so I'm wondering what's possible before I get the Remote. Ty!

 

This fixed the issue for me as well. Updated the plugin to latest version in a test project I have (all this contains is the plugin plus a couple of other test files) and it would not build until I deleted the reference image library asset.

 

Yes, the plugin works with Unity Recorded, but there is one caveat. To fix the flickering, please set the Source to 'Target Camera' and set the Camera to 'ActiveCamera' in the Recorder window.
Please keep in mind that camera video quality is limited. In my tests, I get around 15-20 fps with 1/3 resolution on top-end devices.

 

Been using this plugin on and off for a while now, and it's a great time saver! Just wanted to give a heads up that you can connect to iOS devices over USB on Windows if you have iTunes installed, and the iproxy command line utility from libimobiledevice by forwarding the TCP port over USB. This may be worth integrating directly into the asset

 

Quick feature request: expose the playerResolution in CameraSubsystemReceiver -- would be nice to be able to have an editor script that resizes the game view to match the device resolution when the orientation changes.

 

@KirillKuzyk could you please add support for ARCore Extensions for AR Foundation.

There are two specific features we are looking for in ARCore, namely:
- ARCloudAnchor, an ARCore Extensions object that provides a similar service to AR Foundation's ARAnchor as an anchor for game objects in your scene.
- ARRecordingManager, Provides access to session recording functionality.

Thanks in advace.

 

Yes, I have Google Cloud Anchors on my roadmap.
Also, I'm planning to add the ability to record sessions in the future. But this feature will not be based on the ARCore's ARRecordingManager and will work with any AR Foundation platform.

 

Is there any way the iOS app can support being backgrounded? Every time I pause the ARFoundationEditorRemote iOS app and then resume it I get a warning that it needs to be restarted as it thinks there is more than one instance running? Minor, but frustrating.

 

MeshingExample nothing happend,
ARKit MeshClassificationExample nothing happend too.
other scenes funtion are normal.
AR Foundation Remote 4.8.22
AR Foundation 4.1.5
ARkit XR plugin 4.1.5

 

Thanks for reporting the issue. I think I know how to fix it.

Could you please tell me what iOS device do you use? Meshing is currently supported only on iPad Pro 2020 and iPhone 12 Pro/Pro Max.

 

I use ipad pro 2020

 

Great, thanks!!

 

Hmm, it should work. Are there any errors happening in the Editor or in the companion app?
Meshing starts to work on iOS with delay, it's typically 4 seconds.

 

Done! Please update to version 4.9.3, I added an event you can subscribe to and listen to all the data AR companion sends to the Editor. Here is an example of receiving the resolution and orientation, but you can basically get access to all other data too:

Code (CSharp):

1. using UnityEngine;
2.  
3.  
4. public class CompanionAppDataReceiver : MonoBehaviour {
5.     void Awake() {
6.         #if UNITY_EDITOR
7.         ARFoundationRemote.RuntimeEditor.Receiver.onPlayerMessageReceived += data => {
8.             var resolution = data.cameraData?.screenResolution;
9.             if (resolution.HasValue) {
10.                 Debug.Log($"Editor received the resolution of your AR device: {resolution.Value.Deserialize()}");
11.             }
12.  
13.             var orientation = data.cameraData?.screenOrientation;
14.             if (orientation.HasValue) {
15.                 Debug.Log($"Editor received the orientation of your AR device: {orientation.Value}");
16.             }
17.         };
18.         #endif
19.     }
20. }
21.  

 

I fixed this issue in the latest release, please update to version 4.9.3.

 

Hi,

I've just updated to the latest version 4.9.3 but now I'm getting compile errors.

"plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(87,37): error CS0122: 'Texture2DSerializable.data' is inaccessible due to its protection level
plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(88,37): error CS0122: 'Texture2DSerializable.compressed' is inaccessible due to its protection level
plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(89,37): error CS0122: 'Texture2DSerializable.format' is inaccessible due to its protection level
plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(90,37): error CS0122: 'Texture2DSerializable.width' is inaccessible due to its protection level
plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(91,37): error CS0122: 'Texture2DSerializable.height' is inaccessible due to its protection level"

Was working fine before the update.

Regards
Ryan

Unity 2020.2.7f1

 

The 'DepthImageSerializer' script was removed in the newer version and now causes conflicts.
Please make sure you've uninstalled the previous plugin version completely before updating to a new version.
While Unity Asset Importer deals with new files and modifications to the old ones fine, it doesn't support file removal.