Search Unity

AR Foundation Editor Remote | Test and debug your AR project in the Editor

Discussion in 'Assets and Asset Store' started by KirillKuzyk, May 26, 2020.

  1. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Unfortunately, only one ARPoseDriver is updated, the one that is sitting on ARCamera:
    https://forum.unity.com/threads/ar-...ject-in-the-editor.898433/page-8#post-6727696

    You can implement your custom pose driver by using the ARFoundationRemote.Runtime.InputTracking.
    GetNodeStates() and selecting the node with type XRNode.CenterEye.
    https://forum.unity.com/threads/ar-...ject-in-the-editor.898433/page-7#post-6561910
     
  2. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Thanks for the stack trace! The issue occurs if 'Keep Texture at Runtime' setting is disabled in an image reference library.
    3.0.1 is only the minimum version dependency. The plugin works across all versions of AR Foundation starting from 3.0.1 and ending with 4.1.1.
     
  3. i-n-g-o

    i-n-g-o

    Joined:
    Mar 10, 2020
    Posts:
    1
    Hello.

    I am using AR Foundation Remote (4.8.7) with image-tracking. It works, but sometimes i get this exception (see below) in the Companion app. Where "imagename" is the name of the image in the ReferenceImageLibrary.

    I did not figure out when this happens and how to make it work again after this happens - reinstalling the companion app and restarting unity did not help.

    Any idea how to make it work again?


    imagename image already added
    UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
    UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
    UnityEngine.Logger:Log(LogType, Object)
    UnityEngine.Debug:LogError(Object)
    ARFoundationRemote.Runtime.<addImage>d__16:MoveNext()
    ARFoundationRemote.Runtime.<setNewImageLibraryCor>d__11:MoveNext()
    ARFoundationRemote.Runtime.<countCoroutinesNumber>d__8:MoveNext()
    UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
    UnityEngine.MonoBehaviour:StartCoroutineManaged2(IEnumerator)
    UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
    ARFoundationRemote.Runtime.DontDestroyOnLoadSingleton:addCoroutine(IEnumerator, String)
    ARFoundationRemote.Runtime.DontDestroyOnLoadSingleton:AddCoroutine(IEnumerator, String)
    ARFoundationRemote.Runtime.ImageTrackingSubsystemSender:trySetNewImageLibrary(EditorToPlayerMessage)
    ARFoundationRemote.Runtime.ImageTrackingSubsystemSender:EditorMessageReceived(EditorToPlayerMessage)
    ARFoundationRemote.Runtime.Sender:editorMessageReceived(EditorToPlayerMessage)
    System.Action`1:Invoke(T)
    ARFoundationRemote.Runtime.TelepathyConnection`2:Update()


    thanks
     
  4. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Could you please send me your reference image library in PM or via email? You can export it by pressing the right mouse button on it, then checking the 'Include dependencies' option.

    Could you please also tell me your setup?
    1. Unity version. Windows or macOS?
    2. Packages/manifest.json file.
    3. Your AR device and its OS version.
     
  5. Gangsta-Geek

    Gangsta-Geek

    Joined:
    Aug 11, 2012
    Posts:
    32
    This is almost golden but when I run it in the beginning it's fine then all of a sudden the camera turns pink

    AR Foundation Editor Remote 4.8.11
    Unity 2019.4.18f1
    URP 7.5.2
    AR Foundation 4.0.9
    ARKit XR Plugin 4.0.9
    MacOS 11.1
    iPhone 12 Pro Max iOS 14.2.1
     

    Attached Files:

  6. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    I'm currently investigating the issue with URP, the plugin sometimes doesn't work with some URP setups.
    Could you please send me a reproducible project so I can fix the issue quicker?
     
    Gangsta-Geek likes this.
  7. J1wan

    J1wan

    Joined:
    Jun 4, 2020
    Posts:
    11
    I cannot connect to the ARFoundation Remote.

    I get error: please don't call blocking methods while AR Companion is not connected. I also get another error regarding a ARFoundationRemote Prefab. I've attached the Debug.Log error messages below.

    Your app currently doesn't work at all for me. It just shows a black screen and doesn't connect to my remote. I tried reinstalling, updating, enabling XR Plugin, ensuring correct IP Address, etc.

    How can I fix this?

    AR Foundation Editor Remote 4.8.11
    Unity 2019.4.0f1
    AR Foundation 3.0.1
    ARCore XR Plugin 3.0.1
    Photon PUN 2
    Firebase Unity 7.0.2
    Windows 10
    Pixel 2
     

    Attached Files:

  8. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    1. Do example scenes work for you?
    2. Could you please make a small reproducible project for me so I can fix the bug quicker?
     
  9. halinc

    halinc

    Joined:
    Feb 24, 2019
    Posts:
    16
    I get the same error about the image already being added and no overlay is shown in the ar companion app (building the app works fine). we work with a dynamic image library generated via webrequests. the ReferenceImageLibrary at start (set in Unity editor) only contains 1 dummy image of which specify size and keep texture at runtime are activated as otherwise it didn't seem to work.
    I run Unity 2020.1.17f1 with AR Foundation 4.1.1 on macOS and the AR device is iPhone X with iOS 14.3

    Also not clear to me is if and how I should add the ReferenceImageLibrary in the ObjectTrackingLibraries of your tool?
    Thank you for your help!
     
  10. Gangsta-Geek

    Gangsta-Geek

    Joined:
    Aug 11, 2012
    Posts:
    32
    Do you have a tested list? I would like to know what I should be using for this to work correctly.
     
  11. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    There are two ways to test Image Tracking using the plugin:
    1. Dynamic: doesn't require any setup. Just use any ReferenceImageLibrary or add new images via ScheduleAddImageJob(). The plugin then sends the images via a network to the AR Companion.
    2. Embedded: add your image libraries to 'Settings/Embedded Image Libraries' and make a new build of the AR Companion app.
    To use Object Tracking, please add all your object reference libraries to 'Assets/Plugins/ARFoundationRemoteInstaller/Resources/ObjectTrackingLibraries' and make a new build of the companion app. There is no way to send object ref libraries via the network, so they should be embedded inside the companion app.
     
    Gangsta-Geek likes this.
  12. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Thanks for sending me the reproducible project.
    I hope I can fix the issue at the beginning of the next week.
     
  13. mizan15

    mizan15

    Joined:
    Aug 2, 2020
    Posts:
    4
    Hi I'm using AR Foundation 4.1.3, ARcore XR plugin 4.1.3, Android
    and using AR Plane Manager
    I got this error in the ARcompanion app. I can't see the captured video in the unity editor's game scene. only augmented objects appear in the game scene.
    occurred error img link

    Exception: AssertionException: Assertion Failure. Values are not equal
    Expected: R8G8B8A8_UNorm == R8G8B8A8_SRGB.......
    ...

    please help me
     
    Last edited: Jan 24, 2021
  14. halinc

    halinc

    Joined:
    Feb 24, 2019
    Posts:
    16
    Thanks for explaining. It was my fault, I tried to add the ImageLibrary to the Object Tracking Libraries instead of the Embedded Image Libraries in the Settings File. Unfortunately it doesn't make a difference whether I add the library there or not, still get the error
    Code (CSharp):
    1. Error: test-jpg image already added // test-jpg is the file name of the image
    2. ARFoundationRemote.Runtime.<addImage>d__16.MoveNext()
    any hints what the reason for that may be? I get the error about 10 seconds after all image trigger files are loaded
     
  15. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Please update to the latest version of the plugin, this issue was fixed in version 4.8.7.
     
  16. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    I'm currently investigating the 'image already added' issue, and I hope I can fix it in few days.
     
    halinc likes this.
  17. Jason-RT-Bond

    Jason-RT-Bond

    Joined:
    Mar 26, 2012
    Posts:
    89
    I also asked this! Alas, it seems only one instance of AR Pose Driver (the one attached to ARCamera) is updated by the plugin. So if you want multiple objects to track the device pose you'll have to nest within that or copy the transform (& camera if applicable) properties by script.
     
    KirillKuzyk likes this.
  18. halinc

    halinc

    Joined:
    Feb 24, 2019
    Posts:
    16
    I tried to update the plugin several times today, but somehow it always installs the old version again. i proceeded as described in the FAQ (deinstall, remove folder, import again). the weird thing: in the package manager under 'my assets' the correct version (4.8.11) shows. but then after import, under 'in project' it always shows the old version (4.7.11). any hints how to solve this?
     
  19. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    This is the bug with Package Manager:
    https://issuetracker.unity3d.com/is...91.1491653266.1610358292-869630540.1608749981

    The workaround is to clear the Asset Store cache folder OR wait for 1 minute:
    https://forum.unity.com/threads/the....1610358292-869630540.1608749981#post-6755947

    In my experience, opening the asset store web page, then pressing the 'Open in Unity' button also helps.
     
  20. halinc

    halinc

    Joined:
    Feb 24, 2019
    Posts:
    16
    Thanks for your detailed answer. Unfortunately both workarounds did not work for me. so I'll wait till the fix is released or till you release another update :)
     
  21. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    @halinc Huge thanks for sending me the reproducible project! I was able to fix the 'image already added' bug.
    @i-n-g-o Please check your PM for an unreleased version with a fix.
     
  22. MateSteinforth1

    MateSteinforth1

    Joined:
    Jan 28, 2021
    Posts:
    1
    runs fine in 2019, but when I try to install it in 2020 (same machine) it says AR Foundation Editor Remote: please re-import the plugin or buy the additional license if you're trying to install the plugin on different development machine.
     
  23. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    The plugin consists of two folders: installer and source code. This separation is required for the XR Plugin system to see the plugin. Let me assume that you've copied only the installer folder to your Unity 2020 project. Please re-import the plugin from the Asset Store to fix the issue.
     
  24. karonte

    karonte

    Joined:
    Jun 2, 2013
    Posts:
    38
    Good evening All. i cannot compile the app for my ar device...

    Failed to generated ARCore reference image library 'ReferenceImageLibrary'
    UnityEngine.GUIUtility:processEvent(Int32, IntPtr)

    I don't know why?! my first try was ok. Unity 2019.4.15f1 with ArFoundation 3.1. Now I updated my project to ArFoundation 4.1( arcore 4.10, ec...) but i cannot compile. And the old compile seems not to work because i cannot see the camere in my unity project.
    Thanks.
     
  25. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Does your project compile without the plugin? Or the compilation fails only after the plugin installation?
     
  26. karonte

    karonte

    Joined:
    Jun 2, 2013
    Posts:
    38
    It compiles. But after the upgrade of the plugin no. But I need to test real-time for a quick development :(.
    To investigate, I created a NEW unity project, installed ARfoundation 4.1.3 +Arcore 4.1.3 + arkit 4.1.3 + ArRemote 4.8.11 but compile is always "Failed to generated ARCore reference image library 'ReferenceImageLibrary'
    UnityEditor.BuildPipeline:BuildPlayer(String[], String, BuildTarget, BuildOptions)".

    My option is to use the old "installed" arcompanion app but I need to upgrade my ar foundation because I need a better light estimation. please help. i'm in deadline....
     
  27. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Please delete the example image reference library located at 'Assets/Plugins/ARFoundationRemoteInstaller/Assets/Images/ReferenceImageLibrary.asset' and try again.
    Could you please also send me all error logs in PM or via email?
     
    karonte likes this.
  28. rocket5tim

    rocket5tim

    Joined:
    May 19, 2009
    Posts:
    236
    FYI for anyone having trouble connecting with an iPhone via USB, go to System Prefs > Sharing and make sure Internet Sharing > iPhone USB is checked. After that in your Network settings, select iPhone USB and you should see "iPhone USB is currently active and has the IP address 172.20.10.4." and now that IP should show up in the AR Companion app.
     
    KirillKuzyk likes this.
  29. Bersaelor

    Bersaelor

    Joined:
    Oct 8, 2016
    Posts:
    63
    @KirillKuzyk
    Thank you for building the ARRemote, so far It helped me much while developing mobile targets in the editor.
    My project also has a WebGL target now, which I also test in the editor.

    Is there a way to disable ARFoundationRemote when my editor target is set to WebGL?
    Otherwise I always get warnings and errors at startup, because of the missing XR Plugins.
     
    KirillKuzyk likes this.
  30. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Yes, please disable the plugin in the XR Plugin Management window.
    upload_2021-2-9_12-9-31.png

    Please also ensure you're using the latest version of the plugin (4.8.22).
     
    Bersaelor likes this.
  31. Jason-RT-Bond

    Jason-RT-Bond

    Joined:
    Mar 26, 2012
    Posts:
    89
    @KirillKuzyk does the remote fully support unloading and loading scenes? I'm having issues when transitioning between scenes, with the remote going black or not scanning new planes. I know that ARF 4.x changes the way the ARSession is managed on iOS (pausing rather than stopping the underlying ARKit session when the ARSession object is destroyed), and I'm wondering if it's related. Is this an expected limitation or am I more likely hitting a bug or a problem with my own configuration?

    Just updated to ARF 4.1.3 and remote 4.8.22 to verify this is a current problem but also had it with ARF 4.1.1 and a previous remote version.
     
  32. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Yes, scene reloads should be supported. I tested scene reloads with the AR Foundation Samples repository, but I may have missed some corner cases.
    Could you please create a small reproducible project for me so I can find and fix the bug?
     
  33. Jason-RT-Bond

    Jason-RT-Bond

    Joined:
    Mar 26, 2012
    Posts:
    89
    Currently occurring in a big, complex project and it's possible that it's my config issue. I'll see if it's possible to reproduce in a simpler project.
     
    KirillKuzyk likes this.
  34. laplantefx

    laplantefx

    Joined:
    Feb 15, 2021
    Posts:
    1
    @KirillKuzyk do you know if if works with the Unity Recorder to save face mesh performances? I'm looking for a solution to record ARCore performance data so I'm wondering what's possible before I get the Remote. Ty!
     
    KirillKuzyk likes this.
  35. PSST_Adam

    PSST_Adam

    Joined:
    Aug 26, 2020
    Posts:
    15
    This fixed the issue for me as well. Updated the plugin to latest version in a test project I have (all this contains is the plugin plus a couple of other test files) and it would not build until I deleted the reference image library asset.
     
  36. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Yes, the plugin works with Unity Recorded, but there is one caveat. To fix the flickering, please set the Source to 'Target Camera' and set the Camera to 'ActiveCamera' in the Recorder window.
    Please keep in mind that camera video quality is limited. In my tests, I get around 15-20 fps with 1/3 resolution on top-end devices.
     
  37. SoxLIV

    SoxLIV

    Joined:
    Sep 18, 2020
    Posts:
    2
    Been using this plugin on and off for a while now, and it's a great time saver! Just wanted to give a heads up that you can connect to iOS devices over USB on Windows if you have iTunes installed, and the iproxy command line utility from libimobiledevice by forwarding the TCP port over USB. This may be worth integrating directly into the asset
     
    KirillKuzyk likes this.
  38. SoxLIV

    SoxLIV

    Joined:
    Sep 18, 2020
    Posts:
    2
    Quick feature request: expose the playerResolution in CameraSubsystemReceiver -- would be nice to be able to have an editor script that resizes the game view to match the device resolution when the orientation changes.
     
    KirillKuzyk likes this.
  39. WayneVenter

    WayneVenter

    Joined:
    May 8, 2019
    Posts:
    22
    @KirillKuzyk could you please add support for ARCore Extensions for AR Foundation.

    There are two specific features we are looking for in ARCore, namely:
    - ARCloudAnchor, an ARCore Extensions object that provides a similar service to AR Foundation's ARAnchor as an anchor for game objects in your scene.
    - ARRecordingManager, Provides access to session recording functionality.

    Thanks in advace.
     
    KirillKuzyk likes this.
  40. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Yes, I have Google Cloud Anchors on my roadmap.
    Also, I'm planning to add the ability to record sessions in the future. But this feature will not be based on the ARCore's ARRecordingManager and will work with any AR Foundation platform.
     
    PSST_Adam likes this.
  41. PSST_Adam

    PSST_Adam

    Joined:
    Aug 26, 2020
    Posts:
    15
    Is there any way the iOS app can support being backgrounded? Every time I pause the ARFoundationEditorRemote iOS app and then resume it I get a warning that it needs to be restarted as it thinks there is more than one instance running? Minor, but frustrating.
     
    KirillKuzyk likes this.
  42. orangetech

    orangetech

    Joined:
    Sep 30, 2017
    Posts:
    45
    MeshingExample nothing happend,
    ARKit MeshClassificationExample nothing happend too.
    other scenes funtion are normal.
    AR Foundation Remote 4.8.22
    AR Foundation 4.1.5
    ARkit XR plugin 4.1.5
     
  43. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Thanks for reporting the issue. I think I know how to fix it.

    Could you please tell me what iOS device do you use? Meshing is currently supported only on iPad Pro 2020 and iPhone 12 Pro/Pro Max.
     
  44. orangetech

    orangetech

    Joined:
    Sep 30, 2017
    Posts:
    45
    I use ipad pro 2020
     
  45. PSST_Adam

    PSST_Adam

    Joined:
    Aug 26, 2020
    Posts:
    15
    Great, thanks!!
     
  46. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Hmm, it should work. Are there any errors happening in the Editor or in the companion app?
    Meshing starts to work on iOS with delay, it's typically 4 seconds.
     
  47. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    Done! Please update to version 4.9.3, I added an event you can subscribe to and listen to all the data AR companion sends to the Editor. Here is an example of receiving the resolution and orientation, but you can basically get access to all other data too:
    Code (CSharp):
    1. using UnityEngine;
    2.  
    3.  
    4. public class CompanionAppDataReceiver : MonoBehaviour {
    5.     void Awake() {
    6.         #if UNITY_EDITOR
    7.         ARFoundationRemote.RuntimeEditor.Receiver.onPlayerMessageReceived += data => {
    8.             var resolution = data.cameraData?.screenResolution;
    9.             if (resolution.HasValue) {
    10.                 Debug.Log($"Editor received the resolution of your AR device: {resolution.Value.Deserialize()}");
    11.             }
    12.  
    13.             var orientation = data.cameraData?.screenOrientation;
    14.             if (orientation.HasValue) {
    15.                 Debug.Log($"Editor received the orientation of your AR device: {orientation.Value}");
    16.             }
    17.         };
    18.         #endif
    19.     }
    20. }
    21.  
     
  48. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    I fixed this issue in the latest release, please update to version 4.9.3.
     
    PSST_Adam likes this.
  49. RyanJVR

    RyanJVR

    Joined:
    Jan 15, 2014
    Posts:
    6
    Hi,

    I've just updated to the latest version 4.9.3 but now I'm getting compile errors.

    "plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(87,37): error CS0122: 'Texture2DSerializable.data' is inaccessible due to its protection level
    plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(88,37): error CS0122: 'Texture2DSerializable.compressed' is inaccessible due to its protection level
    plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(89,37): error CS0122: 'Texture2DSerializable.format' is inaccessible due to its protection level
    plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(90,37): error CS0122: 'Texture2DSerializable.width' is inaccessible due to its protection level
    plugins\ARFoundationRemoteSource\Runtime\Sender\Occlusion\DepthImageSerializer.cs(91,37): error CS0122: 'Texture2DSerializable.height' is inaccessible due to its protection level"

    Was working fine before the update.

    Regards
    Ryan

    Unity 2020.2.7f1
     
  50. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    718
    The 'DepthImageSerializer' script was removed in the newer version and now causes conflicts.
    Please make sure you've uninstalled the previous plugin version completely before updating to a new version.
    While Unity Asset Importer deals with new files and modifications to the old ones fine, it doesn't support file removal.
     
unityunity