Search Unity

  1. Are you interested in providing feedback directly to Unity teams? Sign up to become a member of Unity Pulse, our new product feedback and research community.
    Dismiss Notice

AR Foundation Editor Remote | Test and debug your AR project in the Editor

Discussion in 'Assets and Asset Store' started by KirillKuzyk, May 26, 2020.

  1. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Please re-import the plugin from the Asset Store if you want to use the plugin on another development machine.
    If your AR project is closed-source, then you can commit the plugin to your VCS and it will prevent the aforementioned message from happening.
     
  2. nucleartide

    nucleartide

    Joined:
    Jun 15, 2012
    Posts:
    16
    I created a video on setting up your AR workflow with AR Foundation Editor Remote:


    Big fan of this plugin! :) Will be using it for all my AR work going forward, and hopefully it will help others too.
     
    AM-Dev and KirillKuzyk like this.
  3. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Thank you for the great tutorial video!
    Do you mind if I put it on the plugin's description on the Asset Store page?
     
  4. nucleartide

    nucleartide

    Joined:
    Jun 15, 2012
    Posts:
    16
    I'd be honored, please do! :D
     
  5. kidsfab

    kidsfab

    Joined:
    Oct 16, 2018
    Posts:
    9
    Hello!
    I Have been enjoying the app until two days ago when I had to reinstall the app through Xcode.
    Haven't been able to make it work since. The app crashes as soon as I start play in the editor. I don't get any error messages in Unity.
    I ran it with Xcode and the last line in debug says :
    JPEG parameter struct mismatch: library thinks size is 520, caller expects 504
    Help greatly appreciated!

    EDIT: problem still persisting. I'll drop here my specs so far. 2019.4.23f1 companion app built on iOS 14.6
    I had a major corrupted unity project and the crash so I started a new project. Dowloaded the last updated package. Built a new companion app. Problem still there. Companion app crashes as soon as it gets connected to the editor (play). Then the editor only says it has lost connection. What am I doing wrong ?
     
    Last edited: Jun 16, 2021
  6. kidsfab

    kidsfab

    Joined:
    Oct 16, 2018
    Posts:
    9
    Hello KirillKuzyk,
    As a last resort I thought I would go back to the package version that worked, but there doesn't seem to be access to previous versions....
    I don't know if the problem is due to the new version though.
    Here is the debug log in Xcode just before it crashes, which happens as soon as I hit the play button in the editor. I'm on a wired connection.


    [B]Configuration Descriptor 0x1f9d09ba0 (rank 2): Rotation and Orientation, Plane Tracking, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast[/B]

    [B]Configuration Descriptor 0x1f9d09ad8 (rank 0): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Object Tracking, Environment Probes, 2D Body Tracking, People Occlusion Stencil, People Occlusion Depth, Collaboration, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast, Meshing, Mesh Classification, Environment Depth[/B]

    [B]Configuration Descriptor 0x1fb93cf38 (rank 1): World Facing Camera, Rotation Only, 2D Body Tracking, People Occlusion Stencil, People Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Environment Depth[/B]

    [B]Configuration Descriptor 0x1f9d09790 (rank 1): World Facing Camera, Rotation Only, Image Tracking, 2D Body Tracking, People Occlusion Stencil, People Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Environment Depth[/B]

    [B]Configuration Descriptor 0x1f9d0a118 (rank -1): World Facing Camera, Rotation and Orientation, Plane Tracking, Image Tracking, Environment Probes, 2D Body Tracking, 3D Body Tracking, 3D Body Scale Estimation, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Raycast[/B]


    [B]UnityEngine.XR.ARSubsystems.XRSessionSubsystem:DetermineConfiguration(Feature)[/B]

    [B]UnityEngine.XR.ARSubsystems.XRSessionSubsystem:Update(XRSessionUpdateParams)[/B]

    [B]UnityEngine.XR.ARFoundation.ARSession:Update()[/B]


    [B](Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 39)[/B]


    [B]Using session configuration 0x1f9d09ad8[/B]

    [B] Requested Features: World Facing Camera, Rotation and Orientation, Plane Tracking, People Occlusion Stencil, People Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color), Light Estimation (Spherical Harmonics), Light Estimation (Main Light Direction), Light Estimation (Main Light Intensity)[/B]

    [B] Supported Features: World Facing Camera, Rotation and Orientation, Plane Tracking, People Occlusion Stencil, People Occlusion Depth, Auto-Focus, Light Estimation (Ambient Intensity), Light Estimation (Ambient Color)[/B]

    [B] Requested features not satisfied: Light Estimation (Spherical Harmonics), Light Estimation (Main Light Direction), Light Estimation (Main Light Intensity)[/B]

    [B]UnityEngine.XR.ARSubsystems.XRSessionSubsystem:DebugPrintConfigurationChange(Configuration, Feature)[/B]

    [B]UnityEngine.XR.ARSubsystems.XRSessionSubsystem:Update(XRSessionUpdateParams)[/B]

    [B]UnityEngine.XR.ARFoundation.ARSession:Update()[/B]


    [B](Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 39)[/B]


    [B]2021-06-16 13:30:37.523100+0900 ARpuppetARCompanion[6562:4097174] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x28358c840 worldAlignment=Gravity lightEstimation=Enabled frameSemantics=Person Segmentation with Depth videoFormat=<ARVideoFormat: 0x282360910 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=Horizontal collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None appClipCodeTracking=Disabled>[/B]

    [B]2021-06-16 13:30:37.528810+0900 ARpuppetARCompanion[6562:4097413] DEFAULTS: (intermediatePathOverride)=>(<null>)[/B]

    [B]2021-06-16 13:30:37.529939+0900 ARpuppetARCompanion[6562:4097413] DEFAULTS: (logLevel)=>(3)[/B]

    [B]2021-06-16 13:30:37.530248+0900 ARpuppetARCompanion[6562:4097413] DEFAULTS: (jasperColorStillsRunCorrection)=>(1)[/B]

    [B]2021-06-16 13:30:37.713174+0900 ARpuppetARCompanion[6562:4097415] [espresso] Warning: padding deconvolution PersonSeg/decoder/deconv_64/conv2d_transpose:0 in SAME mode will not be pad-invariant for all resolutions[/B]

    [B]2021-06-16 13:30:37.713229+0900 ARpuppetARCompanion[6562:4097415] [espresso] Warning: padding deconvolution PersonSeg/decoder/deconv_32/conv2d_transpose:0 in SAME mode will not be pad-invariant for all resolutions[/B]

    [B]2021-06-16 13:30:37.713248+0900 ARpuppetARCompanion[6562:4097415] [espresso] Warning: padding deconvolution PersonSeg/decoder/deconv_16/conv2d_transpose:0 in SAME mode will not be pad-invariant for all resolutions[/B]

    [B]2021-06-16 13:30:37.713263+0900 ARpuppetARCompanion[6562:4097415] [espresso] Warning: padding deconvolution PersonSeg/decoder/deconv_8/conv2d_transpose:0 in SAME mode will not be pad-invariant for all resolutions[/B]

    [B]2021-06-16 13:30:37.713279+0900 ARpuppetARCompanion[6562:4097415] [espresso] Warning: padding deconvolution PersonSeg/decoder/deconv_4/conv2d_transpose:0 in SAME mode will not be pad-invariant for all resolutions[/B]

    [B]2021-06-16 13:30:37.713293+0900 ARpuppetARCompanion[6562:4097415] [espresso] Warning: padding deconvolution PersonSeg/decoder/deconv_2/conv2d_transpose:0 in SAME mode will not be pad-invariant for all resolutions[/B]

    [B]JPEG parameter struct mismatch: library thinks size is 520, caller expects 504[/B]

     
  7. masai2k

    masai2k

    Joined:
    Mar 2, 2013
    Posts:
    26
    Hi,
    I just bought your plugin, but in project settings -> XR plugin management I don't find AR foundation remote under PC/Mac settings!!
    I do everything correct (twice!).
    I use unity 2021.3.10 on Mac

    Massimo
     
  8. Chaoslife

    Chaoslife

    Joined:
    Aug 16, 2016
    Posts:
    6
    Hi,
    I reimport plugins from packagemanager, and when I played on editor, it throw this exception:

    SerializationException: Unable to load type ARFoundationRemote.Runtime.PlayerToEditorMessage required for deserialization.
    System.Runtime.Serialization.ObjectManager.DoFixups () (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize (System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Serialization.Formatters.Binary.__BinaryParser serParser, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    ARFoundationRemote.Runtime.SerializationExtensions.Deserialize[T] (System.Byte[] bytes) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Utils/SerializationExtensions.cs:32)
    ARFoundationRemote.Runtime.TelepathyConnection.runBackgroundThread () (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/TelepathyConnection.cs:108)
    System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    System.Threading.ThreadHelper.ThreadStart () (at <eae584ce26bc40229c1b1aa476bfa589>:0)
    UnityEngine.<>c:<RegisterUECatcher>b__0_0(Object, UnhandledExceptionEventArgs)



    and, i had installed the plugins
     
  9. TomKhan3

    TomKhan3

    Joined:
    Jul 21, 2017
    Posts:
    18
    I'm wondering how to move my seat to a new computer. Is it as simple has uninstalling the plugin from the original computer and installing the plugin on the new computer?
     
  10. DreamPower

    DreamPower

    Joined:
    Apr 2, 2017
    Posts:
    102
    Edit: I searched this thread, and apparently the package manager doesn't update the folder paths even after removing the package and reinstalling it, so this is a non-issue for new installs.

    Original post:
    There seems to be a bug in the current version. When creating the Companion App, it's looking for a folder called Scenes/ARCompanion, but the folder there is called Scenes/Sender. When I renamed the folder, it was able to create the companion app as intended and everything worked great.

    Note: I doubt this would affect this specific issue, but I'm using Unity 2021.2.0 alpha 21 and AR Foundation 4.2.0-pre.5, with the app running on iPad OS 15 beta, built with XCode 13 beta. The good news, is AR Foundation Remote seems to be working just fine with this pre-release setup.
     
    Last edited: Jun 19, 2021
  11. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Hi!

    I'm sorry for the late reply.
    Google doesn't know much about this specific problem, but it seems like Unity/Xcode version mismatch. Can you afford to upgrade your Unity version to something newer?

    Could you please remember what has changed since the last time you compiled the AR Companion app? Maybe you updated the XCode or iOS on your phone?

    Please send me your invoice number in private messages via email and I'll send you a previous version of the plugin.
     
  12. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Please let me assume you're on Unity 2020.3.10 and not on 2021.3.10. I'll try to install the 2020.3.10 version myself and will try to reproduce the issue.
    Please check that the 'Packages/com.kyrylokuzyk.arfoundationremote' folder is present and try to relaunch Unity Editor.
     
  13. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Please make sure to build a new AR Companion version after the plugin update.
     
  14. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Yes, it is that simple :)
     
  15. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    The 'Scenes/Sender' folder structure is an old one. The most recent version of the plugin uses the 'Scenes/ARCompanion' folder structure.
    When an old file is removed in plugin update, the Unity Package Manager will not delete it, leading to mixing files from several different versions together. Please make sure to uninstall the plugin completely before updating to a newer version to be 100% safe. This applies to the AR Foundation Editor Remote and any other plugin from the Asset Store.
     
  16. kidsfab

    kidsfab

    Joined:
    Oct 16, 2018
    Posts:
    9
    Thanks @KirillKuzyk
    After a lot of swearing and starting dummy projects in hope of debugging the problem it suddenly worked. Unfortunately I have nothing constructive to append to this thread in terms of warning or bug uncovering.

    I had to work the old way (building to phone at each major iteration) for a week and what a pain that is !!!
    It truly shows how your package is a necessary tool for anyone working on an AR project.
     
    KirillKuzyk likes this.
  17. hyeongwoo1

    hyeongwoo1

    Joined:
    Oct 15, 2018
    Posts:
    1
    I have a question.
    i have Window Computer and iPhone but not Mac.
    Is it possible to run on Windows OS + IOS without Mas OS?
    and If it is possible, how do I setup it?
     
    Last edited: Jun 22, 2021
  18. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Unfortunately, there is no simple way. My plugin requires you to build the AR Companion app once to connect your AR device to Unity Editor.
    An iOS Project Builder for Windows can produce iOS builds on Windows, but I haven't tried it myself, so I can't guarantee that it's working properly with AR Foundation.

    In my experience, if you're planning to release an iOS app in the future, it's worth having a macOS computer; this will save you a lot of hadeache.
     
  19. frankrs

    frankrs

    Joined:
    Aug 29, 2009
    Posts:
    298
    Assets\Plugins\ARFoundationRemoteInstaller\com.kyrylokuzyk.arfoundationremote\Runtime\Utils\Settings.cs(7,29): error CS0246: The type or namespace name 'ScriptableObjectSingleton<>' could not be found (are you missing a using directive or an assembly reference?)

    Assets\Plugins\ARFoundationRemoteInstaller\com.kyrylokuzyk.arfoundationremote\Runtime\Utils\Settings.cs(21,51): error CS0246: The type or namespace name 'PackageVersionData' could not be found (are you missing a using directive or an assembly reference?)


    Getting these errors in 2021.1.11f1 building for anroid any ideas?
     
  20. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Please try to delete the plugin completely, restart Unity, then re-import the plugin again from the Asset Store.
     
  21. kunstlabdev

    kunstlabdev

    Joined:
    Feb 25, 2020
    Posts:
    5
    I am getting a new set of errors when trying to get touch input:

    Code (CSharp):
    1. TargetParameterCountException: Number of parameters specified does not match the expected number.
    2. System.Reflection.RuntimeMethodInfo.ConvertValues (System.Reflection.Binder binder, System.Object[] args, System.Reflection.ParameterInfo[] pinfo, System.Globalization.CultureInfo culture, System.Reflection.BindingFlags invokeAttr) (at <202f8c081a504aa2940103428e3fd66b>:0)
    3. System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <202f8c081a504aa2940103428e3fd66b>:0)
    4. System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at <202f8c081a504aa2940103428e3fd66b>:0)
    5. ARFoundationRemote.RuntimeEditor.LegacyInputReceiver.simulate (UnityEngine.Touch touch) (at Packages/com.kyrylokuzyk.arfoundationremote/RuntimeEditor/Input/LegacyInputReceiver.cs:158)
    6. ARFoundationRemote.RuntimeEditor.LegacyInputReceiver.simulateReceivedAndMouseTouches () (at Packages/com.kyrylokuzyk.arfoundationremote/RuntimeEditor/Input/LegacyInputReceiver.cs:145)
    7. ARFoundationRemote.RuntimeEditor.LegacyInputReceiver+TouchSimulationUpdate.Update () (at Packages/com.kyrylokuzyk.arfoundationremote/RuntimeEditor/Input/LegacyInputReceiver.cs:135)
    Using Unity 2021.2.0b1 AR Foundation 4.2.0-pre.9 FoundationRemote 4.11.10, on iOS.
     
  22. WayneVenter

    WayneVenter

    Joined:
    May 8, 2019
    Posts:
    20
    @KirillKuzyk Getting the following error on AR Remote, latest build, iPhone connected to Mac Mini M1
    Lots of errors on JPEG parameter struct mistmatch.

    AssertionException: Assertion failure. Value was False
    Expected: True
    UnityEngine.Assertions.Assert.Fail (System.String message, System.String userMessage) (at /Users/bokken/buildslave/unity/build/Runtime/Export/Assertions/Assert/AssertBase.cs:29)
    UnityEngine.Assertions.Assert.IsTrue (System.Boolean condition, System.String message) (at /Users/bokken/buildslave/unity/build/Runtime/Export/Assertions/Assert/AssertBool.cs:20)
    UnityEngine.Assertions.Assert.IsTrue (System.Boolean condition) (at /Users/bokken/buildslave/unity/build/Runtime/Export/Assertions/Assert/AssertBool.cs:13)
    ARFoundationRemote.Runtime.Texture2DSerializable.deserializeInto (UnityEngine.Texture2D tex) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Utils/Texture2DSerializable.cs:172)
    ARFoundationRemote.Runtime.Texture2DSerializable.ResizeIfNeededAndDeserializeInto (UnityEngine.Texture2D& tex) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Utils/Texture2DSerializable.cs:159)
    ARFoundationRemote.Editor.TextureAndDescriptor.Update (ARFoundationRemote.Runtime.Texture2DSerializable texture2DSerializable, System.Int32 propNameId) (at Packages/com.kyrylokuzyk.arfoundationremote/Editor/Subsystems/TextureAndDescriptor.cs:40)
    ARFoundationRemote.Editor.CameraSubsystem+CameraSubsystemProvider.receiveCameraFrame (ARFoundationRemote.Runtime.ARCameraFrameEventArgsSerializable remoteFrame) (at Packages/com.kyrylokuzyk.arfoundationremote/Editor/Subsystems/CameraSubsystem.cs:370)
    ARFoundationRemote.Runtime.TelepathyConnection+<>c__DisplayClass26_0`1[T].<ARFoundationRemote.Runtime.IConnection.Register>b__0 (System.Object data) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/TelepathyConnection.cs:165)
    ARFoundationRemote.Runtime.TelepathyConnection.Update () (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/TelepathyConnection.cs:140)
     

    Attached Files:

    Last edited: Jun 24, 2021
  23. WayneVenter

    WayneVenter

    Joined:
    May 8, 2019
    Posts:
    20
    Some further update on this, I'm using all the latest versions of Unity, ARE Remote and Xcode (from 3 days ago) ie.
    Unity 2020.3.12f1
    Xcode 12.5.1
    AR Remote 4.11.10

    Also can't go back to my pervious version, not available in Package Manager ... so stuck.
     
  24. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    This beta Unity version is not currently supported by the plugin. I'll make sure to fix this in the next update, thank you for reporting the problem!
     
  25. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    This issue seems very rare, google doesn't help much. Could you please remember what has changed since the last time the plugin was working? Have you updated Unity or Xcode? If yes, please try to re-build the AR Companion app.

    I haven't changed anything regarding the image serialization for a long time, so the previous version should not make a difference. But if you still want to try it, please contact me via email and I'll send you the previous version of the plugin.
     
  26. MehdiKhadir

    MehdiKhadir

    Joined:
    Mar 12, 2018
    Posts:
    3
    Hi @KirillKuzyk thanks for the nice plugin. We've been using it for quite a while now and would like to update it from version 4.5.0 to the latest version 4.11.10 as we are updating Unity from 2019.4.11 to Unity 2020.3.12 and AR Foundation from 3.1.3 to 4.1.7.
    However when I import the new package into Unity I get the following errors

    upload_2021-7-2_11-13-18.png

    Assets\Plugins\ARFoundationRemoteInstaller\Scripts\Editor\ARFoundationRemoteInstallerInspector.cs(22,39): error CS0117: 'FixesForEditorSupport' does not contain a definition for 'Apply'

    Assets\Plugins\ARFoundationRemoteInstaller\Scripts\Editor\ARFoundationRemoteInstaller.cs(60,17): error CS0029: Cannot implicitly convert type 'void' to 'bool'

    Assets\Plugins\ARFoundationRemoteInstaller\Scripts\Editor\CompanionAppInstaller.cs(161,30): error CS1061: 'Settings' does not contain a definition for 'packages' and no accessible extension method 'packages' accepting a first argument of type 'Settings' could be found (are you missing a using directive or an assembly reference?)

    Assets\Plugins\ARFoundationRemoteInstaller\Scripts\Editor\CompanionAppInstaller.cs(161,41): error CS0103: The name 'PackageVersionData' does not exist in the current context

    The project didn't contain any error before importing the update and I cannot find any "Embed AR Foundation Package" button in the Installer. Also maybe worth mentioning the AR Core, AR Kit and AR Foundation packages were already moved to the Package folder earlier in the project.
    What could I try to do to fix these issues?
     
  27. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    It seems like the installer folder version doesn’t correspond to the plugin version. Please uninstall the plugin completely and make a fresh install.
     
  28. knurgb

    knurgb

    Joined:
    Jun 24, 2017
    Posts:
    7
    Does body tracking work? I tried arfoundation-samples and demos work but body tracking don't. Let me know if I can provide more info.
     
  29. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Could you please tell me your AR device model? Body tracking requires iOS 12+ and A9+ CPU.
     
  30. MehdiKhadir

    MehdiKhadir

    Joined:
    Mar 12, 2018
    Posts:
    3
    Thank you that helped! Now it seems I could install the plugin. However, I cannot successfully install the companion app on my Android phone. If I try to hit "Build AR Companion and show in folder" I get an apk but when I launch it the screen is black apart from "Development Build" in the bottom right corner, and I can't successfully connect to the Unity Editor. I get the following error in the Unity Editor console:

    Client Recv Exception: System.Exception: Connection to 192.168.1.238 failed with exception: System.AggregateException: One or more errors occurred. ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it.

    I have made sure both are on the same network and the phone's IP is entered in the plugin's settings. I have also tried to connect via USB.

    If I try to hit "Install AR Companion app" then the companion app simply won't build and I get the following upload_2021-7-9_14-49-3.png

    Any idea of things I could try?
     
  31. kidsfab

    kidsfab

    Joined:
    Oct 16, 2018
    Posts:
    9
    Hi @KirillKuzyk and all !
    I have a question regarding camera stacks and the remote. In order to match the device camera's projection matrix, I copy the specs of the AR Camera by script to any child camera in the stack, however it doesn't seem to be reflected in the editor. The FOV seems to be off as well. Is this only available in a build or can we debug that with the editor remote package?
    I'm also looking towards separating the background from any augmented objects while still applying the occlusion stencil on the augmented object. Is that even feasible? Any insights are greatly appreciated. Thanks !
     
  32. Roflbong

    Roflbong

    Joined:
    Oct 31, 2019
    Posts:
    1
    Hi @KirillKuzyk,

    i have a problem to fetch the current activity. Throws the following error. Do you have a fix for this issue? upload_2021-7-14_15-4-33.png
     
  33. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Could you please tell me what Unity version do you use?
    Could you please attach the ADB debugger and send me the console output of the AR Companion app?

    Please also make sure you're using the recommended (verified) ARFoundation/ARCore packages.
     
    Last edited: Jul 17, 2021
  34. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    It should be possible in the Editor. My plugin supports both projectionMatrix and displayMatrix. Could you please show me your code? A small reproducible project would be ideal.
    It seems like the best solution for your case would be to write a custom camera shader that applies occlusion texture, but doesn't render camera image.
     
  35. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    It seems like you're trying to run native Android code inside the Unity Editor. This is not supported by the plugin. My plugin covers only the AR Foundation API and, unfortunately, can't help you in debugging native code.
     
  36. cribin

    cribin

    Joined:
    Jul 11, 2018
    Posts:
    7
    Hi @KirillKuzyk,

    Still eagerly awaiting the session recording and playback feature :). Any updates on when it will be ready ?

    Best,

    Ribin
     
  37. shiquhudong

    shiquhudong

    Joined:
    May 19, 2017
    Posts:
    25
    Hi @KirillKuzyk, Doesn't it support CpuImage functions like TryAcquireEnvironmentDepthCpuImage()?
     
  38. SolidAlloy

    SolidAlloy

    Joined:
    Oct 21, 2019
    Posts:
    48
    The error states there is not enough space on the disk. Have you checked if the C disk is running out of space?
     
  39. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    The occlusion cpu image APIs are not currently supported. Only the camera cpu image API is supported.
     
  40. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    I’m actively working on it. It turned out to be a more complex task than it seemed. There are a lot of corner cases that should be accounted for. But I’m almost done :)
     
    Last edited: Jul 22, 2021
  41. SolidAlloy

    SolidAlloy

    Joined:
    Oct 21, 2019
    Posts:
    48
    Hello @KirillKuzyk and thank you for the plugin.

    As I can see, your implementation of CameraSubsystem is rigid and does not fetch data about supported features from the device. For example, to define
    supportsAverageBrightness
    , it just checks if the underlying plugin is ARKit.

    First-of-all, those features do not always depend on a plugin. I checked it with one of the iPhones, and it supported average brightness.

    Second-of-all, it does not define parameters such as
    supportsWorldTrackingHDRLightEstimation
    , leaving them
    false
    , although the device may support HDR light estimation.

    Can these parameters be fetched from the device? Do you have such a feature in plans?

    I have code that depends on those "supports..." properties, and it behaves differently through AR Foundation Remote and in a build.
     
    KirillKuzyk likes this.
  42. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Yes, you're right, the plugin's CameraSubsystem supported features are hardcoded. But the ARKitCameraSubsystem also has hardcoded values (except for supportsCameraGrain that is determined at runtime):
    Code (CSharp):
    1. // ARKitCameraSubsystem.cs, ARKit 4.2.0-pre.9
    2. XRCameraSubsystemCinfo cameraSubsystemCinfo = new XRCameraSubsystemCinfo
    3. {
    4.     id = k_SubsystemId,
    5.     providerType = typeof(ARKitCameraSubsystem.ARKitProvider),
    6.     subsystemTypeOverride = typeof(ARKitCameraSubsystem),
    7.     supportsAverageBrightness = false,
    8.     supportsAverageColorTemperature = true,
    9.     supportsColorCorrection = false,
    10.     supportsDisplayMatrix = true,
    11.     supportsProjectionMatrix = true,
    12.     supportsTimestamp = true,
    13.     supportsCameraConfigurations = true,
    14.     supportsCameraImage = true,
    15.     supportsAverageIntensityInLumens = true,
    16.     supportsFocusModes = true,
    17.     supportsFaceTrackingAmbientIntensityLightEstimation = true,
    18.     supportsFaceTrackingHDRLightEstimation = true,
    19.     supportsWorldTrackingAmbientIntensityLightEstimation = true,
    20.     supportsWorldTrackingHDRLightEstimation = false,
    21.     supportsCameraGrain = Api.AtLeast13_0(),
    22. };

    Judging from the snippet above, the supportsAverageBrightness can never be true in ARKit. Could you please double-check that one of your iPhones reports supportsAverageBrightness == true? Because I don't see how this is possible. The supportsAverageBrightness should be only available in ARCore.

    I'll add these missing properties in the next plugin update, thank you for the bug report! I wrote CameraSubstem back in the days the supportsWorldTrackingHDRLightEstimation wasn't there. I'll adapt the plugin for the most recent AR Foundation version.

    Yes, fetching the runtime capabilities is on my roadmap. The XRCameraSubsystem doesn't require any fetching because its capabilities are hardcoded and can be determined ahead of time, but subsystems like XROcclusionSubsystem determine their supported features at runtime, and I plan to fetch them from the connected AR device in the future.
     
  43. urrmurrmur

    urrmurrmur

    Joined:
    Sep 2, 2013
    Posts:
    24
    Hi @KirillKuzyk,

    Thanks for the app. Works like a charm on Android for me, but I'm having issues on iOS. When I run the companion app, I get errors saying my device doesn't have AR capabilities, even though it does. When I run my project in the editor, I see additional errors on the device (iPad). I've attached a screenshot of the full error output on the device.

    Some information:
    Windows 10.0.19041
    Unity 2020.3.14f1
    iPad Pro 12.9 (3rd gen)
    iOS 14.6
    ARFoundation, ARKit XR, ARKit Face Tracking, ARCore XR all v 4.1.7
    AR Foundation Remote v 4.11.10
    Companion app built with Unity Cloud build, with 2020.3.14f1 and XCode 12.4.0

    All the necessary configurations have been checked in project settings.


    I was running Unity 2020.3.5f1 before (with AR packages v. 4.0.12), and I could build and run the companion app without issues then, but got a black screen when running my own app. Tracking worked though, I could see my face being tracked in the editor. I updated to the latest Unity and AR packages in hopes of solving the black screen error, but the issues seem to have gotten worse. Any ideas?
     

    Attached Files:

  44. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    It seems like a bug in ARFoundation/ARKit and I've experienced it too. The first time Unity makes an iOS build, the ARKit library is not added in the Xcode. Please repeat the AR Companion app build and it should fix the issue.
     
  45. urrmurrmur

    urrmurrmur

    Joined:
    Sep 2, 2013
    Posts:
    24
    Unfortunately it hasn't. We've run the cloud build again, with the same result. After that we built the companion app directly from XCode, again with the same result. Then we repeated that XCode build again, still with the same error. Is there anything else we could try?

    EDIT: I've attached a screenshot of the exact error we're getting on the device, BEFORE trying to run our app. After running the app, we get the error from my post above.
     

    Attached Files:

    Last edited: Jul 29, 2021
  46. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Is there a chance you're trying to install the AR Companion app with is a custom build pipeline instead of pressing the 'Installer/Install AR Companion App'?

    Judging from the errors above, it seems like the ARKit plugin is not enabled. Could you please double-check that the ARKit is enabled in the 'XR Plug-in management' window?
    upload_2021-7-29_11-36-18.png

    The correctly installed AR Companion app should display all relevant packages in the bottom left corner:
    upload_2021-7-29_11-42-2.jpeg
     
  47. J_Kost

    J_Kost

    Joined:
    Feb 16, 2021
    Posts:
    11
    Hi,

    is it possible to find out at runtime whether the companion app is connected yet?
    I have many cases where I need to work around errors caused at early runtime by calls to ARCameraManager or XRCpuImages before the app is ready.

    Being able to delay these calls until the connection is active would be great.

    Thanks!
     
  48. SolidAlloy

    SolidAlloy

    Joined:
    Oct 21, 2019
    Posts:
    48
    I've got an interesting result. Yes, supportsAverageBrightness is apparently always false on ARKit. However, when I enabled AverageBrightness on iPhone XR, it started outputting values and reacted to light change correctly. So, either Unity magically transforms AverageIntensityInLumens into AverageBrightness, or some iPhones have AverageBrightness support but since Unity has it hardcoded, it will never say the average brightness is supported.
     
    KirillKuzyk likes this.
  49. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Yes, you can check if the AR Companion app is connected via this property:
    ARFoundationRemote.Runtime.Connection.Instance.isConnected
     
    J_Kost likes this.
  50. KirillKuzyk

    KirillKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    692
    Yes, this is indeed strange behavior.
    My plugin should have the same behavior as you described. Even if
    supportsAverageBrightness == false
    , I still transfer all available data in
    ARCameraFrameEventArgs
    back to the Editor. Here is the serialization code (line 5):
    Code (CSharp):
    1. public static XRCameraFrameSerializable Create(ARCameraFrameEventArgs a) {
    2.     var lightEstimation = a.lightEstimation;
    3.     return new XRCameraFrameSerializable {
    4.         timestampNs = a.timestampNs ?? 0,
    5.         averageBrightness = lightEstimation.averageBrightness ?? 0,
    6.         averageColorTemperature = lightEstimation.averageColorTemperature ?? 0,
    7.         colorCorrection = ColorSerializable.Create(lightEstimation.colorCorrection ?? new Color()),
    8.         projectionMatrix = Matrix4x4Serializable.Create(a.projectionMatrix ?? Matrix4x4.zero),
    9.         displayMatrix = Matrix4x4Serializable.Create(a.displayMatrix ?? Matrix4x4.zero),
    10.         trackingState = TrackingState.Tracking,
    11.         properties = getProps(a),
    12.         averageIntensityInLumens = lightEstimation.averageIntensityInLumens ?? 0,
    13.         exposureDuration = a.exposureDuration ?? 0,
    14.         exposureOffset = a.exposureOffset ?? 0,
    15.         #if ARFOUNDATION_4_0_2_OR_NEWER
    16.             m_MainLightIntensityLumens = lightEstimation.mainLightIntensityLumens ?? 0,
    17.             m_MainLightColor = ColorSerializable.Create(lightEstimation.mainLightColor ?? new Color()),
    18.             m_MainLightDirection = Vector3Serializable.Create(lightEstimation.mainLightDirection ?? Vector3.zero),
    19.             m_AmbientSphericalHarmonics = SphericalHarmonicsL2Serializable.Create(lightEstimation.ambientSphericalHarmonics ?? new UnityEngine.Rendering.SphericalHarmonicsL2()),
    20.             m_NoiseIntensity = a.noiseIntensity
    21.         #endif
    22.     };
    23. }
    Could you please confirm that when you connect your iPhone XR to the Editor, the behavior is the same as in a real build?
     
    Last edited: Jul 31, 2021
unityunity