Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

AR Foundation Remote | Test and debug your AR project in the Editor

Discussion in 'Assets and Asset Store' started by KyryloKuzyk, May 26, 2020.

  1. SolidAlloy

    SolidAlloy

    Joined:
    Oct 21, 2019
    Posts:
    58
    Yes, the behavior is the same. Your plugin works with AverageBrightness in the same way as in builds, which is great! I just wanted to share my findings and why I was confused by the average brightness data on iOS at first. Thanks for the support
     
    KyryloKuzyk likes this.
  2. J_Kost

    J_Kost

    Joined:
    Feb 16, 2021
    Posts:
    32
    Thank you, that does the trick! Did I miss a documentation where I could have looked this up?
    Also, do you by any chance set a #define directive that I could use to detect whether the ARFoundation Remote is present in a project or not? I figured I could detect your namespaces using an approach like this: https://forum.unity.com/threads/run-bit-of-code-if-namespace-exists-c.437745/ but i don't really like the idea.
     
  3. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    No, this wasn't documented. I'll update the documentation in the next plugin version, thank you!

    I don't set any #define directive to leave projects of my customers intact.
    Instead, I would recommend using an Assembly Definition file in your project. In the *.asmdef you can create custom defines for every package in the Unity Package Manager. Here is an example describing how to set up a custom define for the AR Foundation Editor Remote plugin. This is also not included in the documentation and I'll add it in the next update :)
     
  4. OrenD

    OrenD

    Joined:
    Jul 14, 2020
    Posts:
    8
    Hello, I’d like to bring up a possible privacy issue for users of apps developed with AR Foundation Remote. It's probably not an actual privacy issue, but a percieved one for users.

    When opening the app for the first time, this pops up.

    “App title” would like to find and connect to devices on your local network. This app will be able to discover and connect to devices on the networks you use.”

    This seems to be coming from the plugin code related to creating the companion app.

    I tried changing the name of the resources folder within the plugin in case those files were the ones prompting users when they are included in the build. Didn’t work.

    Can’t find anything in the info.plist about it.

    Do I need to remove the plugin from my project just so I can make a development build?

    -Oren
     
  5. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Hi Oren,

    I just created an absolutely empty Unity project and built an empty scene and I also receive the same prompt.
    It seems like the 'Development Build' causes this prompt. Building the project in Release configuration removes the prompt.
    Could you please confirm it?
    https://forum.unity.com/threads/app...ur-local-network-pop-up.1000615/#post-6517547
     
  6. OrenD

    OrenD

    Joined:
    Jul 14, 2020
    Posts:
    8
    I owe you an apology for not doing my due diligence and for assuming the source of the problem instead of doing testing first!

    I confirm this is a part of development build and will take greater care in the future!

    -Oren
     
    KyryloKuzyk likes this.
  7. orangetech

    orangetech

    Joined:
    Sep 30, 2017
    Posts:
    50
    I have two camera in scene.
    one camera for webrtc stream,other one camera to render local.
    ARFoundation remote give me this assert fail:
    Code (CSharp):
    1. AssertionException: RenderTexture.active == null
    2. Assertion failure. Value was not Null
    3. at Packages/com.kyrylokuzyk.arfoundationremote/RuntimeEditor/EditorView/EditorViewSender.cs:47
    when device remote connected, device display the webrtc camera.
    might add option to chose which camera to send to device?
     
    Last edited: Aug 6, 2021
  8. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    The plugin doesn't choose a specific camera, but instead, it reads the screen backbuffer (whole screen) with Texture2D.ReadPixels.
    The plugin erroneously assumes that RenderTexture.active is null when it sends Editor Game View to the AR device. But in your case, the RenderTexture.active is non-null. Please send me your invoice number in private messages and I'll send a new version with a fix.

    Please also ensure that modifying the RenderTexture.active was your actual intention. In almost all cases restoring the RenderTexture.active to the original value is the desired behavior.
     
    Last edited: Aug 6, 2021
  9. WhiteGfx

    WhiteGfx

    Joined:
    Apr 10, 2017
    Posts:
    16
    Hi @KirillKuzyk, your plugin is excelent, as I wrote in review big time saver!. Maybe last update 1.4.11-release.13 bring some unwanted problems with mouse clicking in editor (same with touch in remote on device). When I remove plugin than whole project is again clickable without any lag. I tested one button in empty scene with no plugins or any tweaks, just regular new project with same result. When 1.4.11-release.13 is imported, clicking on any clickable GO is performed one from 5-10 clicks. My question is how can I downgrade? to .11 release, was ok before update to .13. If you know how to fixit on my side here is setup: Unity 2021.1.16f1 on Mac, Old Input System, ARF 4.2.0-pre.12, Or any tips what to check will be great. Thank you

    Edit: Clicking works after plugin install, works after Hitting embed embed plugin.
    When stops works? When I check the plugin provider ARFR in XR plugin management.
    Maybe this will help to track the issue.
     
    Last edited: Aug 6, 2021
    KyryloKuzyk likes this.
  10. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    I managed to reproduce the issue, thank you!
    So the reason why the plugin "worked" for you in version 1.4.11-release.11 is that the touch input remoting/simulation feature was broken in Unity 2021.1 due to the internal UnityEngine.Input API change.
    In version 13 I fixed the previous issue in Unity 2021.1, but not fully: in this version of Unity simulated touch should be of type TouchType.Direct instead of indirect (which worked in all previous versions of Unity, which is strange).
    Please send me your invoice number in private messages and I'll send a new version with a fix.
    Thanks again for a bug report!
     
  11. michaelybecker

    michaelybecker

    Joined:
    Jun 18, 2014
    Posts:
    19
    Hi @KirillKuzyk - loving your plugin, worth every penny.

    I'm running into issues with my Unity Cloud Build which started popping up after integrating your plugin - presumably because I'm ignoring the necessary libs in Perforce per the documentation, but (probably because of the embedding in ARF?), the cloud build still expects those files.

    How might I setup the project and project settings to enable local usage of the plugin without messing up the cloud build? Thanks!

    Relevant log:

    Code (CSharp):
    1. 7206: [Unity] Packages/com.unity.xr.arfoundation@4.0.12/Runtime/AR/ARMeshManager.cs(11,29): error CS0246: The type or namespace name 'ARFoundationRemote' could not be found (are you missing a using directive or an assembly reference?)
    2. 7207: [Unity] Packages/com.unity.xr.arfoundation@4.0.12/Runtime/AR/ARMeshManager.cs(17,30): error CS0246: The type or namespace name 'ARFoundationRemote' could not be found (are you missing a using directive or an assembly reference?)
    3. 7208: [Unity] Packages/com.unity.xr.arfoundation@4.0.12/Runtime/AR/ARMeshManager.cs(18,39): error CS0246: The type or namespace name 'ARFoundationRemote' could not be found (are you missing a using directive or an assembly reference?)
    EDIT: unfortunately un-embedding doesn't solve this - would appreciate your thoughts. I want to do everything I can to respect the TOS and not include the package in version control, but as it currently stands, we cannot deploy our project otherwise... Thank you.
     
    Last edited: Aug 14, 2021
  12. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    If your Perforce project is private (closed-source), then you can push the plugin to a version control system and Cloud Build should work the same way as a local build.

    If you're developing an open-source project, then excluding the embedded AR Foundation package from your repository should do the trick. The plugin modifies a local (embedded) copy of the AR Foundation package so it can work on a local machine in the Editor. If you don't push these modifications to the repository, Cloud Build will take the unmodified version of the package, and a build should succeed.

    Please tell me if this will work for you.
     
  13. michaelybecker

    michaelybecker

    Joined:
    Jun 18, 2014
    Posts:
    19
    Thanks, it's closed source and I can confirm the code won't be externally available - and that I'm the only dev who'll be using the tool, for that matter. (Can confirm this solves the UCB issue as expected.)
    I appreciate your response!
     
    KyryloKuzyk likes this.
  14. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    PLUGIN UPDATE POST #12

    AR Foundation Remote 2.0 is available now on Asset Store!

    As promised in the previous post, a big upgrade is here! Meet AR Foundation Remote 2.0!

    I was working on this big update for the last five months, and it's finally here. This update brings exclusive features aimed to speed up iterations and increase confidence in the quality of AR projects.


    Session Recording and Playback

    Session Recording and Playback feature will allow you to record AR sessions to a file and play them back in the reproducible environment.

    Session recordings feature required a rethink of the whole plugin architecture. I reworked the network and serialization layers from the ground up to support recording and playback of any possible AR setup.

    And because v1.0 and v2.0 share the same codebase, all existing customers of v1.0 will benefit from stability and performance improvements made in the process of making v2.0.

    Features:
    • Record sessions and play them back in the reproducible environment: face tracking, image tracking, plane tracking, touch input, you name it!
    • Fix bugs that occur only under specific conditions. Playing a previously recorded AR session in the reproducible environment will help you track down and fix bugs even faster!
    • Record testing scenarios for your AR app. Your testers don't have to fight over testing devices ever again: record a testing scenario once, then play it back as many times as you want without an AR device.
    • All supported features work together with session record and playback.

    The Session Recording feature comes with a limitation. Your app should be deterministic to play back recorded sessions in a reproducible manner. That is, your AR experience shouldn't rely on framerate-dependent or time-dependent events (animations, timers, ScrollRect moving with inertia, physics simulation, server connection, etc.). Instead, make sure your AR app only responds to AR Foundation and input events to trigger AR functionality while recording or playing back a session.

    For example, the majority of scenes from AR Foundation Samples are deterministic. But ARKit World Map loading is not deterministic because of this framerate-dependent piece of code:
    Code (CSharp):
    1. int bytesPerFrame = 1024 * 10;
    2. var bytesRemaining = file.Length;
    3. var binaryReader = new BinaryReader(file);
    4. var allBytes = new List<byte>();
    5. while (bytesRemaining > 0)
    6. {
    7.     var bytes = binaryReader.ReadBytes(bytesPerFrame);
    8.     allBytes.AddRange(bytes);
    9.     bytesRemaining -= bytesPerFrame;
    10.     yield return null;
    11. }
    We can easily make the code above deterministic in Editor with the following modification:
    Code (CSharp):
    1.  
    2. int bytesPerFrame = Application.isEditor ?
    3.     int.MaxValue : // load the whole file synchronously in the Editor for the code to be framerate-independent
    4.     1024 * 10; // load the file in small pieces to prevent freezes on mobile devices in production
    5.  

    Input System (New) support

    Version 2.0 brings Input System (New) support with all benefits of input events and enhanced touch functionality.
    • Input Remoting is a highly requested feature in the Unity ecosystem. Version 2.0 now allows you to transmit all input events from your AR device back to the Editor. Test Input System multi-touch input right in the Editor!
    • Test Input Actions right in the Editor without making builds.
    • Record all Input System events to a file and play them back in the reproducible environment. Again, all supported features can be recorded and played back!


    ARCore Cloud Anchors

    Testing ARCore Cloud Anchors don't have to be that hard. With a custom fork adapted to work with the AR Foundation Remote 2.0, you can run AR projects with ARCore Cloud Anchors right in the Unity Editor.
    • Host Cloud Anchors.
    • Resolve Cloud Anchors.
    • Record an AR session with ARCore Cloud Anchors and play it back in the reproducible environment.


    What will happen to the existing customers
    of AR Foundation Editor Remote (v1.0)?

    All existing customers will continue to receive bug fixes and the same high-quality support as before. Version 1.0 is not going anywhere and is still an essential AR debugging tool for years to come.

    Existing customers can upgrade to v2.0 anytime by paying only the price difference between the two versions.


     
    Last edited: Jul 14, 2022
    makaka-org likes this.
  15. renukasalayecg

    renukasalayecg

    Joined:
    Jul 8, 2020
    Posts:
    2
    hello i owned the previous version of this plugin before ar foundation remote...i purchased this version 2.0 after importing and clicking on installer, when i try to install the plugin on my device it keeps saying....

    "please re-import the plugin or buy the additional license if you're trying to install the plugin on different development machine"

    i cannot go forward from this step....please any advice?....i removed the previous version of this plugin and did a fresh install in a new project im using unity 2020.3.15f2 URP . The previous plugin works just fine.

    also do u need to uninstall the previous 1.4.11 release 15 plugin before installing v2.0? the instructions are not clear.
     
    Last edited: Aug 16, 2021
  16. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Yes, please uninstall the previous version completely before installing a new one. If a new version doesn't move or remove files, then it's possible to update without uninstalling first. But Unity Package Manager can't tell if anything was moved or removed since the previous version, so the safest way to update is the one that is described in the documentation:
     
  17. renukasalayecg

    renukasalayecg

    Joined:
    Jul 8, 2020
    Posts:
    2
    ok i tried everything above im able to install the companion app on my android, everything working fine in unity editor, but when i launch the companion app, and press play in editor, the app on android keeps crashing...the ip address is correct and every other setting in xr management etc is right. any thoughts?>..im using samsung galaxy S20 ultra note android 11
     
  18. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Could you please send me crash logs from the AR Companion app? You can use ADB or this Unity package to get the logs.
     
  19. Griffo

    Griffo

    Joined:
    Jul 5, 2011
    Posts:
    700
    Just updated to 1.4.11-relawse.16 .. Unity 2020.3.15f2 and are now getting these errors

    Assets/Plugins/ARFoundationRemoteInstaller/Scripts/Editor/ARFoundationRemoteInstaller.cs(52,39): error CS0234: The type or namespace name 'SessionRecordings' does not exist in the namespace 'ARFoundationRemote.Runtime' (are you missing an assembly reference?)

    Assets/Plugins/ARFoundationRemoteInstaller/Scripts/Editor/ARFoundationRemoteInstaller.cs(58,17): error CS0029: Cannot implicitly convert type 'void' to 'bool'

    Assets/Plugins/ARFoundationRemoteInstaller/Scripts/Editor/CompanionAppInstaller.cs(148,30): error CS1061: 'Settings' does not contain a definition for 'inputHandlingData' and no accessible extension method 'inputHandlingData' accepting a first argument of type 'Settings' could be found (are you missing a using directive or an assembly reference?)

    Assets/Plugins/ARFoundationRemoteInstaller/Scripts/Editor/CompanionAppInstaller.cs(148,50): error CS0103: The name 'InputHandlingData' does not exist in the current context

    Any help please. Thanks.
     
  20. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Please remove the old plugin version completely before updating to a new one.
    I recently downgraded the plugin version from 4.11.10 to 1.4.11-release.11 before releasing version 2.0. This makes plugin versioning more logical.
     
  21. creativewax

    creativewax

    Joined:
    Jan 28, 2015
    Posts:
    23
    I have just got the v2 and having a nightmare, using 4.2.0-pre.12 and built the companion app with same settings, and I now have it running, but it always defaults to rear camera and tells me face tracking is not supported, even though I have all the settings setup correctly?

    EDIT

    Got it working, had to downgrade to 4.1.7 verified release and its built and working as expected again now
     
    Last edited: Aug 18, 2021
  22. creativewax

    creativewax

    Joined:
    Jan 28, 2015
    Posts:
    23
    another slight issue can't find Texture2DSerializable.cs that's mentioned in docs? as I can't get this to work with the stencil buffer etc in CPU images
     
    Last edited: Aug 18, 2021
  23. hc4zrr324

    hc4zrr324

    Joined:
    Apr 13, 2021
    Posts:
    5
    Hey @KirillKuzyk, thanks for writing a stellar plugin! Super helpful stuff!

    I'm running into an issue with the companion app on iOS where I get an error in the Unity editor console:

    Code (CSharp):
    1. Client.Send: message too big: 129514397. Limit: 104857600
    2. UnityEngine.Debug:LogError(Object)
    3. ARFoundationRemote.Runtime.<>c:<Awake>b__12_1(String) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/Telepathy/TelepathyConnection.cs:51)
    4. Telepathy.Client:Send(Byte[]) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Telepathy/Client.cs:207)
    5. ARFoundationRemote.RuntimeEditor.TelepathyReceiverConnection:send_internal(Byte[]) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/Telepathy/TelepathyReceiverConnection.cs:65)
    6. ARFoundationRemote.RuntimeEditor.TelepathyReceiverConnection:send(Byte[]) (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/Telepathy/TelepathyReceiverConnection.cs:61)
    7. ARFoundationRemote.Runtime.TelepathyConnection`2:runBackgroundThread() (at Packages/com.kyrylokuzyk.arfoundationremote/Runtime/Connection/Telepathy/TelepathyConnection.cs:108)
    8. System.Threading.ThreadHelper:ThreadStart()
    9.  
    I'm new to Unity & C# but I'm guessing this means AR Foundation Remote is trying to send a message to the Unity editor but there's a limit on the amount of data that can be sent to the Unity editor. I'm using the cabled IP (though this happens with the WiFi IP too) and the following versions according to the text on the AR Foundation Remote screen:

    AR Foundation Remote: 4.7.11
    AR Foundation: 4.0.8
    ARCore XR Plugin: 4.0.8
    ARKit XR Plugin: 4.0.8

    Knowing what AR Foundation Remote is trying to log would be really helpful for debugging my app. Thanks in advance!
     
  24. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    By default, Unity searches only in the Assets folder. Please search for 'Texture2DSerializable' in the 'Packages/AR Foundation Remote' folder.
     
  25. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Please update to the latest version, this issue has already been fixed.
     
  26. BinaryBanana

    BinaryBanana

    Joined:
    Mar 17, 2014
    Posts:
    81
    I am getting this error from AROcclusionManager.TryAcquireEnvironmentDepthCpuImage which I believe should be supported by your plugin. What do I miss?

    Code (CSharp):
    1. InvalidOperationException: Cannot acquire depth image
    2. Cdm.XR.Extensions.ARDensePointCloudManager.Update () (at Assets/..)
    Unity: 2020.3.14f1
    ARF: 4.1.7

    In the OcclusionSubsystemSender you seems to be reading these so maybe I am missing a place where it should be available for me.
     
  27. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Unfortunately, Occlusion CPU Images are not supported. Only GPU occlusion textures are supported at this time. Here is a quote from the Asset Store description:
    I have Occlusion CPU Images on my roadmap. As a temporary solution, you can copy the environmentDepthTexture with the help of Graphics.Blit(), please see Texture2DSerializable.cs for example.
     
  28. dogsaregreat

    dogsaregreat

    Joined:
    Sep 5, 2018
    Posts:
    21
    having issues with the latest 1.4.11-release16.

    every time when i click on"Install AR Companion App" in the past it would build and launch Xcode and install the ARFoundationRemote app like normal. This time it just seems to install whatever scene is currently open in Unity when I click on it...
     
  29. dogsaregreat

    dogsaregreat

    Joined:
    Sep 5, 2018
    Posts:
    21
    temporarily resolved this issue by first opening the ARCompanion scene and then clicking "Install AR Companion App". I thought "Install AR Companion App" step would be enough. Kind of annoying as it took me 3 hours to reinstalling the plugin thinking something went wrong...
     
  30. dogsaregreat

    dogsaregreat

    Joined:
    Sep 5, 2018
    Posts:
    21
    Is there a reason why this was added as a dependency into the manifest.json?

    Code (CSharp):
    1. "com.kyrylokuzyk.arfoundationremote": "file:../Assets/Plugins/ARFoundationRemoteInstaller/com.kyrylokuzyk.arfoundationremote.tgz",
    I don't think my manifest.json was updated before with this. Because we added
    Packages/com.kyrylokuzyk.arfoundationremote into our gitignore to prevent the plugin from being uploaded onto git, it's now causing errors when Unity cloud triggers.

    Will our manifest.json always be updated with this dependency?
     
  31. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Hmm, strange. The installer uses BuildPipeline to deploy the AR Companion app and explicitly specifies the scene that should be used in a build. If the scene is not found, the installer should throw an exception and cancel the installation instead of picking the currently open scene. Please try to uninstall the plugin completely, re-install the latest version and tell me if the issue still persists.

    Previously, the plugin installer moved the source code folder to the Packages folder. When a package is placed at the Packages folder, Unity Package Manager will import the package, but will not reflect this in the manifest.json. This installation method worked ok, but sometimes caused compilation errors when importing from Asset Store in some versions of Unity.

    Now the plugin installs itself from the tarball package. This method never causes compilation issues and is generally a more correct way to install custom packages. Also, it's much quicker.

    Answering your question, this dependency has always been there but just wasn't reflected in the manifest.json. The new installation method via tarball just shows this dependency explicitly.
     
  32. dogsaregreat

    dogsaregreat

    Joined:
    Sep 5, 2018
    Posts:
    21
    Still persists in that i need to load up the ARCompanion then click "Install..' to ensure it is loaded on my device. Did a complete removal and re-embedding.

    Here's the error message I get when I try a Unity cloud build w/ the manifest.json line that has
    "com.kyrylokuzyk.arfoundationremote": "file:../Assets/Plugins/ARFoundationRemoteInstaller/com.kyrylokuzyk.arfoundationremote.tgz",

    Code (CSharp):
    1.  
    2. 163: [Unity] An error occurred while resolving packages:
    3. 164: [Unity]   Project has invalid dependencies:
    4. 165: [Unity]     com.kyrylokuzyk.arfoundationremote: The file [/BUILD_PATH/PROJECTNAME/Assets/Plugins/ARFoundationRemoteInstaller/com.kyrylokuzyk.arfoundationremote.tgz/package.json] cannot be found
    5. 166: [Unity] A re-import of the project may be required to fix the issue or a manual modification of /BUILD_PATH/PROJECTNAME/Packages/manifest.json file.
    6. 167:        Unity ran in '79.023162' seconds
    7. 168:        ! Unity player export failed!
    8. 169: ! build of 'PROJECTNAME' failed. exit
    Removing
    "com.kyrylokuzyk.arfoundationremote": "file:../Assets/Plugins/ARFoundationRemoteInstaller/com.kyrylokuzyk.arfoundationremote.tgz", I am able to build without issues.

    However, because I remove this, i'll have to reinstall the plugin again in my project to have it functional and then when i push to my master, I need to make sure I remove it from the manifest.json to ensure cloud build works.

    Is there any other alternative to my process so i don't need to remove this line when doing a Unity Cloud built and reinstalling the plugin whenever I want to dev on my workstation?
     
    Last edited: Aug 20, 2021
  33. OneEyeOnly

    OneEyeOnly

    Joined:
    Sep 16, 2020
    Posts:
    17
    AR Foundation Remote + Unity 2021.2 beta installation errors.
    I tried multiple times, all end up the same ambiguous reference error . Please help
    upload_2021-8-20_22-25-51.png
     
  34. Griffo

    Griffo

    Joined:
    Jul 5, 2011
    Posts:
    700
    Can it be run by connecting the phone to the Mac and not using WiFi ? as it's a bit laggy through WiFi?

    Or what's the best way to get it to run smoother? Or is version 2 smoother?

    Thanks.
     
    Last edited: Aug 20, 2021
  35. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Could you please tell me your setup?
    1. Unity version. Windows or macOS?
    2. Send me your Packages/manifest.json file and Packages/packages-lock.json files.

    If your repository is closed-source, please commit the contents of ARFoundationRemoteInstaller folder and cloud build should work.

    In case if your repository is open-source, things are getting a bit trickier:
    1. Either don't commit the manifest.json dependency to the repository.
    2. Or embed the plugin source code inside the Packages folder, remove the dependency from manifest.json and ignore the embedded plugin folder.
     
  36. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    The beta version of Unity 2021.2 is not currently supported, please install a stable version of Unity 2019.2 or newer.
    I fixed the NotNull attribute error internally, but, unfortunately, this is not enough to support Unity 2021.2: data serialization/deserialization is not working properly in this version.
     
  37. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Yes, it's possible to connect the AR Companion app to Editor by wire, here is a quote from the documentation:
    Also, you can reduce the network load by decreasing the quality and framerate of camera video in the plugin's settings.
    v1.0 and v2.0 share the same codebase, so they are equal in terms of network performance.
     
    Last edited: Aug 23, 2021
    Griffo likes this.
  38. ibompuis

    ibompuis

    Joined:
    Sep 13, 2012
    Posts:
    100
    Hi @KirillKuzyk
    I found your script on another thread to add gameobject on mesh but I try to keep the previous gameobject and not delete it. How to do that ?
    Code (CSharp):
    1. public class PlacePrefabOnMesh : MonoBehaviour {
    2.         [SerializeField] ARMeshManager meshManager = null;
    3.         [SerializeField] ARSessionOrigin origin = null;
    4.         [CanBeNull] [SerializeField] GameObject optionalPointerPrefab = null;
    5.         [SerializeField] bool disableObjectsOnTouchEnd = false;
    6.         readonly Dictionary<int, Transform> pointers = new Dictionary<int, Transform>();
    7.         void Update() {
    8.             for (int i = 0; i < Input.touchCount; i++) {
    9.                 var touch = Input.GetTouch(i);
    10.                 var pointer = getPointer(touch.fingerId);
    11.                 var touchPhase = touch.phase;
    12.                 if (touchPhase == TouchPhase.Ended || touchPhase == TouchPhase.Canceled) {
    13.                     if (disableObjectsOnTouchEnd) {
    14.                         pointer.gameObject.SetActive(false);
    15.                     }
    16.                 } else {
    17.                     var ray = origin.camera.ScreenPointToRay(touch.position);
    18.                     var hasHit = Physics.Raycast(ray, out var hit, float.PositiveInfinity, 1 << meshManager.meshPrefab.gameObject.layer);
    19.                     if (hasHit) {
    20.                         pointer.position = hit.point;
    21.                         pointer.rotation = Quaternion.FromToRotation(Vector3.up, hit.normal);
    22.                     }
    23.                     //pointer.gameObject.SetActive(hasHit);
    24.                 }
    25.             }
    26.         }
    27.         Transform getPointer(int fingerId) {
    28.             if (pointers.TryGetValue(fingerId, out var existing)) {
    29.                 return existing;
    30.             } else {
    31.                 var newPointer = createNewPointer();
    32.                 pointers[fingerId] = newPointer;
    33.                 return newPointer;
    34.             }
    35.         }
    36.         Transform createNewPointer() {
    37.             var result = instantiatePointer();
    38.             Assert.AreNotEqual(result.gameObject.layer, meshManager.meshPrefab.gameObject.layer, "Pointer layer should not be the same as the mesh prefab layer");
    39.             result.parent = transform;
    40.             return result;
    41.         }
    42.         Transform instantiatePointer() {
    43.             if (optionalPointerPrefab != null) {
    44.                 return Instantiate(optionalPointerPrefab).transform;
    45.             } else {
    46.                 var result = GameObject.CreatePrimitive(PrimitiveType.Sphere).transform;
    47.                 result.localScale = Vector3.one * 0.05f;
    48.                 result.gameObject.layer = LayerMask.NameToLayer("Ignore Raycast");
    49.                 return result;
    50.             }
    51.         }
    52.     }
    Best,
     
  39. ardhies

    ardhies

    Joined:
    Sep 19, 2020
    Posts:
    1

    in this new version, support manomotion SDK input ?
     
  40. keil_unity

    keil_unity

    Joined:
    Mar 2, 2021
    Posts:
    28
    hello i keep receiving cast exceptions and cannot seem to use this plugin anymore. I have tried uninstalling and reinstalling and going into the code and fixing the cast myself but to to luck anywhere


    Code (CSharp):
    1. nvalidCastException: Specified cast is not valid.
    2. ARFoundationRemote.Runtime.TelepathyConnection.processIncomingMessages () (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@7bfb4d405d4a-1629202539000/Runtime/Connection/TelepathyConnection.cs:98)
    3. ARFoundationRemote.Runtime.TelepathyConnection+<processIncomingMessagesCor>d__14.MoveNext () (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@7bfb4d405d4a-1629202539000/Runtime/Connection/TelepathyConnection.cs:80)
    4. UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at <ef667cdc82bf4a4e911d1ddea9ff581d>:0)
    5. UnityEditor.PlayModeView:RenderView(Vector2, Boolean)
    6. Unity.DeviceSimulator.SimulatorWindow:OnGUI() (at Library/PackageCache/com.unity.device-simulator@2.2.3-preview/Editor/SimulatorWindow.cs:134)
    7. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&)
     
  41. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Code (CSharp):
    1. void Update() {
    2.     for (int i = 0; i < Input.touchCount; i++) {
    3.         var touch = Input.GetTouch(i);
    4.         if (touch.phase == TouchPhase.Began) {
    5.             var ray = origin.camera.ScreenPointToRay(touch.position);
    6.             var hasHit = Physics.Raycast(ray, out var hit, float.PositiveInfinity, 1 << meshManager.meshPrefab.gameObject.layer);
    7.             if (hasHit) {
    8.                 var pointer = createNewPointer();
    9.                 pointer.position = hit.point;
    10.                 pointer.rotation = Quaternion.FromToRotation(Vector3.up, hit.normal);
    11.             }
    12.         }
    13.     }
    14. }
     
    ibompuis likes this.
  42. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    I haven't tested my plugin with Manomotion SDK yet. But my plugin supports accessing the camera image on CPU via AR Foundation API, so it should be relatively easy for Manomotion developers to run their SDK inside the Editor.
    If you are somehow related to the Manamotion team, please let me know, I would love to help them in the integration with my plugin.
     
  43. hc4zrr324

    hc4zrr324

    Joined:
    Apr 13, 2021
    Posts:
    5
    Hey @KirillKuzyk,

    I took your advise and upgraded the app to 2.0 and followed your install instructions:

    How to update the plugin to a newer version?
    1. Press the 'Installer/Un-install Plugin' button twice.
    2. Delete the folder 'Assets/Plugins/ARFoundationRemoteInstaller'.
    3. Restart Unity Editor.
    4. Import new plugin version.
    5. Make a new build of the AR Companion app by pressing 'Installer/Install AR Companion App' button.

    I'm now getting the following error on my iPhone XR:



    Any advise appreciated! Thanks!
     
  44. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    Hmm, I can think of only one setup that can cause this error: if your Image Tracking library size is more than 100mb.
    Could you please confirm it? I'll increase the max message size in the next plugin version if this is the case.
     
  45. OneEyeOnly

    OneEyeOnly

    Joined:
    Sep 16, 2020
    Posts:
    17
    Thank you for the explanation. When will it support Unity 2021.2?
     
  46. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    I think the serialization issue is related to this specific beta version of Unity and the Unity team will fix it till the official release. In any case, I'll ensure the plugin works with Unity 2021.2 before it will go out of beta.
     
  47. KyryloKuzyk

    KyryloKuzyk

    Joined:
    Nov 4, 2013
    Posts:
    1,128
    This exception can appear if you didn't make a new AR Companion app build after updating the plugin.
    Please try to make a new AR Companion build and tell me if this will fix the issue.
     
  48. hc4zrr324

    hc4zrr324

    Joined:
    Apr 13, 2021
    Posts:
    5
    I'm not 100% sure how to find out the exact size of the image library at runtime as it's being built at runtime from a folder of cached jpgs. However this folder has about 26 images totaling at about 11mb so I'd be surprised if that turned into 100mb somehow during runtime.
     
  49. hc4zrr324

    hc4zrr324

    Joined:
    Apr 13, 2021
    Posts:
    5
    Also there's the following errors in the Unity console:

    Code (CSharp):
    1. ThreadAbortException
    2. System.String.memset (System.Byte* dest, System.Int32 val, System.Int32 len) (at <9577ac7a62ef43179789031239ba8798>:0)
    3. UnityEngine.StackTraceUtility.ExtractFormattedStackTrace (System.Diagnostics.StackTrace stackTrace) (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/StackTrace.cs:110)
    4. UnityEngine.StackTraceUtility.ExtractStackTrace () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/StackTrace.cs:35)
    5. UnityEngine.Debug:Log(Object)
    6. Telepathy.Common:SendMessagesBlocking(NetworkStream, Byte[][]) (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/Runtime/Connection/Telepathy/Common.cs:110)
    7. Telepathy.Common:SendLoop(Int32, TcpClient, SafeQueue`1, ManualResetEvent) (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/Runtime/Connection/Telepathy/Common.cs:259)
    8. Telepathy.Client:<ReceiveThreadFunction>b__10_0() (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/Runtime/Connection/Telepathy/Client.cs:76)
    9. System.Threading.ThreadHelper:ThreadStart()
    10.  
    11.  
    12. Send: stream.Write exception: System.IO.IOException: Unable to write data to the transport connection: The socket has been shut down. ---> System.Net.Sockets.SocketException: The socket has been shut down
    13.   at System.Net.Sockets.Socket.Send (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Net.Sockets.SocketFlags socketFlags) [0x00016] in <ef151b6abb5d474cb2c1cb8906a8b5a4>:0
    14.   at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 size) [0x0009b] in <ef151b6abb5d474cb2c1cb8906a8b5a4>:0
    15.    --- End of inner exception stack trace ---
    16.   at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 size) [0x000e2] in <ef151b6abb5d474cb2c1cb8906a8b5a4>:0
    17.   at Telepathy.Common.SendMessagesBlocking (System.Net.Sockets.NetworkStream stream, System.Byte[][] messages) [0x000d7] in /Users/ceto/Dropbox/_CLIENTS/Amplifier.org/unity app 10/Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/Runtime/Connection/Telepathy/Common.cs:103
    18.  
    19.  
    20. AR Foundation Remote: Editor lost connection with AR Companion app. Please restart Editor scene.
    21. UnityEngine.Debug:LogError(Object)
    22. ARFoundationRemote.RuntimeEditor.Receiver:ARFoundationRemote.Runtime.IConnectionDelegate.OnDisconnected() (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/RuntimeEditor/Receiver/Receiver.cs:78)
    23. ARFoundationRemote.Runtime.TelepathyConnection:processIncomingMessages() (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/Runtime/Connection/TelepathyConnection.cs:112)
    24. ARFoundationRemote.Runtime.<processIncomingMessagesCor>d__14:MoveNext() (at Library/PackageCache/com.kyrylokuzyk.arfoundationremote@c9f1e958af64-1629203936000/Runtime/Connection/TelepathyConnection.cs:80)
    25. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr) (at /Users/bokken/buildslave/unity/build/Modules/IMGUI/GUIUtility.cs:197)
    26.  
     
  50. rhikos

    rhikos

    Joined:
    Sep 5, 2012
    Posts:
    2
    Hi @KirillKuzyk

    I have a trouble with installation of AR Foundation Remote.

    - errors:

    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Examples/CpuImagesExample.cs(59,49): error CS1061: 'AROcclusionManager' does not contain a definition for 'TryAcquireSmoothedEnvironmentDepthCpuImage' and no accessible extension method 'TryAcquireSmoothedEnvironmentDepthCpuImage' accepting a first argument of type 'AROcclusionManager' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Examples/CpuImagesExample.cs(61,49): error CS1061: 'AROcclusionManager' does not contain a definition for 'TryAcquireRawEnvironmentDepthCpuImage' and no accessible extension method 'TryAcquireRawEnvironmentDepthCpuImage' accepting a first argument of type 'AROcclusionManager' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(83,154): error CS1061: 'AROcclusionManager' does not contain a definition for 'TryAcquireSmoothedEnvironmentDepthCpuImage' and no accessible extension method 'TryAcquireSmoothedEnvironmentDepthCpuImage' accepting a first argument of type 'AROcclusionManager' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(83,79): error CS1061: 'XROcclusionSubsystemDescriptor' does not contain a definition for 'environmentDepthImageSupported' and no accessible extension method 'environmentDepthImageSupported' accepting a first argument of type 'XROcclusionSubsystemDescriptor' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(83,114): error CS0103: The name 'Supported' does not exist in the current context
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(85,154): error CS1061: 'AROcclusionManager' does not contain a definition for 'TryAcquireRawEnvironmentDepthCpuImage' and no accessible extension method 'TryAcquireRawEnvironmentDepthCpuImage' accepting a first argument of type 'AROcclusionManager' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(85,79): error CS1061: 'XROcclusionSubsystemDescriptor' does not contain a definition for 'environmentDepthImageSupported' and no accessible extension method 'environmentDepthImageSupported' accepting a first argument of type 'XROcclusionSubsystemDescriptor' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(85,114): error CS0103: The name 'Supported' does not exist in the current context
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(87,79): error CS1061: 'XROcclusionSubsystemDescriptor' does not contain a definition for 'environmentDepthConfidenceImageSupported' and no accessible extension method 'environmentDepthConfidenceImageSupported' accepting a first argument of type 'XROcclusionSubsystemDescriptor' could be found (are you missing a using directive or an assembly reference?)
    Library/PackageCache/com.kyrylokuzyk.arfoundationremote@bd3ae84aa361-1629980104000/Runtime/Sender/CpuImagesSender.cs(87,124): error CS0103: The name 'Supported' does not exist in the current context


    - Unity 2020.3.16f1
    - AR Foundation 4.1.7
    - ARCore XR Plugin 4.1.7
    - ARKit Face Tracking 4.1.7
    - AR Foundation Remote 2.0.3

    I would be grateful if you could give me some advice.
     
    AlwayzPatrick likes this.