Search Unity

AR Remoting & Simulation

Discussion in 'Handheld AR' started by mfuad, Aug 1, 2019.

  1. mfuad

    mfuad

    Unity Technologies

    Joined:
    Jun 12, 2018
    Posts:
    18
    Update: December 5, 2019
    Hi everyone, an update on where we are. We aimed to have an initial version of remoting for ARKit and ARCore devices by the end of 2019, but this work is still in development and has been pushed out to early 2020. In the meantime, feel free to complete this form if you are interested in early access when it’s available.

    Latest information on AR Foundation can be found in our session at Unite Copenhagen.

    August 1, 2019

    Hi everyone, there have been a lot of conversations around reducing iteration time when working with AR Foundation. We owe the community an update and in an effort to improve our communication and transparency, we have created a dedicated post to keep you updated as we progress. You can expect more details to be posted on this thread moving forward.

    It’s clear from many of you that iteration time continues to be a significant pain point. Remoting is the most requested feature and we are currently solidifying our plan to ensure we have the tools that fit your needs. At a high level, we plan to include the following features:

    Remoting & Simulation:
    • Connect to a remote device from the editor
    • Test AR features (planes, raycast, reference points, etc.) on-device without having to build your app
    • Record sessions and play them back in the editor
    • Simulate AR features (planes, raycast, reference points, etc.) in the editor
    This is a large, comprehensive project that will take time and continue to evolve as new AR features are supported. However, we want to make sure you have the proper tools to test features you use the most, as soon as possible. We are working on this now, so stay tuned for more updates.
     
    Last edited: Dec 5, 2019
  2. Superdeath

    Superdeath

    Joined:
    Nov 6, 2013
    Posts:
    2
    Thanks for the update! Question about recorded sessions - is the idea that the raw video feed and full suite of sensor data will be recorded for playback and simulation? Would we basically be able to launch the recorded video + sensor data from editor and otherwise interact with the app via UI or other events normally?
     
  3. Dark_Swordsman

    Dark_Swordsman

    Joined:
    Dec 4, 2018
    Posts:
    27
    Thanks for the update. Not sure how far you guys are but it would be great if you could push out a preview as soon as possible. The recording feature is nice but a working preview of just the remoting (incl. raycasts) would already help tremendously.
     
    henriqueranj, R0man, jotamaza and 6 others like this.
  4. juliodutracora

    juliodutracora

    Joined:
    Jun 11, 2019
    Posts:
    16
    Agree.
     
  5. BuoDev

    BuoDev

    Joined:
    Nov 28, 2018
    Posts:
    45
    I basically just want to see changes happening in the editors hierarchy and to check variables in the inspector. Would help a lot to understand how AR Foundation works.
     
    BBIT-SOLUTIONS likes this.
  6. Hugoliv

    Hugoliv

    Joined:
    Dec 8, 2017
    Posts:
    2
    watching this.
     
  7. UniTaz

    UniTaz

    Joined:
    Dec 4, 2017
    Posts:
    10
    Looking forward to it too. Thanks mfuad for the update!
     
  8. adrian-taylor09

    adrian-taylor09

    Joined:
    Dec 22, 2016
    Posts:
    21
    Awesome that you guys are working on a solution. I’m really looking forward to the in-editor simulation of AR features as I can see this being useful for my projects.

    Is there any information you can give us about expected timeframes?
     
    cabrera-juan likes this.
  9. vincentfretin

    vincentfretin

    Joined:
    Jul 4, 2019
    Posts:
    6
    @BuoDev this may interest you, Satwant Singh published this video just a few days ago:
     
    sskenth and BuoDev like this.
  10. adrian-taylor09

    adrian-taylor09

    Joined:
    Dec 22, 2016
    Posts:
    21
    @vincentfretin Thanks for the video. Those are some really helpful assets. The creator of the Runtime Inspector asset has some really useful Unity tools (including an in game console tool) his GitHub page is worth a visit for sure

    https://github.com/yasirkula
     
  11. MrDos1

    MrDos1

    Joined:
    Dec 14, 2016
    Posts:
    19
    Hi! Thanks for the update, this was much needed.

    Although I greatly disagree with the approach. We know that when you announce to be willing to be more transparent but don't give any estimated release date, it's most likely because the development didn't start yet.
    I have the feeling this feature is passed beyond due by now, and is a good candidate for the new "nested prefab" type of feature, where everyone says it's gonna be soon but it takes years.

    Now there's an opportunity to change that, please release a WIP as soon, that will work for few people, help them out, then expend the support one by one instead of adding all the features at beforehand.
    Helping 10 people is still better than not helping.
    We want a preview package. We want a pre-alpha, a early access, whatever. Anything will be better than nothing, to solve your transparency problem.
     
    R0man and naiqus like this.
  12. Bastienre4

    Bastienre4

    Joined:
    Jul 8, 2014
    Posts:
    73
    Hello ! I'm also starting to play around with AR and not having this feature is a real bummer for newcomers. Please keep us informed of the current state of this solution ;).
     
  13. unnanego

    unnanego

    Joined:
    May 8, 2018
    Posts:
    119
    8th Wall have had AR Remote for more than a year...
     
    pwshaw and naokiijobs like this.
  14. riccardogiorato

    riccardogiorato

    Joined:
    Aug 26, 2017
    Posts:
    2
    Any news on this project @mfuad ?
    Is it possible to be part of a pre-alpha of AR Foundation Remote?
     
    unnanego likes this.
  15. Shizola

    Shizola

    Joined:
    Jun 29, 2014
    Posts:
    215
    I'm new to AR and really surprised this isn't already a thing. The remote tool shown here
    is one the way or the one that doesn't work anymore? Must be a real pain to make a non trivial AR app without these kind of tools? Or is everyone just using another solution like 8th wall?
     
  16. DrSharky

    DrSharky

    Joined:
    Dec 7, 2016
    Posts:
    8
    No updates...
    I understand that the original post says it's a large feature and will take time. I get that the big new updates to AR Foundation might make it more difficult, and that it's a help for them to implement this in the first place.

    Making the thread, and then not posting any updates in over a month though. Why make the thread in the first place?
     
    Tarrag and unnanego like this.
  17. Dy5funct10n

    Dy5funct10n

    Joined:
    Feb 2, 2018
    Posts:
    3
    They wanted to be more transparent, but they've gone completely invisible :(
     
  18. chilipino

    chilipino

    Joined:
    Mar 27, 2019
    Posts:
    2
    Has anyone here attempted to create a remote app for AR Foundation? I'm curious to see if it'd be relatively straightforward port given that we have access to the ARKit Remote source.
     
  19. MaxXR

    MaxXR

    Joined:
    Jun 18, 2017
    Posts:
    67
    Hi @mfuad
    can you please release even alpha for this because i'd really help us save time?
    Dev AR atm with unity is painful :/
     
    mranger and Tarrag like this.
  20. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    503
    +1
     
    mranger and Tarrag like this.
  21. StanK_

    StanK_

    Joined:
    Dec 5, 2017
    Posts:
    6
    +1
     
    mranger and Tarrag like this.
  22. thibautvdumont

    thibautvdumont

    Joined:
    Oct 6, 2015
    Posts:
    21
    To those who wonder why the community hasn't done anything about this, it's simple : closed sources. Considering Unity's build iteration time, ARFoundation is basically unusable as it is for any AR project that goes a bit beyond a table top game. I'm stuck with it atm, but I really regret upgrading from the ARKit plugin.
     
    distastee, AdrienMgm and henriqueranj like this.
  23. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,294
    It seems like things have got a lot harder with AR since I was last using it over a year ago :(
    It's pretty frustrating to watch the keynote where they are happily making AR projects with the greatest of ease. Cut to the real world and it's nothing like that.
     
  24. rsodre

    rsodre

    Joined:
    May 9, 2012
    Posts:
    190
    I remember seeing a Unity video with remoting a long time ago, and now that I'm building my first AR app, I was like really? Not available yet?

    +1
     
  25. MaxXR

    MaxXR

    Joined:
    Jun 18, 2017
    Posts:
    67
    newguy123 likes this.
  26. henriqueranj

    henriqueranj

    Joined:
    Feb 18, 2016
    Posts:
    143
    In the Unity Roadmap @ Unite Copenhagen 2019 slides (https://on.unity.com/Unite2019RoadmapTalk), in page 62, it states that ARFoundation "On device Remoting (Preview)" will be launched under 2020.1.

    This will mean a whole lot more of waiting... for a preview. Or am I seeing this wrong and this is another sort of Remoting?
     
    unnanego, PTW_AR and DrSharky like this.
  27. henriqueranj

    henriqueranj

    Joined:
    Feb 18, 2016
    Posts:
    143
    unnanego, pwshaw, Tarrag and 3 others like this.
  28. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    503
    most of us don't need a complete and amazing solution... just something dirty we can test with...
     
    Burkard, bsizzle, Thimo_ and 9 others like this.
  29. Tarrag

    Tarrag

    Joined:
    Nov 7, 2016
    Posts:
    163
  30. unnanego

    unnanego

    Joined:
    May 8, 2018
    Posts:
    119
  31. StefanoCecere

    StefanoCecere

    Joined:
    Jun 10, 2011
    Posts:
    160
    henriqueranj and Tarrag like this.
  32. asus4

    asus4

    Joined:
    Dec 7, 2012
    Posts:
    3
  33. Thimo_

    Thimo_

    Joined:
    Aug 26, 2019
    Posts:
    2
    Hi @mfuad , Can you share any progress on the AR Foundation remote tool? Maybe an estimated time until release of the Alpha version?
     
    foshe_5 likes this.
  34. StefanoCecere

    StefanoCecere

    Joined:
    Jun 10, 2011
    Posts:
    160
    thank you @asus4 i'm trying your package tomorrow... very happy to see you here ;)
     
  35. Mal_Duffin

    Mal_Duffin

    Joined:
    Jan 22, 2015
    Posts:
    45
    @ASUSasus1234 I tried adding the info to the manifest.json file ( Unity 2019.2.11f1 ), it's giving the following error.

    upload_2019-11-4_16-4-12.png

    Here's where I pasted the code, I also put it in different locations, but the error about 12:3 above stayed the same.
    When I replaced "https://github.com/asus4/ARKitStreamer.git#upm" with "1.0.0" as a test, the error above went away but I got a message saying that it couldn't find that version of the package.

    Code (JavaScript):
    1. {
    2.   "scopedRegistries": [
    3.     {
    4.       "name": "Keijiro",
    5.       "url": "https://registry.npmjs.com",
    6.       "scopes": ["jp.keijiro"]
    7.     }
    8.   ],
    9.   "dependencies": {
    10.     "jp.keijiro.klak.ndi": "0.2.3",
    11.     "com.koki-ibukuro.arkitstream": "https://github.com/asus4/ARKitStreamer.git#upm",
    12.     "com.unity.2d.sprite": "1.0.0",
    13.     "com.unity.2d.tilemap": "1.0.0",
     
  36. asus4

    asus4

    Joined:
    Dec 7, 2012
    Posts:
    3
  37. mistergreen2016

    mistergreen2016

    Joined:
    Dec 4, 2016
    Posts:
    156
    Remote doesn't work with ARFoundation 3.0
     
  38. Hypertectonic

    Hypertectonic

    Joined:
    Dec 16, 2016
    Posts:
    27
    Bumping. Is there any progress on this? Developing with ARfoundation is a pain with the constant building...
     
  39. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    503
    another year trundles by with no simple standalone remoting tool...
     
  40. unnanego

    unnanego

    Joined:
    May 8, 2018
    Posts:
    119
    the building is not the issue - not seeing the scene is the pain!
     
    KarlMoeller likes this.
  41. Tech-Unity3d

    Tech-Unity3d

    Joined:
    Apr 25, 2016
    Posts:
    2
    I decide to move on AR core SDK.
     
  42. mistergreen2016

    mistergreen2016

    Joined:
    Dec 4, 2016
    Posts:
    156
    Too bad you're leaving behind the paying iOS demographic.
     
  43. almirb

    almirb

    Joined:
    Oct 1, 2015
    Posts:
    6
  44. mfuad

    mfuad

    Unity Technologies

    Joined:
    Jun 12, 2018
    Posts:
    18
    Please refer to the top of the thread for an update on where we are.
     
  45. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    Man, its unbelievable that after having the feature in 2017, and the ability to quickly iterate in editor, that now close to 3 yrs after killing it there is STILL no in editor testing.

    Guys, its a camera, that raycasts towards planes. How freaking hard can it be to map a camera to some RTS style controls, and allow us to spawn debug planes at 0,0,0?? That solves the 90% use case, unblocks developers, and actually makes it feasible to create AR apps in unity.

    I get that theres lots of advanced features you want to support, but for the love of god, please DO FIRST THINGS FIRST. Get the core AR functionality of planes/camera/raycast features (that have existed for 5 years)working in editor, before you add a single new feature. Enough is enough, this is insanity.

    You guys really need to do a better job of eating your own dog food here. If you assigned a team, to make a production AR game, they would've come back after 2 weeks, and explained how ridiculously inefficient it is to build to device each time. That really should've happened in 2017.

    It feels more like you guys are developing this for yourselves, as a fun technical experiment, than actually giving a S*** about the developers and whether we can actually use this thing to make a living. 3 YEARS!!?!
     
    Last edited: Dec 18, 2019
    mranger, MSFX, itiz_steffend and 4 others like this.
  46. JakHussain

    JakHussain

    Joined:
    Oct 20, 2016
    Posts:
    143
    I'm just gonna go ahead and assume the team assigned to AR developments in Unity is way too small for their workload. I'm aware they're already working with Google to support the new depth API announced for AR Core and then there's DOTS support as well as Tiny support as well as HoloLens and LeapMotion support for AR foundation, all of which I think would be a higher priority than remoting because Unity's biggest strength is being first to market with features like these.

    They must really have their hands full because it doesn't seem like other teams in Unity have this much of a diverse workload on.
     
    IntegralAR likes this.
  47. almirb

    almirb

    Joined:
    Oct 1, 2015
    Posts:
    6
    Remoting is the last thing I'm waiting for doing a deep dive inside of AR Foundation. But this wait is making me sad. No one deservers an every-time compile to see the changes. :oops:
     
    ina, mariosgakuto, unnanego and 2 others like this.
  48. FUBAR2077

    FUBAR2077

    Joined:
    Nov 16, 2013
    Posts:
    30
    +1 this build times makes AR Projects much more costly for everyone! if you want for AR to expand this is critical!

    give us alpha/beta package at least! for most people simple plane finding/raycasting will be enough
     
    unnanego likes this.
  49. Marebiz

    Marebiz

    Joined:
    Jun 26, 2018
    Posts:
    1
    please ;(
     
    unnanego likes this.
  50. MSFX

    MSFX

    Joined:
    Sep 3, 2009
    Posts:
    115
    main issue in delays is likely the team dissolving and being rebuilt, quite a few of the main contributors to the AR libs coming out of Unity that I know have all left in the last year..

    Couldn't agree more with the notion of just ship a basic thing to start with, stop trying to build GTA IX in such a fast moving industry
     
    FUBAR2077, unnanego and ina like this.
unityunity