Search Unity

3D Reconstruction for ARCore [RELEASED]

Discussion in 'Assets and Asset Store' started by lvonasek, Jan 30, 2019.

  1. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    Hi everyone!

    I'm happy to announce that my 3D Reconstruction tool for Android is now available as a Unity plugin, any Android smartphone that supports ARCore (https://developers.google.com/ar/discover/supported-devices) can be turned into a portable 3D scanner that works in real-time using your normal phone camera, works out of the box with Augmented Reality as well, allowing to place virtual objects more accurately than ever before as you're dealing with the REAL geometry and occlusions of your environment rather than generic virtual planes and walls.

    You can get it on the Asset Store right here: https://assetstore.unity.com/packag...reconstruction-for-arcore-android-only-136919 This is my life project and I've been working on it for quite some time, at the time I couldn't find anything like this so I decided to make my own, I believe it's the first of it's kind that comes with a Unity plugin for it, making it very suitable for AR applications and gaming, no more virtual characters clipping or floating through walls and objects!

    I'm currently working on version 1.1, so stay tuned! I'm very curious to see what you can create with it, feel free to share your results here, if you find any problems or have any specific requests, please let me know!
     
    AndroYD and Circool like this.
  2. facebookprofile1

    facebookprofile1

    Joined:
    Jan 31, 2019
    Posts:
    2
    Thanks! It looks great!

    Does your asset store contains the code which is converting feature points into depth map, before it's passed into Tango or it's all wrapped and hidden in the liboc.aar? I would like to buy it and use it but I need to have a full control over the code in case I will have to make any modifications.

    In the package preview I only see ReconstructionManager.cs which could be what I am looking for.
     
  3. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    The conversion from feature points into depthmaps is done inside liboc.aar.

    I am willing to sell you complete source code for $6000 on invoice (except Tango3DR library as I do not have the code of it). If you have an interest then contact me on tridosm@gmail.com.

     
    Last edited: Mar 11, 2019
  4. JediNizar

    JediNizar

    Joined:
    Nov 13, 2016
    Posts:
    43
    wow this is one of the greatest thing I've seen the interaction with the real world is something I've been looking for for quite some time.
    definitely on my wish list and as soon as I finish with my current project . I'll go for it.
     
  5. EstudioVR

    EstudioVR

    Joined:
    Jul 26, 2013
    Posts:
    54
    I am interested in tha plane detection funcion, Do you have an apk for tests?
     
  6. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    The example scenes are avaible here as APKs:
    https://drive.google.com/open?id=1TjKrcyXdbwkOMdc0QoNWzoIRohkBYCcE

    Note that numbers in APK name means plugin version 10 is 1.0, 11 is 1.1.
    Version 1.1 will be uploaded next week into Asset Store and it will be bring better performance.

     
  7. EstudioVR

    EstudioVR

    Joined:
    Jul 26, 2013
    Posts:
    54
    Thank you. I ill try
     
  8. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    The plugin update 1.1 has been uploaded and it should be available during the next week.


    The updated demo APKs are here: https://drive.google.com/open?id=1TjKrcyXdbwkOMdc0QoNWzoIRohkBYCcE
    The link to Unity Asset Store: https://assetstore.unity.com/packag...reconstruction-for-arcore-android-only-136919

    Here is the changelog:
    Meshing
    -better handling of pausing/resuming meshing
    -bug fixing
    -color per vertex added
    -faster walls estimation

    New functions
    -example how to render camera texture on mesh only
    -possibility to store the mesh as OBJ
    -support for Unity 2017.4 and higher
     
    facebookprofile1 likes this.
  9. facebookprofile1

    facebookprofile1

    Joined:
    Jan 31, 2019
    Posts:
    2
    Thanks for sharing an update. I tried the app, 3dr mesh builder 1.1 and I feel like meshing from your previous app, "3D scanner" works much better, covers more area and in a better way. Is it different implementation? I am using the same phone so I would expect similar results.
     
  10. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    It is slower because of vertex coloring, you can disable it reconstruction prefab then it should behave as in previous version
     
  11. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    To the topic performance - I did a lot of improvements which speeds up meshing. I plan to release it as the version 1.2 before the end of February.
     
  12. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    AndroYD likes this.
  13. AndroYD

    AndroYD

    Joined:
    Nov 3, 2014
    Posts:
    25
    You're truly a miracle worker, this update is amazing! It opens the door to even more new possibilities now!
     
  14. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    Thanks, if you create any app or video with it then please share it with me.
     
  15. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    Version 1.3 was uploaded and should be available soon.

    It will bring holes-free meshing and walls detection in plane detection demo.
     
    AndroYD likes this.
  16. AndroYD

    AndroYD

    Joined:
    Nov 3, 2014
    Posts:
    25
    Tried it today, and I'm still not sure if this is real life or just fantasy, I'm in total bliss of how amazing this new experimental texturing feature is! That's the culmination of what I wished to happen since the very first version of this plugin, and now all that it's happening right in front of me like magic, to think that day has finally come fills me with joy!
     
  17. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    This is just a beginning. In 2017 was possible with Chisel library and Lenovo Phab2 Pro do this:

    (the device used depth camera for scene understanding and fisheye camera for motion tracking)
     
    Last edited: Mar 18, 2019
  18. BinaryBanana

    BinaryBanana

    Joined:
    Mar 17, 2014
    Posts:
    61
    Hi,
    Just wanted to share that soon there will be also a solution that works on iOS. The code is in alpha, so still not perfect but getting better.

    You can find more here, if anyone is interested: http://ARMeshing.com

    Early alpha version video:
     
  19. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    AndroYD likes this.
  20. AndroYD

    AndroYD

    Joined:
    Nov 3, 2014
    Posts:
    25
    The occlusion feature is a great addition to this asset! I'm glad I didn't have to thinker how to implement it, imagine that Google is still TALKING about adding occlusion features to ARCore in the future https://sa2018.siggraph.org/en/attendees/technical-papers/session/22/details , that's what I absolutely love and appreciate of your work, you DO things rather than TALK, unlike other so-called "competitors" who teases you with similar solutions but never actually release anything usable like SOMEONE up here, or worse, have over-inflated egos and don't even think you're worthy of a reply (am I not "elite" enough?), I'd understand that attitude if someone had the most exclusive thing in the universe, but not when alternatives already exists right here right now.
     
  21. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    The situation is a bit more difficult. Google does not care about mobile AR so much (you can see it on the level of support on ARCore github pages). What they care about is the future and that will be AR glasses which should replace the smartphones one day. However their AR glasses need this things:
    1) Great and not so expensive HW (exact opposite of Magic Leap)
    -here Google does a research they don't talk about much (except Google Glass Enterprise)
    2) Many apps, something user can play with
    -that's why we have ARCore, Google needs some content which will developers easily bring on AR glasses
    -also Google Glass Enterprise should help here to bring some serious apps and not just games
    3) Great technologie
    -they need AR persistence, which is their VPS technology (which is delayed for few years and not only because of lack of data), it should be a paid technology which mostly does not work - not a good idea to release it
    -environmental understanding, they have it ready for limited amount of devices (you can see in their paper that it does not work on iPhone), they cannot release it in this state
    -environmental lighting - they talked about it on Google I/O because it is only one thing which is sure that it will work this summer

    However talking about technology brings more customers than doing the technology. I see on my LinkedIn many posts of cheap AR tests which became viral and because of them posts like this becomes invisible.
     
  22. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14
    The update 1.5 will be available soon. It adds support of ToF cameras using Huawei AREngine and it works currently on this devices:
    Honor View 20
    Huawei P30
    Huawei P30 Pro
    More devices will be added by Huawei soon.

    -One APK can handle ARCore and AREngine easily using ARSelector class
    -Different meshing parameters for active/passive depth sensing
    -Plane detection improvements
    -Support for ARCore 1.10
    -Enabling and disabling AR in the app possible

    For now you can test the demo app:
    https://drive.google.com/file/d/1-M-VJwEwxC4BnRWJIHtsVC2-Sft-oOs0/view?usp=drivesdk

    This update should be available during/shortly after this weekend (depends on Unity's team package check).
     
  23. lvonasek

    lvonasek

    Joined:
    Jan 16, 2019
    Posts:
    14