Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Latest ARFoundation experiment: Blending digital objects with your environment.

Discussion in 'Handheld AR' started by KingOfSnake, Apr 4, 2019.

  1. KingOfSnake

    KingOfSnake

    Joined:
    Feb 24, 2013
    Posts:
    28


    Breakdown
    1/6 Blob Shadow - Ground your digital object with a blob shadow.
    2/6 Light Estimation - Calculate light intensity based on the average color of the camera feed.
    3/6 Sun Direction - Determine sun direction based on your mobile's longitude, latitude, time of day and true north.
    4/6 Real-time Shadow - Use ARKit's ARSurface shader for real-time shadow and determine shadow intensity based on light intensity.
    5/6 Realtime Reflection - Use the camera feed as input for your real-time reflection probe.
    6/6 Camera Effects - Add additional camera effects like Color Grading and Fast Vignette for a stylish look.
     
    hawken, dnnkeeper, Squize and 2 others like this.
  2. Saicopate

    Saicopate

    Joined:
    Sep 25, 2017
    Posts:
    56
    Does it work for Android as well?
    3/6 seems very innovative, would love to know how you made it and how effective it is knowing the wobbliness of compass readings :)
     
  3. m3rt32

    m3rt32

    Joined:
    Mar 5, 2013
    Posts:
    54
    I got a similar result with my own implementation. My question is, did you use anything special for realtime reflections ? I use microsoft's mixedreality toolkit, it comes with a runtime cubemapper, than assigns it to reflection probe. But I wonder about your implementation, because performance is a concern in realtime reflections.
     
  4. BinaryBanana

    BinaryBanana

    Joined:
    Mar 17, 2014
    Posts:
    61
    Looks great! Which values are you taking from light estimation? Any chance to get a sample code at some point?
     
  5. KingOfSnake

    KingOfSnake

    Joined:
    Feb 24, 2013
    Posts:
    28
    This is a screencast from my Samsung S8 and runs at 45 FPS.

    For the sun direction, I am taking the moving average of the last x values of compass reading before the object is placed. When the object is placed, the moving average at that specific moment is set as true north.
     
  6. KingOfSnake

    KingOfSnake

    Joined:
    Feb 24, 2013
    Posts:
    28
    I update the reflection probe via script every x frames, set the time slicing to individual faces and set resolution low. This step has definitely got the biggest performance hit.
     
  7. KingOfSnake

    KingOfSnake

    Joined:
    Feb 24, 2013
    Posts:
    28
    First I calculate the average color of the camera feed.
    Second I calculate the light intensity of the average color.
    Result is actually the same as ARCore light estimation :p
     
    BinaryBanana likes this.
  8. pedrobarrosbrasil

    pedrobarrosbrasil

    Joined:
    Apr 4, 2019
    Posts:
    8
    amazing work!
     
  9. soorya696

    soorya696

    Joined:
    Dec 13, 2018
    Posts:
    13
    Really a great job.
    Please make a tutorial.
     
  10. dnnkeeper

    dnnkeeper

    Joined:
    Jul 7, 2013
    Posts:
    57
    Super cool! I'd like to know how did you make Realtime Reflection? Is it ARKit specific? I'm looking for an android solution.
     
  11. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    475
    reflection probes are now cross platform afaik
     
  12. dnnkeeper

    dnnkeeper

    Joined:
    Jul 7, 2013
    Posts:
    57
    Couldn't get them working in the latest AR Foundation sample project. Have you got some info about it? Can't find anything about XREnvironmentProbeSubsystem supported platforms.
     
  13. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    475