Latest ARFoundation experiment: Blending digital objects with your environment.

Breakdown
1/6 Blob Shadow - Ground your digital object with a blob shadow.
2/6 Light Estimation - Calculate light intensity based on the average color of the camera feed.
3/6 Sun Direction - Determine sun direction based on your mobile's longitude, latitude, time of day and true north.
4/6 Real-time Shadow - Use ARKit's ARSurface shader for real-time shadow and determine shadow intensity based on light intensity.
5/6 Realtime Reflection - Use the camera feed as input for your real-time reflection probe.
6/6 Camera Effects - Add additional camera effects like Color Grading and Fast Vignette for a stylish look.

 

Does it work for Android as well?
3/6 seems very innovative, would love to know how you made it and how effective it is knowing the wobbliness of compass readings

 

I got a similar result with my own implementation. My question is, did you use anything special for realtime reflections ? I use microsoft's mixedreality toolkit, it comes with a runtime cubemapper, than assigns it to reflection probe. But I wonder about your implementation, because performance is a concern in realtime reflections.

 

Looks great! Which values are you taking from light estimation? Any chance to get a sample code at some point?

 

This is a screencast from my Samsung S8 and runs at 45 FPS.

For the sun direction, I am taking the moving average of the last x values of compass reading before the object is placed. When the object is placed, the moving average at that specific moment is set as true north.

 

I update the reflection probe via script every x frames, set the time slicing to individual faces and set resolution low. This step has definitely got the biggest performance hit.

 

First I calculate the average color of the camera feed.
Second I calculate the light intensity of the average color.
Result is actually the same as ARCore light estimation

 

amazing work!

 

Really a great job.
Please make a tutorial.

 

Super cool! I'd like to know how did you make Realtime Reflection? Is it ARKit specific? I'm looking for an android solution.

 

reflection probes are now cross platform afaik

 

Couldn't get them working in the latest AR Foundation sample project. Have you got some info about it? Can't find anything about XREnvironmentProbeSubsystem supported platforms.

 

@KingOfSnake this is really cool, would you mind sharing your solution to get a gameObject to point due north? I've tried a few approaches using gyro and compass, compass works well on a screenspace canvas but in world space it needs to be offset somehow.

Another approach using gyro is here but it doesn't seem to work https://forum.unity.com/threads/geo-localised-objects-in-ar-core.540223/

Thanks!

 

Hey bro really mind blowing example , can you share your project for learning or create tutorial ? because light is most headache for me or as well as lots of member.

 

@KingOfSnake looking awesome!!!

 

Hello
Skybox Reflection Probe not working in ARFoundation, I would appreciate if any professional can help me
Thanks

 

backed you probe it defiantly work

 

Hello! How I can add realtime shadows for android with ar foundation? Can you give example?