Shaders and Virtual Reality

Hi,

Till now I have developed a simple game using unity so you could say I am kind of newbie. However, for my thesis, I have decided to implement an application which uses VR and natural elements such as trees, water, etc. I've already managed to put some things together however, I have been reading up on shaders and have come across multiple threads which say that certain shaders such as the "FX/Water shader in unity harm's performance at an alarming rate". Therefore I have decided to start dedicating time to learning shaders with VR in mind. It would be incredibly helpful if anyone could provide some information on how I could get started with writing my own shaders which are optimized for virtual reality purposes. The final product should be able to work on a Samsung S6 using Gear VR.

Many thanks in advance
Andre

 

There isn't really a ton of stuff that's "VR specific" when it comes to shaders, especially if your target is Gear VR. There's single pass instanced rendering stuff, which is mainly just putting the appropriate macros in your vertex shader, but that's not supported on the Gear VR devices. There's some specifics you need to be aware of when doing post processing, but the Gear VR devices don't have enough performance to do any post processing and keep framerate. So really it's down to two things you need to be mindful of when doing VR for Gear VR or similar class devices.

1. Remove everything from the shader you don't absolutely need.
2. Now remove some more because that's still too much.

Ideally shaders for the Gear VR should have no real time lighting, or only very basic per vertex lighting, and a max of one or two textures per shader (with the second texture for baked lighting). Remove specular calculations on anything that does not absolutely need it. Basically, all objects should have baked in lighting either from lightmaps, or per vertex, or baked into the object's textures, or be predominantly Lambert shading only, or don't use lighting at all. Get any ideas about using reflections, or PBR shading, or any fancy shader techniques out of your head, because there's no performance for that.

VR on mobile devices like Gear VR or Daydream is more about removing as much as possible from the scene and game than doing cool and fancy stuff. Some of those devices are just barely capable of running over 60 fps with a single spinning cube in an otherwise empty world, so you have to be very mindful of what you do put in. There are lots, and lots of links with suggestions on how to go about stuff.

https://unity3d.com/learn/tutorials/topics/virtual-reality/optimisation-vr-unity
https://unity3d.com/how-to/optimize-mobile-VR-games
https://developer.oculus.com/blog/carmacks-critiques-graphics-optimization-for-gear-vr/

None of them really talk about shaders much, because really you're going to want to stick to unlit or legacy mobile shaders as anything more complex than that is going to be hard to keep the framerate high enough with.



That FX/Water shader is crazy expensive for mobile, even when not doing VR. The main two things it does to make it look good is it renders two extra views of the scene into textures to capture what is below the water, and for what is reflected by the water. For VR this means you're now rendering the scene 6 times (3 times per eye)! Considering a Gear VR device is generally going to struggling to render just the first two views, the extra 4 aren't going to help. And worse there are a lot of extra optimizations Unity is able to do with the two VR eye views to make them less expensive that it won't be doing for those 4 extra views making them even more expensive.

The way you do water for VR would be to rely on older techniques. Fake it with a basic cube map for the sky & world reflection. For stuff underwater the most you'd want to do is have a custom shader for underwater objects to wiggle their vertices around to fake refraction, and paint them darker blue the deeper underwater they are. For objects stuck in the water that you want to look like they're reflecting in the water, use duplicated geometry that's flipped upside down and maybe fades out, or sprite approximations. Look at old PS2 and N64 games and see what tricks they used.

 

Did you ever find a good water shader for VR? I'm currently working in single pass on Oculus Quest/Go and having trouble, forum post about it separately here: https://forum.unity.com/threads/mob...uest-go-single-pass-forward-rendering.739556/