Cross-Platform Fixed Foveated Rendering (FFR) with Vulkan for VR

I have just seen in the Oculus Connect 6 talk about Vulkan for VR, they have announced that Vulkan has it's own Fixed Foveated Rendering feature through the Vulkan Fragment Density Map which is completely cross-platform and not tied to Qualcomm hardware.

Will this be added as a feature once Unity supports Vulkan for the Quest?
It sounds like it will be very useful for scalable performance boosts on all supported Vulkan platforms such as Windows and Android VR! See more details in the video below:

VK_EXT_fragment_density_map extension

 

Version 1.1.3 of the Oculus Loader XR Plugin will have FFR hookups in Vulkan support in Unity 2020

https://docs.unity3d.com/Packages/com.unity.xr.oculus@1.1/changelog/CHANGELOG.html

 

As a quick update, it looks like MultiPass rendering on quest supports Vulkan which also supports FFR on UniversalRP. However Vulkan with Multiview rendering isn't ready yet so I haven't looked into it.

How to activate FFR with Oculus XR Plugin is not yet documented either, although the changelog seems to say the functionality is there.

 

One of the better things about the vulkan implementation is you can properly segregate the various parts of the screen to be rendered for multi resolution shading. It's not easy to do but you could for example have sky being a lower shading rate, or darker areas of the screen being a lower resolution - it's not just a circular or (fixed) foveated but dynamic.

If you had a transparent HUD then you could also mark those areas out so the first pass is rendered lower res behind those elements.

 

Still wondering when we can actually use UniversalRP (URP) with FFR for the Oculus Quest, it was advertised as being usable back in September 2019 at this video, where they even talk about using UniversalRP before announcing that FFR basically doesn't work with it..

 

Update on this here, this feature is now in Unity 2019.3 release, but it is not documented anywhere to explain how to actually use it

https://unity.com/releases/2019-3/platforms

 

Everywhere i see the BIG news but when I try it:

 

That is built into the code for the Oculus Integration pack, a line of code that checks if you are building with Vulkan and disables it (daft I know), double click the error to where that line is and comment it out. Or google it if you are stuck other people have had the same issue.

 

Big news, FFR should be working soon properly on URP with this modification to the ForwardRenderer.cs once merged. Copy it now if you need it!

https://github.com/Unity-Technologies/ScriptableRenderPipeline/pull/5900

 

Hey, so far I have got vulkan and multiview to work. Problem is, that using any (even empty) CustomRenderPassFeature breaks the multiview. The right eyes rotation seems of and objects move away the further they move to the side of the right eye screen. Any idea if that is a bug or if CRPF are simply not supported on Vulkan & URP yet? Note that this doesn't happen on multipass.

 

Please report as a bug using Unity Bug Reporter, the Devs won't see it here.

 

So, URP + MultiPass Stereo Rendering + Vulkan = FFR? Is this automatic, I see no APIs to activate it. I'm using XR Plugin Management, rather than the Oculus Integration?

 

You obviously haven't looked far! Here you go

https://docs.unity3d.com/Packages/c..._Oculus_Utils_SetFoveationLevel_System_Int32_

https://forum.unity.com/threads/fix...-quest-not-working.686662/page-3#post-5441937

 

Ah, excellent. I had the wrong version selected. Thanks

 

Bumping this FFR thread.
I'm using unity 2019.4.31 with standard forward renderer and trying to get FFR to work consistently. I have it working on a simple scene (just a few cubes). However, as soon as I get to more complex scenes with my code running, FFR stops working. I am not using anything that requires intermediate render to texture on the main view (as far as I can tell), no post processing of any kind.

Has anyone figured out what breaks FFR ?

Could it be related to some fancy visual effects that use the grab pass in the shaders ?
Are there other types of shaders that could break FFR that I should be aware of ?

[Edit] For those interested, I managed to narrow it down a little. It seems that FFR works when I start my game, but stops working after I am doing some fancy off screen rendering using command buffers. It is not clear what in the command buffer breaks it, because a unit test in another project with simple blits and drawmesh using a commandbuffer didn't affect FFR. I'll keep digging