Fixed foveated rendering on Oculus Quest not working

I have not had any luck getting OVRManager.tiledMultiResLevel working on Quest. With an older Oculus SDK I got the error "Tiled-based Multi-resolution feature is not supported" but now nothing shows up and seemingly there is no resolution change around the edge of the screen. It's as if there's no effect. This is on 2019.1.4f1

 

Seems like it does enable in the script, but I agree nothing changes on screen. I printed the value of OVRManager.tiledMultiResLevel onto a text mesh in my game to check it's value. It does return that it's enabled, but definitely no pixel change.
Maybe they haven't officially released this on Quest? Though it's clear it exists as can see it on published games like Vader Immortal.
Odd!

 

So Oculus just published the Quest documentation:
https://developer.oculus.com/documentation/quest/latest/concepts/unity-advanced-quest/

Its the same as Go, so no different to what we were already trying. Need to contact support.

 

Hi! Did you manage to make it work?

 

Same issue here.

Using Unity 2018.4.1f1

 

FWIW no probs for me to have it to work using the latest Oculus mobile SDK, Unity 2019.1.2f1, standard render pipeline. Obviously this won't work on the Rift, only on Quest. Also, it didn't worked in LWRP.

 

Trying with Unity 2019.1.4, Oculus SDK 1.36, using LWRP.
Changing FFR settings won't do anything, also the Render Scale option in the LWRP settings doesn't affect the final resolution (both with FFR enabled and disabled). Might definitely be a problem with LWRP then...
EDIT: switched back to Standard Pipeline, and FFR works fine now.

As a side question, how much does the LWRP help with Quest compared to Standard Pipeline?

 

@DanjelRicci I didn't took much time experimenting with, at first tries results were bad (I mean worst than standard RP).
But once again i didn't investigated it further.

 

@Gruguir “good” to know then. I was reading more or less the same feedback in other non-VR related threads...

On a side note: I’m not trying complex scenes actually but it’s been really hard for me to understand what makes the game stutter and what not, because it stutters regardless if there’s a cube or some characters (thread here). I ended up here when trying to enable FFR to save on performance.

 

@DanjelRicci for us performance improvements are huge with LWRP. 10 fps to 50fps with a scene with two lights with shadows. Performance is about the same for simple unlit scenes though.

Has anyone from this thread reported a bug to Unity with it? If yes, please post the case number here.

@LeonhardP @phil_lira any information about this? Is the FFR implementation something on Unity's end or on Oculus' end?

Edit: reported as (Case 1162746) [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP

 

@phil_lira and everyone here, I just spent half a day just debugging FFR on LWRP and found a fix.
My assumption was that FFR only works if graphics are directly put on screen without inbetween resolving into a temporary render texture, and turns out this is true.
I added in a couple dozen debug logs into the LWRP itself to track it down, found that "requiresIntermediateColorTex" was always true and that the only reason for it is that Display.main.requiresBlitToBackbuffer was always returning true for the Quest. Knowing that the Built-in pipeline renders directly to the backbuffer I assumed that this is incorrect and just overwrote it with "false".
Now FFR works, and since I didn't do any other change to the LWRP I think everything else should still be working as well.

For anyone who needs this as desperately as I did: change "Display.main.requiresBlitToBackbuffer" in "RequiresIntermediateColorTexture" in ForwardRenderer.cs in the LWRP package to "false". Done. Double that performance.

 

@fherbst tried that but the RequiresIntermediateColorTexture returned true anyway. Didn't have time to investigate so i put a glorious "return false" on that method AND IT WORKS LIKE A CHARME
Frame rate is now fixed at 72 (before it was oscillating between 70 and 52).
Thank you very much!
Hope that a fix will arrive from Unity or Oculus.

 

I was also getting "isTargetTexture2DArray" and "renderingData.killAlphaInFinalBlit" as true, so I went with the nuclear "return false;" option. One thing to note is that I believe this workaround will disable the ability to cast the content to a mobile device or Chromecast, though I can't verify this.

 

@fherbst I tried out your fix and it looks like it enables FFR, but breaks MSAA. Fiddlesticks.

 

I double-checked that again and it doesn't seem to break MSAA for me (or it doesn't work in either mode). Weird.

 

I made a quality switcher that can cycle from MSAA off to MSAA 8x, and when using the "return false" work-around, I see no difference in aliasing, but when using the unchanged file, I can see it shift, so it breaks MSAA for me.

 

Hi,
I've resolved by switching off the option "Force Into RT" in the camera component, it appears if you switch to Debug mode:

 

@NicolaZangrando What does this setting fix? I'm using the OVRCameraRig and this setting is off by default on all the cameras in the scene and I still have either broken FFR or broken MSAA.

The simplest way to test this is use a shell command to turn on FFR across all apps:

Code (CSharp):

1. adb shell setprop debug.oculus.foveation.level 4

You'll be able to see that FFR is running even in Oculus Home.

In Oculus Home you can see FFR on the window beams, but you can also see the rest of the scene is using MSAA (no jaggies).

Then go to your LWRP app and see if FFR and/or MSAA are running.
Without FFR Hack:

With FFR Hack:


Even though the plank near the center of the screen is outside of the area affected by FFR you can see the aliasing on it when the hack is applied.

Has anyone from Unity commented on this issue yet?

 

No, my bug report from June 13 ((Case 1162746) [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP) is still unanswered. Over different channels (success team) I got the vague reply that it's "discussed internally", but that's 2 weeks ago now.
@phil_lira any news?

 

I'm also getting this: confirmed that FFR is being set to different levels with the OVR Metrics Tool, it's working in Oculus Home but not in app. Adding "return false;" as the first line of RequiresIntermediateColorTexture did not fix the issue. Unity 2019.2.0b7, LWRP 6.7.1.

Edit: turns out since I'm using >0 Renderer Features (using a Render Objects pass) I had to go a level up and override bool createColorTexture in Setup() to false. Confirmed using Frame Debugger that opaques were now rendering to a <no name> target instead of _CameraColorTexture. FFR is working again and we saw nearly 2x performance increase even without FFR compared to having to blit the intermediate color texture at the end of the frame.

Edit 2: Yeah, it breaks MSAA on Quest. Hoping not too much of that performance boost was going from 4x to 0x MSAA..

 

I submitted a bug to Oculus and yesterday they replied with this:

I was already using 2019.1.10 and LWRP 5.16.1, and FFR was still broken. It's interesting that they seem to think it's working there. Maybe I'm doing something wrong, or they're doing something right?

Unity just released 2019.12 and Oculus just released integration 1.39. I'm going to retest with both of those and then I'm going to see if I can reproduce @rustinlee's results hacking the LWRP package.

 

Curious what you find.
I can also confirm @Dark-Table that FFR didn't work in 5.16 in 2019.1. With the "hack" it is still working for me on latest LWRP in latest 2019.2.

 

I'm working on a way to set Foveated rendering without using all of the Oculus integration scripts and assets inside the project, does anyone here know if it can be set in the android manifest file or something similar to the adb setprop example above?

At the moment I am importing the "OVRPlugin" .dll file and making calls to ovrp_SetTiledMultiResLevel(FixedFoveatedRenderingLevel level);

 

Having exactly the same issue now after confirming some code I wrote sets it in game and the logcat results showing FOV = 4 during the game but nothing changes on screen and no performance increase is noticed

 

@ROBYER1 try https://forum.unity.com/threads/fix...oculus-quest-not-working.686662/#post-4694618, works for me at least. With FFR + MSAA

 

I'm using built-in renderer at the moment so I have no idea why it's not working!

 

By standard pipeline, do you mean built-in renderer? What Unity engine version were you using? I can't seem to get it working on the latest 2019 versions and will probably submit a bug report for it this monday coming.

 

@ROBYER1 I've found that after I've used the "setprop" command to adjust the FFR setting I need to reboot the device to be able to set it programmatically. So if you used "setprop debug.oculus.foveation.level 0" to reset your FFR level, you'll actually be locked at FFR 0 until you reboot.

There may be some other way to "clearprop," but a reboot definitely works.

 

I assume that overrides the per-application FFR levels then?

As I was also trying to set the FFR level using the OVRplugin calls included with the Oculus Integration. I will check this again on monday - hopefully it's the fix!

I will report back with my findings

 

@ROBYER1 Yes. It globally overrides application FFR levels. You can use the setprop command to play Beat Saber with "High-Top" FFR if you think it will help you get the high score (I'm in the top 50 worldwide on a couple songs).

 

As for my testing with FFR + LWRP. What I've finally settled on is Unity 2019.2.1f1 w/ LWRP 6.9.1 & Oculus Integration 1.39. I apply the following patch to ForwardRenderer.cs at line 117:

Code (CSharp):

1.  
2. #if FFR_HACK
3.             bool createColorTexture = false;
4. #else
5.             bool createColorTexture = RequiresIntermediateColorTexture(ref renderingData, cameraTargetDescriptor)
6.                                       || rendererFeatures.Count != 0;
7. #endif
8.  

Then I add the FFR_HACK define to the scripting define symbols section in the Player Settings. This lets me easily toggle the hack. I _think_ MSAA breaks or gets reduced (stuff in the distance does twinkle more), but in the scene I'm using everything still looks great and I get a nice stable 72fps even when there are several object with real time shadows. Until there's an official fix I can live with this.

 

Have you reported this to Unity through the Bug Reporter in the editor? I'm currently still using Built-in until LWRP (Now Universal RP) is more stable and release ready. Bugs like this confirm to me that I should still avoid using it unless I want to constantly hit bugs during development!

 

Yes, the FFR bug with LWRP is already in Unity's hands since June 28, without response, despite asking back multiple times.
Case 1162746 [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP

 

Seems it hasn't been officially pushed to their bug tracker which is annoying as I wanted to upvote it

 

I'm trying to get to the very bottom of where and how to activate FFR using Oculus scripts, it seems the very minimum thing you need is the 'OVR Camera Rig' script, and another script that has

Code (CSharp):

1.  OVRPlugin.fixedFoveatedRenderingLevel = OVRPlugin.FixedFoveatedRenderingLevel.HighTop;

in the start function.

But apart from that, I'm having no luck simplifying the implementation of the FFR as we don't really want to use all the Oculus-specific camera rig and Utilities as our app is supporting other headsets. It seems to all lead back to the OVRPlugin referenced in the scripts, see this example from the 'OVRCameraRig.cs' that I am currently trying to get working without the rest of the code

Code (CSharp):

1.     if (OVRNodeStateProperties.GetNodeStatePropertyVector3(Node.Head, NodeStatePropertyType.Position, OVRPlugin.Node.Head, OVRPlugin.Step.Render, out pos))
2.             headPose.position = pos;
3.         if (OVRNodeStateProperties.GetNodeStatePropertyQuaternion(Node.Head, NodeStatePropertyType.Orientation, OVRPlugin.Node.Head, OVRPlugin.Step.Render, out rot))
4.             headPose.orientation = rot;

 

Coincidentally, I just got an update from Oculus about my LWRP+FFR bug:

A little vague on what "being pushed" means. Maybe next time a system update gets pushed we'll see FFR working (and MSAA breaking) without the hack? I kinda wish Oculus would get directly in touch with Unity, because they probably know exactly what needs to be fixed and which graphics engineer handles it.

 

This definitely puts me off using LWRP yet! Sounds like pushed to the main branch of the SRP branch. Since the whole thing is being renamed from LWRP to URP I would advise waiting for a stable release of URP? (Universal render pipeline)

 

I doubt anyone will ever have the issue I had but I found the fix, when activating

Code (CSharp):

1.  OVRPlugin.fixedFoveatedRenderingLevel = OVRPlugin.FixedFoveatedRenderingLevel.HighTop;

The game camera needs to be using the default Unity camera settings, just create a new camera in the scene and copy its values across to you rig's camera. I believe it was either the depth, the near/far distance or HDR being off switched back to on that fixed it but I cannot be bothered to nail it down after hours wasted looking into this.

 

Was there ever a fix for this or do I need to report it as a bug?

 

@ROBYER1
(Case 1162746) [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP
Is open with Unity since June, no reply at all so far.

From SRP changelogs it seems they are refactoring XR all over the engine right now, so it's in another "let's fix a broken system by introducing a new preview system which might be broken" state I'd say. Hopefully Unite gives us better insight.

 

I've also reported it with UniversalRP here (Case 1184851) UniversalRP - Oculus Quest/Go Fixed Foveated Rendering not working in URP/LWRP

 

Lolol, I've also reported it as case 1177737 "[Oculus Quest] Fixed foveated rendering (FFR) is broken in Unity 2019.2.1f1 using the Lightweight Render Pipeline (LWRP)"

I think it's best to be loud and specific that the bug is occurring in the latest version of Unity & SRP (now 5 revisions old and still broken).

Also, I think what "being pushed" means is that Oculus pushes an update to headsets once a month (around the 20th) that contains the most recent bundle of fixes and features, and then releases a new Oculus Integration around the 25th
that gives access to those features. So I'm hoping that before Oculus Connect next week (and coincidentally Unite CPH) FFR will be working without a hack on the Quest. But MSAA will break when it's enabled unless there's a surprise at Unite.

It's really frustrating because between this and the post processing effect bug it seems to be nearly impossible to ship something for Quest using LWRP/URP.

 

I have also had an update on the UniversalRP/LWRP VR template being missing from the editor presets, a post will be coming soon to address the reason why (it is in the works)

I've asked about 4 of the QA team members to take a look at the FFR issue with UniversalRP so hopefully at least one of them flags it as an issue.

I have been invited to the Unity Copenhagen office from being so active with bug reports and also invited to a beta round-table discussion there where I will definitely mention how these dependencies and slow updates to major bugs like this are overlooked. I am feeling like I should suggest that major platform-specific performance features with regards to rendering or script execution should be tested with these new additions to the engine.

If you have any other suggestions please do message me with any other feedback you would like passed on to their team!

 

@ROBYER1 If there's something you want to bring up with Unity, here's a great example of something that absolutely cannot be done with LWRP/URP in its current state:

This is Red Matter (Unreal Engine) running on the Oculus Quest. In this video you can see foveated rendering is working (look at pixelation at edges of video), MSAA is working (no jaggies on horizontal lines) and the glass is using a simple refraction shader. Also, I'm betting this is running using Vulkan instead of OpenGLES which won't work for Unity VR.

All of this needs to get fixed (soon) by Unity if they want to remain competitive in XR.

 

For anyone still looking at this, 2019.3b4 with URP 7.0.1 seems to break with the workaround @Dark-Table proposes above. It causes what looks to me like clear issue with the backbuffer's swap chain, where the color does not clear and has a terrible alternating frame flicker with the splash screen.

 

In this talk (from today on Occulus Connect) there is some information on URP and quest compatibility.
The interesting part starts at 33 minutes. There is also a part about ffr.
In detail he mentions, that there is an extra blit that makes it incompatible with ffr.

When I understand it right, the URP with it's standard template should not be used with QUEST right now.
They think it is "very close to be used in production". So that means not ready right now.

 

@Dark-Table I spoke to fherbst about this yesterday and can confirm the MSAA is only broken in the editor view and on other VR platforms (not including the Quest/Go), it works perfectly fine when built for Quest running on device. The workaround for that is to add a platform specific compile line around the hack (for now). However the Unity developers have been made aware of the issue and are looking into working on a fix from what I have been told.

 

That rundown is overly pessimistic and is kind of a displaying a worst case scenario. Obviously you bake shadows on any static objects. Obviously you only have one shadow casting light. But you definitely can have objects with runtime shadows and run @72fps with the LWRP (with or without FFR). Super Hot isn't using LWRP, but they do use realtime shadows in their app. They went with "no cascades" and a very close shadow distance (~4 meters). Hopefully their talk gets to youtube soon. I've had a similar setup in my prototype for a while (my app occasionally dips below 72, but I think it's more about the number of triangles in certain parts of the scene).

You don't have to give up on dynamic shadows on the Quest, just be realistic.

Unfortunately, I have a sinking feeling they're being overly optimistic about the timeline for fixing FFR with LWRP/URP.

 

I'm certain that on my initial tests MSAA was broken on device when FFR was working (these were tests running in the Unity Sample Scene). In my more recent tests running in my own prototype level I thought I was going crazy because it seemed like MSAA was working. I think there may still be some conditions that can break MSAA if you enable the FFR hack? I think I turned off the post processing effects, but fog might have still been on?

 

In the example I saw, @fherbst had fog on for sure, not sure about post processing but could only assume they were off too. An official fix will come but I will try out the fix today and see how my project performs with it.