Search Unity

Fixed foveated rendering on Oculus Quest not working

Discussion in 'AR/VR (XR) Discussion' started by AmbOcclusion, May 29, 2019.

  1. AmbOcclusion

    AmbOcclusion

    Joined:
    Sep 17, 2012
    Posts:
    28
    I have not had any luck getting OVRManager.tiledMultiResLevel working on Quest. With an older Oculus SDK I got the error "Tiled-based Multi-resolution feature is not supported" but now nothing shows up and seemingly there is no resolution change around the edge of the screen. It's as if there's no effect. This is on 2019.1.4f1
     
    ROBYER1 likes this.
  2. Hyperactive

    Hyperactive

    Joined:
    Aug 13, 2013
    Posts:
    2
    Seems like it does enable in the script, but I agree nothing changes on screen. I printed the value of OVRManager.tiledMultiResLevel onto a text mesh in my game to check it's value. It does return that it's enabled, but definitely no pixel change.
    Maybe they haven't officially released this on Quest? Though it's clear it exists as can see it on published games like Vader Immortal.
    Odd!
     
    ROBYER1 likes this.
  3. Hyperactive

    Hyperactive

    Joined:
    Aug 13, 2013
    Posts:
    2
  4. NicolaZangrando

    NicolaZangrando

    Joined:
    Feb 11, 2019
    Posts:
    2
    Hi! Did you manage to make it work?
     
  5. digitaljohn

    digitaljohn

    Joined:
    Aug 14, 2012
    Posts:
    8
    Same issue here.

    Using Unity 2018.4.1f1
     
  6. Gruguir

    Gruguir

    Joined:
    Nov 30, 2010
    Posts:
    311
    FWIW no probs for me to have it to work using the latest Oculus mobile SDK, Unity 2019.1.2f1, standard render pipeline. Obviously this won't work on the Rift, only on Quest. Also, it didn't worked in LWRP.
     
  7. DanjelRicci

    DanjelRicci

    Joined:
    Mar 8, 2010
    Posts:
    161
    Trying with Unity 2019.1.4, Oculus SDK 1.36, using LWRP.
    Changing FFR settings won't do anything, also the Render Scale option in the LWRP settings doesn't affect the final resolution (both with FFR enabled and disabled). Might definitely be a problem with LWRP then...
    EDIT: switched back to Standard Pipeline, and FFR works fine now.

    As a side question, how much does the LWRP help with Quest compared to Standard Pipeline?
     
    Last edited: Jun 11, 2019
    ROBYER1 likes this.
  8. Gruguir

    Gruguir

    Joined:
    Nov 30, 2010
    Posts:
    311
    @DanjelRicci I didn't took much time experimenting with, at first tries results were bad (I mean worst than standard RP).
    But once again i didn't investigated it further.
     
  9. DanjelRicci

    DanjelRicci

    Joined:
    Mar 8, 2010
    Posts:
    161
    @Gruguir “good” to know then. I was reading more or less the same feedback in other non-VR related threads...

    On a side note: I’m not trying complex scenes actually but it’s been really hard for me to understand what makes the game stutter and what not, because it stutters regardless if there’s a cube or some characters (thread here). I ended up here when trying to enable FFR to save on performance.
     
  10. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    @DanjelRicci for us performance improvements are huge with LWRP. 10 fps to 50fps with a scene with two lights with shadows. Performance is about the same for simple unlit scenes though.

    Has anyone from this thread reported a bug to Unity with it? If yes, please post the case number here.

    @LeonhardP @phil_lira any information about this? Is the FFR implementation something on Unity's end or on Oculus' end?

    Edit: reported as (Case 1162746) [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP
     
    Last edited: Jun 14, 2019
    ROBYER1 likes this.
  11. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    @phil_lira and everyone here, I just spent half a day just debugging FFR on LWRP and found a fix.
    My assumption was that FFR only works if graphics are directly put on screen without inbetween resolving into a temporary render texture, and turns out this is true.
    I added in a couple dozen debug logs into the LWRP itself to track it down, found that "requiresIntermediateColorTex" was always true and that the only reason for it is that Display.main.requiresBlitToBackbuffer was always returning true for the Quest. Knowing that the Built-in pipeline renders directly to the backbuffer I assumed that this is incorrect and just overwrote it with "false".
    Now FFR works, and since I didn't do any other change to the LWRP I think everything else should still be working as well.

    For anyone who needs this as desperately as I did: change "Display.main.requiresBlitToBackbuffer" in "RequiresIntermediateColorTexture" in ForwardRenderer.cs in the LWRP package to "false". Done. Double that performance.
     
    paulw_cloudhead and DanjelRicci like this.
  12. StudioEvil

    StudioEvil

    Joined:
    Aug 28, 2013
    Posts:
    51
    @fherbst tried that but the RequiresIntermediateColorTexture returned true anyway. Didn't have time to investigate so i put a glorious "return false" on that method AND IT WORKS LIKE A CHARME :D
    Frame rate is now fixed at 72 (before it was oscillating between 70 and 52).
    Thank you very much! :)
    Hope that a fix will arrive from Unity or Oculus.
     
    justdizzy, fherbst and DanjelRicci like this.
  13. justdizzy

    justdizzy

    Joined:
    Mar 8, 2016
    Posts:
    52
    I was also getting "isTargetTexture2DArray" and "renderingData.killAlphaInFinalBlit" as true, so I went with the nuclear "return false;" option. One thing to note is that I believe this workaround will disable the ability to cast the content to a mobile device or Chromecast, though I can't verify this.
     
  14. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    @fherbst I tried out your fix and it looks like it enables FFR, but breaks MSAA. Fiddlesticks.
     
  15. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    I double-checked that again and it doesn't seem to break MSAA for me (or it doesn't work in either mode). Weird.
     
  16. justdizzy

    justdizzy

    Joined:
    Mar 8, 2016
    Posts:
    52
    I made a quality switcher that can cycle from MSAA off to MSAA 8x, and when using the "return false" work-around, I see no difference in aliasing, but when using the unchanged file, I can see it shift, so it breaks MSAA for me.
     
  17. NicolaZangrando

    NicolaZangrando

    Joined:
    Feb 11, 2019
    Posts:
    2
    Hi,
    I've resolved by switching off the option "Force Into RT" in the camera component, it appears if you switch to Debug mode:

    fovproblem.png
     
    fherbst likes this.
  18. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    @NicolaZangrando What does this setting fix? I'm using the OVRCameraRig and this setting is off by default on all the cameras in the scene and I still have either broken FFR or broken MSAA.

    The simplest way to test this is use a shell command to turn on FFR across all apps:
    Code (CSharp):
    1. adb shell setprop debug.oculus.foveation.level 4
    You'll be able to see that FFR is running even in Oculus Home.
    ffrtest.png
    In Oculus Home you can see FFR on the window beams, but you can also see the rest of the scene is using MSAA (no jaggies).

    Then go to your LWRP app and see if FFR and/or MSAA are running.
    Without FFR Hack:
    lwrp_no_ffrtest.png
    With FFR Hack:
    lwrp_ffrtest.png

    Even though the plank near the center of the screen is outside of the area affected by FFR you can see the aliasing on it when the hack is applied.

    Has anyone from Unity commented on this issue yet?
     
    ROBYER1 likes this.
  19. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    No, my bug report from June 13 ((Case 1162746) [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP) is still unanswered. Over different channels (success team) I got the vague reply that it's "discussed internally", but that's 2 weeks ago now.
    @phil_lira any news?
     
  20. rustinlee

    rustinlee

    Joined:
    May 31, 2014
    Posts:
    8
    I'm also getting this: confirmed that FFR is being set to different levels with the OVR Metrics Tool, it's working in Oculus Home but not in app. Adding "return false;" as the first line of RequiresIntermediateColorTexture did not fix the issue. Unity 2019.2.0b7, LWRP 6.7.1.

    Edit: turns out since I'm using >0 Renderer Features (using a Render Objects pass) I had to go a level up and override bool createColorTexture in Setup() to false. Confirmed using Frame Debugger that opaques were now rendering to a <no name> target instead of _CameraColorTexture. FFR is working again and we saw nearly 2x performance increase even without FFR compared to having to blit the intermediate color texture at the end of the frame.

    Edit 2: Yeah, it breaks MSAA on Quest. Hoping not too much of that performance boost was going from 4x to 0x MSAA..
     
    Last edited: Jul 30, 2019
    Dark-Table likes this.
  21. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    I submitted a bug to Oculus and yesterday they replied with this:
    I was already using 2019.1.10 and LWRP 5.16.1, and FFR was still broken. It's interesting that they seem to think it's working there. Maybe I'm doing something wrong, or they're doing something right?

    Unity just released 2019.12 and Oculus just released integration 1.39. I'm going to retest with both of those and then I'm going to see if I can reproduce @rustinlee's results hacking the LWRP package.
     
    ROBYER1 likes this.
  22. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    Curious what you find.
    I can also confirm @Dark-Table that FFR didn't work in 5.16 in 2019.1. With the "hack" it is still working for me on latest LWRP in latest 2019.2.
     
  23. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    I'm working on a way to set Foveated rendering without using all of the Oculus integration scripts and assets inside the project, does anyone here know if it can be set in the android manifest file or something similar to the adb setprop example above?

    At the moment I am importing the "OVRPlugin" .dll file and making calls to ovrp_SetTiledMultiResLevel(FixedFoveatedRenderingLevel level);
     
    Claytonious likes this.
  24. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    Having exactly the same issue now after confirming some code I wrote sets it in game and the logcat results showing FOV = 4 during the game but nothing changes on screen and no performance increase is noticed
     
  25. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
  26. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
  27. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    By standard pipeline, do you mean built-in renderer? What Unity engine version were you using? I can't seem to get it working on the latest 2019 versions and will probably submit a bug report for it this monday coming.
     
  28. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    @ROBYER1 I've found that after I've used the "setprop" command to adjust the FFR setting I need to reboot the device to be able to set it programmatically. So if you used "setprop debug.oculus.foveation.level 0" to reset your FFR level, you'll actually be locked at FFR 0 until you reboot.

    There may be some other way to "clearprop," but a reboot definitely works.
     
    ROBYER1 likes this.
  29. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    I assume that overrides the per-application FFR levels then?

    As I was also trying to set the FFR level using the OVRplugin calls included with the Oculus Integration. I will check this again on monday - hopefully it's the fix!

    I will report back with my findings
     
  30. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    @ROBYER1 Yes. It globally overrides application FFR levels. You can use the setprop command to play Beat Saber with "High-Top" FFR if you think it will help you get the high score (I'm in the top 50 worldwide on a couple songs).
     
    fherbst and ROBYER1 like this.
  31. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    As for my testing with FFR + LWRP. What I've finally settled on is Unity 2019.2.1f1 w/ LWRP 6.9.1 & Oculus Integration 1.39. I apply the following patch to ForwardRenderer.cs at line 117:

    Code (CSharp):
    1.  
    2. #if FFR_HACK
    3.             bool createColorTexture = false;
    4. #else
    5.             bool createColorTexture = RequiresIntermediateColorTexture(ref renderingData, cameraTargetDescriptor)
    6.                                       || rendererFeatures.Count != 0;
    7. #endif
    8.  
    Then I add the FFR_HACK define to the scripting define symbols section in the Player Settings. This lets me easily toggle the hack. I _think_ MSAA breaks or gets reduced (stuff in the distance does twinkle more), but in the scene I'm using everything still looks great and I get a nice stable 72fps even when there are several object with real time shadows. Until there's an official fix I can live with this.
     
    paulw_cloudhead likes this.
  32. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    Have you reported this to Unity through the Bug Reporter in the editor? I'm currently still using Built-in until LWRP (Now Universal RP) is more stable and release ready. Bugs like this confirm to me that I should still avoid using it unless I want to constantly hit bugs during development!
     
  33. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    Yes, the FFR bug with LWRP is already in Unity's hands since June 28, without response, despite asking back multiple times.
    Case 1162746 [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP
     
    ROBYER1 likes this.
  34. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    Seems it hasn't been officially pushed to their bug tracker which is annoying as I wanted to upvote it
     
  35. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    I'm trying to get to the very bottom of where and how to activate FFR using Oculus scripts, it seems the very minimum thing you need is the 'OVR Camera Rig' script, and another script that has
    Code (CSharp):
    1.  OVRPlugin.fixedFoveatedRenderingLevel = OVRPlugin.FixedFoveatedRenderingLevel.HighTop;
    in the start function.

    But apart from that, I'm having no luck simplifying the implementation of the FFR as we don't really want to use all the Oculus-specific camera rig and Utilities as our app is supporting other headsets. It seems to all lead back to the OVRPlugin referenced in the scripts, see this example from the 'OVRCameraRig.cs' that I am currently trying to get working without the rest of the code


    Code (CSharp):
    1.     if (OVRNodeStateProperties.GetNodeStatePropertyVector3(Node.Head, NodeStatePropertyType.Position, OVRPlugin.Node.Head, OVRPlugin.Step.Render, out pos))
    2.             headPose.position = pos;
    3.         if (OVRNodeStateProperties.GetNodeStatePropertyQuaternion(Node.Head, NodeStatePropertyType.Orientation, OVRPlugin.Node.Head, OVRPlugin.Step.Render, out rot))
    4.             headPose.orientation = rot;
     
  36. Dark-Table

    Dark-Table

    Joined:
    Nov 25, 2008
    Posts:
    243
    Coincidentally, I just got an update from Oculus about my LWRP+FFR bug:
    A little vague on what "being pushed" means. Maybe next time a system update gets pushed we'll see FFR working (and MSAA breaking) without the hack? I kinda wish Oculus would get directly in touch with Unity, because they probably know exactly what needs to be fixed and which graphics engineer handles it.
     
    ROBYER1 likes this.
  37. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    This definitely puts me off using LWRP yet! Sounds like pushed to the main branch of the SRP branch. Since the whole thing is being renamed from LWRP to URP I would advise waiting for a stable release of URP? (Universal render pipeline)
     
  38. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    I doubt anyone will ever have the issue I had but I found the fix, when activating
    Code (CSharp):
    1.  OVRPlugin.fixedFoveatedRenderingLevel = OVRPlugin.FixedFoveatedRenderingLevel.HighTop;
    The game camera needs to be using the default Unity camera settings, just create a new camera in the scene and copy its values across to you rig's camera. I believe it was either the depth, the near/far distance or HDR being off switched back to on that fixed it but I cannot be bothered to nail it down after hours wasted looking into this.
     
  39. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    Was there ever a fix for this or do I need to report it as a bug?
     
  40. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    262
    @ROBYER1
    (Case 1162746) [LWRP] Quest FFR (Fixed Foveated Rendering) not working on LWRP
    Is open with Unity since June, no reply at all so far.

    From SRP changelogs it seems they are refactoring XR all over the engine right now, so it's in another "let's fix a broken system by introducing a new preview system which might be broken" state I'd say. Hopefully Unite gives us better insight.
     
  41. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    272
    I've also reported it with UniversalRP here (Case 1184851) UniversalRP - Oculus Quest/Go Fixed Foveated Rendering not working in URP/LWRP
     
    fherbst likes this.