Question MRTK and extremely anomalous performance from FrameEvents.xrBeginFrame

Hey everyone,

I'm porting a VR project from 2019.2 and Oculus to 2021.2.4f1 and the HP Reverb G2.

I'm trying to use the Windows Mixed Reality Toolkit -- seemed like a good fit, and we may need other platforms besides HP Reverb.

I'm on an i9 with a 2080. Without VR I'm running in the 120 fps + range in Editor.

However, when I run in editor, I'm getting 2-3 fps due to 100-200ms on FrameEvents.xrBeginFrame. Occasionally I get 10 fps due FrameEvents.xrBeginFrame being ~ 50-75 ms. I am even seeing this sort of behavior in the MRTK example scenes that have almost zero content or performance cost, and runs outside of VR in the 600 fps range.

Where this gets CRAZY: I can change my Game view display from Display 1 to Display 2, see the performance run at a solid 90 fps, and then put it back to Display 1 and see a continuation at 1-2 fps. (fully expected, nothing surprising here). If I open the MRTK Optimize Window, it defaults to the "AR Headets" dropdown -- I can change this to VR Tethered which is our use case (this value doesn't seem to save, I don't think it actually does anything). Nothing surprising yet.

But if I do BOTH of the above, while the game is running, I have about a 20% chance of snapping into a good state. Once in this good state, I see a solid 90 fps with full tracking, full VR, and the full resolution. It looks and runs GREAT. The good state lasts until I exit play mode.

This says this anomaly is not a normal performance drop, as demonstrated by the PC's capability to drive and maintain the 90 fps good state version in the editor.

My understanding is there is some locking so that things are frame-synchronized, but somehow it looks out of sync until it gets back into sync with the magic combination of Display switch (perhaps unblocking a Fixed Update death spiral?) and the Optimize window (perhaps something within the MRTK, perhaps just an expensive editor window). I have not been able to trigger the good state through any other means, and can confirm that the game is doing the exact same thing in both good and bad states.

Everything I can find on the topic speaks to "performance will be worse in the editor -> standard performance tricks", but this is an already VR optimized project and when we are in the good state I can demonstrate that performance is not an issue. In the good state, my GPU time is about 6.3 ms, so we should have plenty of headroom. CPU is a bit spicier, but is typically about 11 ms.

Is there any information on what I might be seeing? I've tried turning off vsync without any change. Is there any way to unlock the waiting for FrameEvents.xrBeginFrame behavior? Is this potentially reprojection related, and is that something I can experiment with in Unity / the MRTK? Is this potentially related to some other MRTK setting?

On the MRTK side, I've tried to strip all that down to the minimum. No input, no boundary, only thing being the Open XR camera.

Is this something related to the MRTK itself, and perhaps our best result is to ditch Windows MR and go Open XR? What exactly is the relationship between the MRTK and Open XR? What would be the correct API / setup for an HP Reverb based system?

--

I have attached a screenshot of the profiler showing the good state and then bad state on a new play session, as well as my MRTK settings.

Note, I'm using the Multi Pass setup as some of our shaders require it, but when I'm in the good state this is a non issue. Additionally I get the same performance anomaly with Single Pass Instanced.

 

So the plot thickens a bit.

If I turn off HDR or MSAA on the camera, I get mostly the 'good state' above. Not quite as solid (the good state above has almost no time spent waiting for xrBeginFrame), but basically what I'd expect for in the editor. 60-90 fps, some xrBeginFrame related spikes. I'm curious to see how this works in a build as I expect that may actually go away. The key thing is we're at the point of viable in the editor.

We originally had the FP16 HDR (plenty of bloom) and 8x MSAA. I find that turning this to the R11G11B10 HDR and reducing MSAA to 4x results in the same improvement that turning it off on the camera does.

Further, it seems that Free Aspect for the game view is helpful too.

I'm wondering if there is something video memory related and that if it goes over the top everything just jams up? That doesn't quite explain why there was that 'good state' above, though perhaps something about that magic recipe caused the memory to be under some threshold.

Is there anything here that matches documentation or expectation? Is the MRTK still the correct approach for something like this, or should I be going more direct for Open XR?

If this is memory related, is there a way to downscale resolution in the headset, perhaps during the editor only?

 

Any update on this? We are getting the same issue even without the MRTK .

 

Basically dropping the MSAA, using the R11G11B10 HDR, Free Aspect in the editor and using the Microsoft Open XR Runtime all seem to be the happiest. Something seemed weird with the Steam VR runtime. I honestly don't know where this bad state happened but we're in a good spot now. Hope something above helps and let me know if you find anything concrete

 

I'd also be curious for a fix here - I'm seeing unusably bad performance in a near-empty scene on a 3080 on a HP Reverb G2.