Irregular (rare) screen flickering when binding opaque framebuffer in pipeline (XRWebGLLayer)

I am currently working on a WebAR game using the WebXR Device API (https://www.w3.org/TR/webxr/) in "immersive-ar" mode. I figured out how to get it to work with Unity WebGL by substituting framebuffers, but I am getting some rare, irregular (as in once or twice every 2-3 seconds) screen flickering. As it works perfectly 99.9% of the time, I'm suspecting it probably has nothing to do with WebXR and I'm just missing some nuance in the Unity WebGL rendering pipeline.

Quick background: The WebXR Device API allows you to create an XRSession which exposes something called an opaque framebuffer (https://developer.mozilla.org/en-US/docs/Web/API/XRWebGLLayer/framebuffer). This opaque framebuffer contains imagery from the camera but prevents any reading operations on it out of privacy concerns, which means it is write-only.

The setup: I logged the pipeline functions, and it appears Unity WebGL uses multiple framebuffers stored in an array, with the framebuffer at index 0 being the "output" framebuffer of a cycle (correct me if I'm wrong!). This resulted in this .jslib:

Code (JavaScript):

1.  
2. // Edited glClear to keep the scene transparent
3. glClear: function(mask)
4. {
5.         if (mask === 0x00004000)
6.         {
7.             var v = GLctx.getParameter(GLctx.COLOR_WRITEMASK);
8.             if (!v[0] && !v[1] && !v[2] && v[3])
9.             // We are trying to clear alpha only -- skip.
10.                 return;
11.         }
12.         GLctx.clear(mask);
13. },
14.  
15. //Bind the WebXR framebuffer when the index is 0 and the mainloop is running
16. glBindFramebuffer: function(target, framebuffer)
17. {
18.         if (framebuffer !== 0) {
19.             GLctx.bindFramebuffer(target, GL.framebuffers[framebuffer]);
20.         } else if (/*WebXR framebuffer not null AND mainloop is running*/) {
21.             GLctx.bindFramebuffer(target, <WebXR framebuffer>);
22.         } else {
23.             // Only called during setup when mainloop is not yet running
24.             GLctx.bindFramebuffer(target, null);
25.         }
26. }

The code above binds the resulting graphic data to the write-only framebuffer. And this just works, as I get to see a spinning cube on the screen (which is the scene I am testing). The only thing that should not happen is flickering in the cube, and only in the cube. The underlying WebXR imagery is continuous and free of any flickering. It happens irregularly with usually .5 to 2 seconds in between.

Some thoughts and remarks:

• WebXR has an FPS of "nearly" 30. Not 60 as Unity WebGL considers standard. (A syncing issue?)
• It's not a requestAnimationFrame issue, I replaced the window.requestAnimationFrame with the XRSession's requestAnimationFrame function, and debugging the count of frames (in Unity's Browser object and on the XR side) shows absolute consistency.
• I tried messing around with antialiasing/vsync settings, but nothing seemed to have any effect. I did not try every combination of these settings.
• It may also be unrelated to glClear, as when I don't clear anything (and just "return" on _glClear without doing anything), the spinning cube frames just gets drawn and drawn on top of each other making it look like a cilinder over time. And even then the cilinder flickers from time to time.
• The flickering happens once guaranteed in the beginning, and usually takes 5-6 seconds to manifest itself as a more noticeable, recurring thing (computational load related?).
• The more objects are in the scene, the more the flickering seems to happen, although this may just appear so because it's just more noticeable when more things are being rendered.

Can anyone with substantial knowledge about the rendering pipeline shed a light on this? Let me know if I have to debug/log certain things/functions for further information.

Thanks in advance!

 

This was the only thing that came up on google, so I'll leave my solution here.
I don't use Unity, I code directly in webgl javscript , but the principle will be the same.

This symptom happens to me in Chrome 81 WebXR AR support. I had a mad flicker because I rendered a high polygon character with three dynamic lights.

The problem is in the asynchronous OpenGL pipeline. That is, render callback being complete, while the rendering itself isn't complete.

After knowing the cause, the solution was simple - block the OpenGL pipeline until it completes. Specifically, call glctx.ReadPixels at the end of the render process. Read small bytes from the framebuffer and cause the pipeline to block.

Also, glctx.Finish didn't work for me.

I hope it helps anyone come across this thread in the future

*EDIT) I've completed my implementation. There's some performance hit, but ReadPixels seems to be the answer.

 

Hey,

I saw this thread just now, but I also got to implement the WebXR immersive-ar session.
I didn't get those flickering frames that you are talking about.

In my case,
I use inline XRSession rAF when the WebGL starts, instead of window.rAF.
And I use the relevant immersive sessions rAF for VR/AR.
At the start of each frame, I use the XRFrame data for the views and the glLayer framebuffer.
And only then I call Unity's frame loop, when the XRFrame's glLayer framebuffer is already bound.

You can look for the lines

Code (JavaScript):

1. let glLayer = session.renderState.baseLayer;
2. this.ctx.bindFramebuffer(this.ctx.FRAMEBUFFER, glLayer.framebuffer);

In this file:
webxr.js

The code git repo:
https://github.com/De-Panther/unity-webxr-export

You can check the demo here:
https://de-panther.github.io/unity-webxr-export

Regarding the FPS, 30 is the right frame rate for the immersive-ar session on chrome for android.

Regular WebGL FPS in most of the browsers is based on the screen refresh rate, in most android phones it'll be up to 60, in the Oculus Quest browser it'll be up to 72, on desktop with a monitor that defined as 60 Hz you'll get up to 60 FPS, on the same machine with a monitor the defined as 240 Hz you'll get up to 240 FPS.

Although there are devices that can process the AR Frame data(camera image + pose) 60 times per second, it seems like the Chrome dev team decided to start with 30 times per second, as low end supported devices can process the image only 30 times, so it means that you'll render the same camera image more than once.
You can see a discussion about it here and here

 

Also your glClear edit will skip every time that it'll get color mask with alpha only. You should make sure that it skip only in the first time of a frame.
There are calls to clear color mask with alpha only when using Unity Canvas for GUI, and you'll get some artifacts if you skip those calls.

 

Hey, I've got another one. The flicker depends on the device.

For example, my Samsung Note8 (Exynos CPU) has the flicker. LG V30 (Snap 835) has no flicker. Hardware specific irregularities - these things happen. So if any of you have flicker or no flicker, it might be that any of our codes aren't the problem.

To be safe, I'd say use glctx.ReadPixels to stall the pipeline. It fixes the flicker on my NOTE8.

 

Can you please check my demo and tell if it flickers on your devices?
https://de-panther.github.io/unity-webxr-export

 

Tested your demo with my problematic Exynos NOTE8, it seems okay, shows no flickers. Blends with the camera feed okay.

It could be that my program has done something (or haven't done what's necessary), and that's causing the problem in the Exynos Mali GPU.

Anyway, I'm done with this problem for now. I'm reading the WebGL Unmasked Render string, if it's not 'Adreno', blocking the pipeline with glReadPixels. Not an elegant solution, but it's working, so I'm going to leave it at that for now.

Later I'll get more test devices and tackle this again

** One More - I've got Samsung A50 today. It has the same flicker as NOTE8. It seems like the symptom of Mali GPU. Also, one thing to consider, De-Phanter's example runs at 30fps without skips. Mine runs at 10-15fps. So it's possible that I'm getting some kind of 'pipeline interruption'.

This thread is the first thing comes out when google for 'webxr flicker' Hope this helps someone in the future.

I also found the Android notch problem in A50. The notch must be turned off to get a correct input.