Interaction between a few Unity subsystems causes weird "game-breaking" render bug

The original title would be "XR Plugin Providers breaks the rendering with multiple camera and layers setups when using the New Input System" but, as you can see, it would be too big.

Bear with me, this will be a big one.

Context (You can skip this section if you want)
Unity 2020.3.0f1, XR Plugin Management 4.0.1, OpenXR Plugin 1.0.3, Mock HMD Plugin 1.3.0-preview.1, Input System 1.0.2
I was having a weird bug, and when I went to the internet to search about it I found 2 other similar bugs that got me really puzzled. They had the same symptom but different causes. I'm posting here first because this bug is so intricate that I have absolutely no clue how to report it.

First, I'll explain what I was trying to do, the behavior that I got and what I learned when recreating a minimal scene to reproduce it.

I'm developing a VR experience for my thesis and I needed to render to the monitor the view of a independent camera (with a Picture-n-Picture of the user's view from inside the HMD). Not only that but this secondary camera will render some extra stuff that should be visible only on the monitor.
So, I placed in a new layer all those helper objects. The culling mask of the camera within the XR Rig was set to ignore that layer. I created the secondary camera and set its culling mask to everything. Same target display as the main camera but the target eye set to none (main display). To achieve the PnP effect of the HMD view in the display, I created an extra camera as a child of the XR Rig Main Camera, and set it exactly like its parent, except for its target eye, which I set to none. Using the viewport rect settings, I placed it in the corner with 1/8th its size.

The first bug happens here. Changing the depth of the cameras was giving some really weird artifacts. Like a camera with a depth of -2 being rendered on top of the one with a depth of 1. But putting -3 in the depth was working fine. -4 would be weird, -5 it'd be ok. And so on. However, setting everything with a positive depth seemed to solve that.

Alright then. In the game view inside the editor (not on playmode), it was working perfectly. When I pressed play and put my HMD on, it was working fine too. No helper object in sight. However, when I took it of to look at the monitor, the PnP camera was completely black. It was supposed to be rendering everything but the helper object.

That was bug nº 2. To make the PnP camera render properly, I simple had to deactivate HDR and MSAA on its properties (didn't even need to deactivate in the whole project).

Ok, now I got everything working, right? All cameras rendering and the correct content. So then, I tried to make a build to test the performance and then it got wild. In the HMD, things were still fine, but in the monitor nothing was being rendered but the skybox and the helper objects in both the secondary camera and the PnP camera. The secondary camera had to be rendering everything and the PnP camera should be rendering everything BUT the helper objects. I quadruple checked everything. The layers, the culling masks, target display and eyes. I even deleted all those cameras and started again but nothing. It was working perfectly fine in the playmode but in the build everything was wrong.

This was bug nº3. Cameras were not rendering the correct layers.

Trying to reproduce
To try to understand and fix it, I tried to reproduce the bug in a simple scene in a new project. After a long time I got it and it was even weirder than I thought.

First, I noticed that if I went to Windows > Analysis > Input Debugger and then deactivated Options > Lock Input to Game View, I didn't need to turn off HDR and MSAA on the PnP camera for it to properly render in the playmode.

Second, I noticed that, if I had no provider selected on the XR Plugin Management it would work fine. With or without the Lock Input. Both on the playmode and in the build! The only way to make it work on the build was to not have any provider selected. But then it wouldn't be a VR application.

Another workaround that I found was to use the old Input System. However, I'm so far into the development that the cost of downgrading is not worth it. And, if I understood everything correctly, this would work only on the playmode. The built one would still be broken.

In summary, what I found was that:
- Having any XR Loader active (Mock HMD, Oculus if you have an Oculus, OpenXR if you have a Vive, etc... Even if you have OpenXR with the Mock runtime... As long as it successfully loads) is going to break the camera rendering layers on the build and break the MSAA+HDR thing on both the playmode and build
- Having any XR Loader active but deactivating Lock Input to Game View fixes the MSAA+HDR thing only on the playmode, but you lose VR capabilities.

The other somewhat relevant posts that I saw were these two: https://forum.unity.com/threads/objects-not-showing-up-in-camera-observation.1056755/#post-6831602 and https://answers.unity.com/questions/1817785/render-layers-differing-between-editor-and-build.html

What is weird about them is that, as I said in the beginning, they seem to be the same issue but caused by totally different things. Since I don't have access to their projects, I can't try to figure out if we had something in common. To me, this feels like a really weird bug and I might not have properly found him.

Oh well, I did my best, but this is as far as I can go.

How do I workaround it? How do I report it?

Sample Project
Unity 2020.3.0f1, XR Plugin Management 4.0.1, OpenXR Plugin 1.0.3, Mock HMD Plugin 1.3.0-preview.1, Input System 1.0.2
I'm uploading a sample project here so you guys can test it out. You can play around with it a bit. Below is a brief description of how this project is laid out and how to trigger the bugs.

- Plane
A normal object in the default layer
- Cube
"helper object" in the "NewLayer" layer
- Main Camera (HMD)
A camera simulating what would be seen in the HMD, i.e., everything but the helper objects
- PnP Camera [TOP LEFT]
A camera with the exact same settings as Main Camera, but to be rendered in the main display
- Secondary Camera (MSAA+HDR OFF) [TOP RIGHT]
A camera that should render everything in the scene

Since the main camera isn't in a XR Rig (and doesn't even need to be in one to make this bug happen), its target display is none to ease testing, but you can change it to both if you want to see in your HMD. You can transform into an XR Rig if you want to have position+rotation tracking of your HMD, but again, not needed to test this.

- On editor as downloaded: Everything renders correctly
- Entering playmode as downloaded: Main Camera renders correctly, PnP will be black, Secondary Camera will render correctly
- Build+running as downloaded: Main Camera renders correctly, PnP will be black, Secondary Camera renders only the cube
- Deactivating Link Input to Game View and entering playmode: Everything renders correctly
- Deactivating Link Input to Game View and build+running: Main Camera renders correctly, PnP will be black, Secondary Camera renders only the cube
- Deactivating all XR Loaders and entering playmode: Everything renders correctly but no VR
- Deactivating all XR Loaders and build+running: Everything renders correctly but no VR

BTW: this sample project was not able to perfectly replicate the weird behavior on my application, but I believe it reproduces the same bug.

 

After trying out some more stuff, I've got some updates.

I managed to isolate bugs nº 2 and nº 3. Nº1 simply hasn't happened again and I couldn't reproduce, let alone isolate it. So I'm letting it alone. I just filled a bug report for them.

About bug nº2
Turns out that, as long as you are in forward rendering mode and have any XR Plugin Provider successfully loaded, having either HDR or MSAA active will cause the X and Y of a camera's view rect to be applied correctly for the clear but twice as much for the render.

About bug nº3
Still weird but I have found a workaround: Setting the clear flags of the cameras where the layer is mis-behaving to something else other than skybox seems to fix it. Since, in my project, all of the glitchy cameras are not for the user, this is fine by me. Even though it isn't an indoor environment. I have noticed that if I deactivate all XR Plugin Providers the bug disappears. Since my application is in VR, this is a no-go.

I'll add a link for the issues in the public tracker once they get published.

 

Updates:

Bug nº2 is fixed in 2020.3.3f1. It's not on the patch notes but talking with the bug reporting guys we found it out.
Bug nº3 is fixed in 2021.2.0a10. Since it's not on beta yet, a public issue was opened https://issuetracker.unity3d.com/is...ers-only-skybox-when-project-is-ran-in-player


This thread can be locked.