XR Rendering Broken for Right Eye with Custom Shaders (Unity 2020.1.5, Windows XR Plugin 4.0.2)

I have several custom shaders, some very simple variants of the built-in shaders, some more complicated volume rendering shaders. Just migrated from 2019.4 to 2020.1 and switched over from the built in XR system to Windows XR plugin 4.0.2 (note: I also tried Window XR plugin 3.3.1 without success)

Each of the below was working correctly before migration.

Simple variant of the built-in default UI "sprite" shader (see uploaded shader)
Note: icons at the bottom of the screen are not present in the right eye rendering, also transparent clouds can be seen coming down below the mountains in the left eye, and not in the right.


The following is actually a second scene, but a static image from the first scene gets stuck on the right eye unless I disable my custom OnRenderImage script on my camera in the second scene.
Here several custom shaders are used to do a volumetric rendering of a cube.
Note: only the custom rendering is complete of the cube, OnRenderImage should be merging the volumetric rendering with the rest of the scene using Graphics.Blit(source, destination) and we should see a black skybox as well as controllers and UI which are not rendered.


I have tried switching between UWP and Windows Standalone platforms with no luck.
This seems to be a "multipass" issue, but the settings no longer exist as they did in the built in XR system. I have output the Rendering mode during playback to see what is being used and it does claim "StereoRenderingMode: MultiPass"

 

@ThomasZeng Thank you for the quick reply! I will try this out immediately and let you know how it goes. Out of curiosity, why does the system report "StereoRenderingMode: MultiPass" at runtime even though it is not supported, as you said?

EDIT: works for the shader you edited, now I'm having a little trouble with my volumetric shader in that the image in the right eye is the same as that in the left (not stereo), I think that it is related to using _WorldSpaceCameraPos and it now returning the same position for both left and right eye in the shader frag, do you have any suggestions here? Is there an alternative to _WorldSpaceCameraPos for SPI or maybe I should be passing this info in the v2f into frag instead?

 

@ThomasZeng I did try this, but am still getting the same image as though _WorldSpaceCameraPos is still returning the same value for both eyes. Any other ideas?


float4 frag(v2f i) : COLOR
{
UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(i);
float3 viewDirection = normalize(i.wPos - _WorldSpaceCameraPos);
...
...

I have even tried using _WorldSpaceCameraPos from within the vert function and storing it as another TEXCOORD in v2f, but no luck.

 

@ThomasZeng actually it appears that the problem is NOT with _WorldSpaceCameraPos, but is happening earlier with the vertex positions... I'll investigate further, but I am not using URP at this time, just the built in default render pipeline.

 

@ThomasZeng so it seems to be an issue with Graphics.Blit(source, destination, blendMat) and/or with my blendShader (attached), OR with rendering to a renderTexture from the MainCamera:

private void OnRenderImage(RenderTexture source, RenderTexture destination)
{
...
MainCamera.targetTexture = volumeTex //a RenderTexture;
MainCamera.Render();
...
this.blendMat.SetTexture("_VolumeTex", volumeTex);
Graphics.Blit(source, destination, this.blendMat);
}


I am wondering if there is a specific way I should create a temprorary renderTexture, ie:
volumeTex = RenderTexture.GetTemporary(source.width, source.height, source.depth, source.format, RenderTextureReadWrite.Default, 1, RenderTextureMemoryless.None, VRTextureUsage.DeviceSpecific);


Let me know if anything obvious stands out to you,
Thanks for all your help thus far!


EDIT: when I use
volumeTex = RenderTexture.GetTemporary(source.descriptor);
I get the following error at runtime:
Error assigning 2DArray texture to 2D texture property '_VolumeTex': Dimensions must match
If I change the shader property to a 2DArray, it works for the left eye only.
I think the standard SetTexture("_TexName", RenderTexture) is mistakenly treating the vr renderTexture as an array, and therefore not allowing it to be assigned to the material

 

To summarize a simple repro:

0. using XRSDK WMR
1. have a simple post processing Material using a shader that has 2 2D texture properties "_MainTex" and "_VolumeTex".
2. use the following to make the shader SPI compatible:
UNITY_DECLARE_SCREENSPACE_TEXTURE(_MainTex); //Insert
UNITY_DECLARE_SCREENSPACE_TEXTURE(_VolumeTex); //Insert
3. have an active script that uses OnRenderImage(RenderTexture source, RenderTexture destination).
4. attempt to assign the source (XR) RenderTexture to the material _VolumeTex from (1).

Fails with:
Error assigning 2DArray texture to 2D texture property '_VolumeTex': Dimensions must match

 

@ThomasZeng
After even further investigation, I am able to get the _VolumeTex to work if I do not define it in properties, however, the renderTexture (volumeTex) is still only displaying the left eye, whether I pass it as the _VolumeTex or _MainTex, while the source renderTexture is behaving correctly whether I pass it as _VolumeTex or _MainTex. I think the issue is in using a copied camera to render to the temp render texture. I am creating a repro project now.

 

@ThomasZeng

Narrowed the issue down to rendering from a camera to a render texture does NOT render the right eye.
Very simple repro:
Add the attached script to the scene. Press play. Only left eye is rendered.

Please let me know if there is a variant of Camera.Render() that is SPI compatible, or if this is a known issue.

Thanks

 

Will do, I'll try to get this done Sunday!

 

@ThomasZeng I have issued a bug report (Case 1279112)

 

@ThomasZeng
Yes this is a migration effort from multi-pass to single-pass rendering: when using multi-pass, OnRenderImage occurs twice, once for each eye, where Source and Destination are the expected eyes defined by Camera.stereoActiveEye of the rendering camera, therefore I am able to correctly use Camera.Render() once for each stereoscopic eye. Now, with single-pass, there is only one call to OnRenderImage which passes a stereoscopic image as Source, and expects a stereoscopic image as Destination, however, Camera.stereoActiveEye is now always left, and there is no way to use Camera.Render() to render the right eye to a render texture to do post-processing for the right eye. Is there some method to achieve Camera.stereoActiveEye = Right during single-pass??

 

@ThomasZeng
It seems that what I'm looking for is a time during rendering that Camera.stereoActiveEye == Right. I am using shaders that need to use shaderReplacement (and other stereoscopic eye specific post-processing rendering):
https://docs.unity3d.com/Manual/SL-ShaderReplacement.html
I have checked the value Camera.stereoActiveEye at OnPreRender, OnPostRender, OnRenderObject, and OnRenderImage, all have the value set to Left. At Update and LateUpdate the values are set to mono.

 

@ThomasZeng ,

I am wondering if there has been any further development on this as this issue breaks post-processing with single-pass instanced VR rendering and our team is not able to migrate from Unity 2019 to 2020 until this issue is solved.

Again the issue lies in Camera.Render() only rendering the left eye during single-pass instanced VR rendering. This was not an issue with multi-pass rendering as Camera.Render() would work correctly for each eye, but now that Unity has removed the option for multi-pass in 2020 we are forced to use single-pass instanced where Camera.Render() no longer works for the right eye.

Thanks again,

Chad

 

Hmmm, this might also explain the issue I'm having. Will keep an eye on this thread just in case.

 

@badawim-idcc @ThomasZeng See also my filed bug report status here:
Unity Issue Tracker - [XR SDK][WMR] Single-Pass Instanced Camera.Render() does not Render Right Eye (unity3d.com)

 

So is this still broken, or just abandoned?
I'm still only getting stereoActiveEye = Left for OnPreRender(), OnPostRender(), etc.