VR Single Pass Stereo with [ImageEffectOpaque] OnRenderImage

I have an effect that renders a skybox and fog at the end of the opaque queue, using OnRenderImage and [ImageEffectOpaque]. I'm trying to get this effect to work with Single-Pass Stereo, but it seems that it's not behaving the same way OnRenderImage works without [ImageEffectOpaque]. Instead of executing my shader once per eye, it's executed once for the whole buffer. Also, unity_StereoScaleOffset, unity_StereoEyeIndex, and _MainTex_ST don't contain any useful data.

The biggest problem I'm having is going from screenspace back to worldSpace, so I can sample the skybox texture. Which matrices should I use for this?

When doing a graphics capture, I see that there are borders around the left and right eye buffers. What's the purpose of these buffers? How can I programatically determine the size of these borders?

Is there documentation available for Single-Pass integration?

 

Currently UnityStereoScreenSpaceUVAdjust doesn't work, because this is a "ImageEffectOpaque" post process effect, and it seems that Unity forgot about "ImageEffectOpaque" techniques, so they work even less in VR single pass mode. This is unique to "ImageEffectOpaque" effects. If I remove "ImageEffectOpaque", UnityStereoScreenSpaceUVAdjust then works as expected.

 

Hi Robin,

Thank you for the reply!

The rendering of my image effect is fairly simple, I'm using Graphics.Blit(Texture, RenderTexture, Material, int pass), and am not using any intermediate render targets.

As far as I can tell, unity is then submitting a full-size quad to the GPU, drawing the post-process effect over all pixels in the buffer, including left and right eye, and the bordered areas. The behavior changes if [ImageEffectOpaque] is not specified before OnRenderImage

-Andrew

 

anollan:
Are you running 5.4.0b20?

I'm getting "undeclared identifier UnityStereoScreenSpaceUVAdjust" with 5.4.0b20 (32-bit), so this doesn't seem to be available yet. Also, none of the standard asset image effect shaders that came with 5.4.0b20 use UnityStereoScreenSpaceUVAdjust.

 

5.4.0b21 seems to have support for it. The fixed example shaders aren't available yet though (not in the standard assets bundle on the 5.4.0b21 beta download page anyway).

 

On b21 using UnityStereoScreenSpaceUVAdjust fixed several of my image effects, which is great. On b22 though, they're completely broken again.

If I modify an effect that worked on b21 to act as a simple pass-through effect (output == input) it still doesn't work at all.

 

@robinb-u3d Sure thing! Here's a project with pretty much just the failing effect in it: https://fogbugz.unity3d.com/default.asp?808089_nkn1ode1rb97eiaa

 

Here's a screenshot of the intended result (rendered without VR) and the current result (with VR and Single-pass), as taken from the test case included with the report.



As mentioned, the specifics of the shaders don't really matter. Something seems to be going wrong either before or after the fragment shader, not inside of it.

 

@robinb-u3d Thanks, that clarifies a lot. To gauge whether to invest time in this, do you have a loose indication of when this round of fixes will make it into a release?

 

When using UnityStereoScreenSpaceUVAdjust() with single pass the right eye blit is offset one pixel to the left when using the Rift (haven't tried on the Vive). This isn't super noticeable with most image effects but it results in the right eye being many pixels off when using bloom or similar image effects that use downscaled blur as 1 pixel of a 4x downscaled image is 4 pixels in the final image multiplied by each downscaling and blur pass. The this means the standard assets blur can be significantly offset by the time it renders to screen. Usually this isn't super obvious since multiple iterations usually results in a diffuse enough blur that the stereo disparity isn't jarring. If you use the bloom optimized script with the blur cranked down as low as possible and the intensity cranked up it's more obvious.

This is using 5.4b24, didn't check b25 yet.

This issue is not present on the PS4.

 

A related note is I think almost all of the standard effects are using UnityStereoScreenSpaceUVAdjust() incorrectly, though I haven't entirely confirmed it.

Example use case is often something like this:
uv = UnityStereoScreenSpaceUVAdjust(v.texcoord.xy + _MainTex_TexelSize.xy * 0.5, _MainTex_ST);

However I would expect the proper form to be like this:
uv = UnityStereoScreenSpaceUVAdjust(v.texcoord.xy, _MainTex_ST) + _MainTex_TexelSize.xy * 0.5;

Assuming it's supposed to work exactly like the TRANSFORM_TEX macro (which it does), and _MainTex_TexelSize is being set the same way it is in the usual non singlepass stereo case then applying the uv adjust to a line with a uv offset will cause the offset to be half as narrow as it should be. The result is down sampling passes will introduce vertical aliasing, and blur passes will blur more vertically than horizontally.


Also, unrelated to these issues, the down sample pass in MobileBloom.shader is done asymmetrically causing it to be way more aliased than it needs to be. The first sample is done with an offset of 1 texel, and the other 3 are half texel offsets. With the default hidden "low" resolution preset they should all be 1 texel offsets, and the "high" should be 0.5. Pretty sure this bug has been there for a few years.

 

The UnityStereoScreenSpaceUVAdjust issue has been posted as issue #813534

 

Hey @robinb-u3d, have these fixes been released in one of the beta releases yet? I'm using 5.4.0f1 RC 1 and fixing the ColorfulFog shader, which we use in our game. Looking at the GlobalFog.shader I was able to fix the double rendering issue that was present. Now the problem is that the canvas and some other meshes with some image effects are rendering upside down. One of these meshes uses the Particles/Additive material, if I change it to the standard shader, it fixes the flipping but doesn't render the transparency correctly.

Thanks!

 

Turns out the GlobalFog effect that ships with the standard assets package also has this issue of rendering things upside down. I submitted a test scene with the bug on issue #50895.