Deferred Renderer + Multiple Cameras + Posts/SSAO = Bugged! (UFPS, OnRenderImage())

Hi,

it seems something strange is going on if u use the Deferred Renderer (Pro Version) and have multiple cameras setup with Post/Image effects.

As example UFPS uses the default FPS camera setup, which means having everything, except the weapon model in the main scene and overlay this with just a camera that renders the weapon model on top. This helps with z-fighting, since u can finetune the near-plane per camera. It also solves the problem of bigger weapon models clipping through near walls/objects.

The problem is that all image effects are broken for those setups and the Deferred Renderer. Don't get confused by UFPS inspector camera values, those will be correctly overridden per script on play.

So the main camera correctly has camera.depth set to 0, while the weapon is set to 1, so the render order is ensured, also the weapon camera is set correctly to only clear-z. The main camera also correctly renders everything except weapons and the local player, while the weapon camera only renders the weapon.

So any image based effect that is attached to the main camera should correctly process only this camera/rendertarget inside OnRenderImage(), while the weapon camera should have no impact at all, if no image effect is attached to it.

What happens is, as if the "OnRenderImage()" function inside the SSAO script on the main camera, gets the final render result including all cameras, therefor the z-values are already cleared by the weapon camera and SSAO cant work.

The correct way would be to call and process OnRenderImage() per camera, not after all cameras are rendered. I suspect this is some kind of bug and not the general behavior of the Deferred Renderer, since each camera should always be processed in order and separately, since otherwise it would be impossible to have different image effects per camera.

so @Unity whats going on here?


thx Andy

 

no one can help on this?

 

Man I need some screenshots, showing the problem.

Perhaps even a small sample project. I've used multiple cameras, deferred rendering and made it work. But I believe my main camera rendered last.

 

Did u attach image effects like the ambient obscurance/occlusion to the first rendered camera? The other way is to have 2 different setup image effects on both cameras, which should show that only the second version is applied.


I found the problem + fix and suspect its a wrong or missing resolve + copy in the deferred pipeline.

Here is the bug:
The first camera renders normally into the ScreenRT and than applies all image filters correctly in order, now the resolve+copy is missing, which means the second camera does not get the final image processed RT, but simply works with the normal screenRT, that has no image effects applied.

I suspect the normal screenRT is resolved into new separated RT for the "OnRenderImage(RenderTexture src, RenderTexture dest)" call to work and at the end the last RT is either not used as input for the next camera or is not copied over to the normal screenRT.

The fix is simple, at the end of the image effects a script copys the result back to the main screen RT, so the next camera correctly uses this as input.

I attached the script (CopyToScreenRT.cs), simply put it ordered lastly (after all image effect scripts) on the same camera that has image effects. This will make sure the processed RT is copied to the old screen RT the next camera gets as input.


bye Andy

PS: I also setup a example project (TestDeferredBug2.7z) showing the problem and fix in 2 scenes.

 

Your two scenes show the problem perfectly. In cases like this where you have a clearly wrong behavior with a simple repro you should just file the bug, and perhaps use the forum to raise awareness.

Anyways that is what I call a perfect repro project, I filed the bug report and passed it on to the developers!

Thanks

 

can you post a scene with multiple cameras? ssao works with script perfectly, but cameras - not.

 

The test scene i attached has 2 cameras, just check the main camera it has a secondary child camera, that mimics what normal FPS do.

 

maybe i dont understand you, but my goal - add another camera (so. main came will have 2 childs) and render at one camera weapon, at other - selected layer with glow, then main camera compose that. i achive that with simple cameras, but in UFPS - not.

 

Like i noted my script "CopyToScreenRT.cs" should be attached to all camera's that also have image effects like SSAO attached, while my script needs to-be ordered last. There is nothing else required. If something does not work u have to check your camera "depth" values which basically act like layers, means the cameras are rendered one by one from lowest to highest depth value. Also check your clear flags, the second and all following cameras should only clear depth, not color/skybox.

Maybe explain in more detail why your 3 cameras in UFPS do what is not working atm?

PS: Also keep in mind that UFPS sets the layers and depth values via script, so all inspector values are overridden, for the main and weapon camera.

 

seems - that a problem, because i changed manually depth, culling and other, nothing help. will check it, thank you)

 

I'd the same problem as you (2 cameras with deferred lighting and image effects), and I've used your fix script, and all works perfectly! Thank you Andy2222! Problem solved!

 

Awesome! we're using multiple cameras too... world, vehicle, viewmodel. this fix is priceless!

 

Just replying to thank you for fixing my DoF problem!! Previously the transparent parts were DoFed, so I split the rendering out to separate cameras... which killed the DoF effect!



You rule!!

 

Thanks for this script! It fixed all my problems with shaders like SSAO and the multiple camera setup in UFPS.

 

Works really well! however I found a bug when using this method and a shader that uses Grabpass. as soon as the shader goes on screen it's like the depth buffer doesn't get clear anymore and you only see second cam. I think it's more a bug with unity though...

 

I've been avoiding multiple cameras because they never worked with image effects, but if this miracle script fixes everything I will marry you.

 

This bug Still exists in 4.6b10!

And this script still fixes it, so not all bad

 

Hey!
Your script works very well, but I discovered a couple of similar bugs, clearly connected with deferred rendering and usage of multiple cameras.


Firstly, while using muliple cameras to render the environment with camera A and the gun with camera B, the gun does not receive shadows for some reason. This is also the case while using forward rendering.

Furthermore I discovered a gamebraking bug after updateing to Unity 4.3.4, which has not been fixed in Unity 4.5.
When I am using a second camera to render only some particular objects, the screen simple freezes when I am looking in certain directions. These directions seem to be random.

This makes my game unplayable and brought very much frustration with it.
It would be very nice if someone could help me with my problems or if anyone has to deal with similar issues.

Greets, Felix

 

I think I've experienced the same problem.

I found these 2 solutions:

• Set the Clear Flags property of the first camera to Depth only
• Enable anti-aliasing in the quality settings

Let me know if that fixes it

 

Did you ever get this script to work with UFPS? I am having no luck. My SSAO shuts off every time I press play and I'm thinking this is the problem.

 

Hi,

if you have further probems, try to set the main cam on where you have ssao to deffert, and the others to forward.
Seems it's a depth buffer issue. (mixed up the the others)

 

The grabpass problem has been resolved with another workaround, you can read about it here: http://forum.unity3d.com/threads/grabpass-problems-when-using-multiple-cameras.167676/

 

This still happens in U5 RC3

 

still happening.

 

Tought you saved my life, unfortunatly, it's not working.
Your test scene is not working too in unity5.

 

Still happening.

 

I confirm that @Andy2222 's solution is still working in Unity 5, you just need to modify his script to use "Deferred Shading" instead of deferred lighting which is now legacy.

Thank you Andy, you saved me many hours !

I'm attaching a copy of the modified script.

 

Are you sure this solution still works? I'm attempting to use it and for me the cached active RenderTexture goes null between OnPreRender and OnRenderImage. I've been using this solution for a while in my project, but cannot get it to work in Unity 5.

EDIT: I should have been more thorough, in my case RenderTexture.active is actually null in OnPreRender.

 

Does the "target texture" property of your main camera is set to something ?

 

No, none of my cameras are rendering to a different target. I have a 3 camera setup, one for rendering 2990-100000, another for 1-3000, and the final camera is 0.1-25 rendering the first-person perspective of my ship. The Far camera clears to skybox and the closer two clear Depth Only. All have HDR and occlusion culling enabled.

 

I've also run into this issue after switching to HDR and deferred on my current project. It's in Unity 5, though clearly this has been an issue for a long time. I tried that latest script and it didn't work for me. It's a bit frustrating, as there are multiple issues when you turn on all of the graphical goodies and you happen to be using multiple layered cameras (I have a six-camera setup).

For now, I've been applying all of my effects to a single camera, but that isn't really what I want.

 

Same issue here. Has it been reported yet? Can we expect some solution to this issue?

 

Also looking for a solution this is actually stopping us from moving to Unity 5. The changes to the CopyScreenToRT Script don't seem to work in Unity 5.

 

https://unity3d.com/unity/whats-new/unity-5.0

• OnPreRender script callback is called before rendering camera's depth texture now.

 

Anybody figured this out yet for Unity 5? I'm running into the same issue.

 

Same issue here, anybody know a solution to this?

 

I seems found a solution for Multi camera and image effects issue:

Lets say you have a Pistol gameobject that put it to a "Weapon" layer.

FPS Camera settings:
- Depth = -1. (default)
- Rendering path = Deferred
- Clear Flags = Skybox. (default)
- Culling Mask = unselect the "Weapon" layer.
- Apply the ImageEffect script (SSAO, GlobalFog..etc).

WeaponCamera settings:
- Depth = 0. (Render on top of FPS camera)
- Rendering path = Forward.
- Clear Flags = Depth Only.
- Culling Mask = select the "Weapon" layer only.

Attached the example project (zipped with mac)

 

Thank you Luckymouse. I've been searching for a couple days now to a solution for this in Unity 5.0 and yours is the first camera combination that actually works. Unfortunately, this prevents you from having image effects applied to both cameras. If anyone ever finds a way to handle image effects in a "stack" of cameras, using the depth buffers from all, please please please let me know

 

So, I've been struggling with this very annoying problem too, for quite some weeks now. Although Luckymouse "fix" works, it forces the first camera to be in forward mode, what can be quite disturbing when one needs the deferred pipeline (for e.g. Alloy or other stuff).
After searching Google to death about this topic and finding only other frustrated people like me, I tried every possible camera setting myself.
What I have is a 3-camera setting:
Cockpit Camera -> depth 0; 0.1 - 5 (Renders only the cockpit layer)
Near Camera -> depth -1; 0.1 - 1000 (Renders everything else)
Stellar Camera -> depth -2; 1000 - 9000 (This is a scaled space camera like they use in Kerbal Space Program, everythings to an 1:6000 scale)

What seems to break this camera setup:
1.) The first two cameras have overlapping frustums, even if they render different layers -> depth buffer wrecked.
2.) The first camera's frustum is too small, when I put the far value to >10, the depth buffer clears correctly.

So, still no really god fix out there - but maybe someone else will find something.

Edit: Well, also post-processing shader only work on the first camera, guess it's the known multiple-camera-grabpass-issue. Too bad that even with Unity5 this still persists.

 

It is my #1 issue with Unity. On the published roadmap, they've implied they're focusing on bug fixing next. Sure hope this issue makes the cut.

 

I cannot believe there is still no solution to this. This is such a common technique. Can someone from Unity please let us know where is this on their roadmap? This is the single issue that has prevented us from switching to Unity 5 for months now.

 

Ha, i just ran into this problem. Deferred + multiple cameras + asset store tonemapping = broken.
It completely works in forward, but not in deferred.

 

Here too, hope Unity get this fixed soon.

 

same problem here, Unity folks? @Aras any idea?

 

I found a solution that should work for some cases using rendertextures and full-screen quads. Will post more infos tomorrow, after a good night sleep.

Edit: More like tomorrow, after a second good night sleep^^

 

So, I really am not sure how far this can be of use to you guys, but here is what I needed and finally came up with (I'm a bit sick right now so please excuse if my thoughts are a bit chaotic):
The setup:
Cockpit Camera -> depth 0; 0.1 - 5 (Renders only the cockpit layer and to depth only)
Near Camera -> depth -1; 0.1 - 1000 (Renders everything else and to depth only)
Stellar Camera -> depth -2; 1000 - 9000 (This is a scaled space camera like they use in Kerbal Space Program, everything is on an 1:6000 scale and to the skybox)

I wanted to add different post-process fx on different cameras, which unfortunately broke the depth buffer. Also, effects on the second and third camera didn't worked at all.
What finally worked for me:
- Change the second camera clear flag to Color(0f, 0f, 0f, 0f) (It is important that the alpha is 0!).
- Place a full screen quad in front of the first camera.
- Render the second and third camera each to different rendertextures and combine them in a shader using the alpha from the second camera as lerp (see code and pictures below) and then placing the result as texture on the quad.

Edit: I forgot one thing. You need to deactivate the camera components on the second and third camera, because the script is handling everything.

Hopefully this will help you guys a bit!


(This is what it looks like inside the editor)

Code (CSharp):

1. using UnityEngine;
2.  
3. [ExecuteInEditMode]
4. public class FillScreenWithQuad : MonoBehaviour
5. {
6.     public float DistanceFromCamera = 0.01f;
7.     public Camera SourceCamera1;
8.     public Camera SourceCamera2;
9.     public int depth = 16;
10.     public RenderTextureFormat RTFormat = RenderTextureFormat.DefaultHDR;
11.  
12.     RenderTexture rt1, rt2;
13.     Camera cam;
14.  
15.     void Start()
16.     {
17.         rt1 = new RenderTexture(Screen.width, Screen.height, depth, RTFormat);
18.         rt1.Create();
19.         rt2 = new RenderTexture(Screen.width, Screen.height, depth, RTFormat);
20.         rt2.Create();
21.  
22.         this.gameObject.GetComponent<MeshRenderer>().sharedMaterial.SetTexture("_RT1", rt1);
23.         this.gameObject.GetComponent<MeshRenderer>().sharedMaterial.SetTexture("_RT2", rt2);
24.     }
25.  
26.     void Update()
27.     {
28.         if ((SourceCamera1 == null) || (SourceCamera2 == null))
29.             return;
30.  
31.         SourceCamera1.targetTexture = rt1;
32.         SourceCamera1.Render();
33.  
34.         SourceCamera2.targetTexture = rt2;
35.         SourceCamera2.Render();
36.  
37.         cam = Camera.main;
38.  
39.         float pos = (cam.nearClipPlane + DistanceFromCamera);
40.         transform.position = cam.transform.position + cam.transform.forward * pos;
41.         float h = Mathf.Tan(cam.fieldOfView * Mathf.Deg2Rad * 0.5f) * pos * 2f;
42.  
43.         var height = 2.0f * Mathf.Tan(0.5f * Camera.main.fieldOfView * Mathf.Deg2Rad) * pos;
44.         var width = height * Screen.width / Screen.height;
45.  
46.         //transform.localScale = new Vector3(h * cam.aspect, h, 0f);
47.         transform.localScale = new Vector3(width, height, 0f);
48.     }
49. }

(This is the Shader Forge node tree, it's pretty simple.)

 

Thanks it worked perfectly!

Though for some reason it broke in 5.2 (with deferred rendering), I had to revert back to forward rendering

Edit: Even in forward rendering I'm getting some visual artifacts
Edit 2: After some testing it seems the culprit of the artifacts is the new UI

 

Interesting Idea FreakForFreedom but it doesn't work for me. Are all 3 cameras in your post deferred? Once I start mixing deferred renderers whether the cameras are disabled and I'm only pushing to a render texture it still doesn't work.

 

This is a complete nightmare, I can only guess this is a for those in the know fix. Hence the silence from Unity devs? For those using UFPS there is a very simple workaround to multiple cameras and thats using Retraction camera methods. Basically you only have a single camera and make some changes to the UFPS setup to make your players hand retract when it comes in contact with a wall. Read the UFPS manual it's all there. Your also get GL shadows and be able to apply effects on a single camera if that fits your setup.

So any progress anyone? I'd love to get a solution to this so I can use fog with deferred cameras on every layer, otherwise I'm stuck with a single deferred cam and all above set as Forward - trashes month's of work with cool fx. That CopyToScreen script actually worked for me but only with the Unity Standard FX, I need to work with Colorful FX and SE Bloom.

 

Eureka! I've managed to get some results with my post FX from Colorful FX. So I used the above CopyToScreen script (thanks) and it worked but there is a gotcha! Unless I included a Unity Standard Effect my Colorful FX would not render through onto the top camera. In my case I just added the fisheye effect and set the params to 0 and then my Colorful FX would work with the CopyToScreen script set as the last component. As soon as I deactivate the fisheye it does not work???