Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bloom only emissive surfaces in forward renderer

Discussion in 'Image Effects' started by argosy_ops, Jun 2, 2016.

  1. argosy_ops

    argosy_ops

    Joined:
    Dec 27, 2012
    Posts:
    49
    Hey there!
    So I was testing some different technical approaches with a scifi scene where various emissive materials contribute to the atmosphere. Since the game is made for VR, using MSAA is pretty much vital for the visuals, enforcing usage of the forward renderer.

    Now here's my problem: I want these lights (and only these lights) to be affected by the bloom filter, but I cannot use HDR and MSAA at the same time in Unity. Without HDR, I cannot reasonably set the bloom threshold to 1 and any value lower than that will bloom any somewhat brighter surface, which is then amplified to an uncomfortable degree by the lenses in VR - something I would like to avoid.

    Anyone with more shader knowledge than I have with an idea how to approach this issue?
     
  2. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    So i'm not going to answer the question you asked, but instead answer a different, and probably more interesting question.

    HDR + MSAA in unity. It can be done! Just not using the default rendering settings. The reason for this is because of how the backbuffer gets created and a bunch of legacy stuff. But if you know the hardware you are targeting has HDR and MSAA support it's pretty simple.

    In your quality settings disable MSAA, and set the camera to HDR mode. This will allow you to work in the scene quite well.

    Using this technique MSAA will only be enabled in play mode.

    Code (csharp):
    1.  
    2. using UnityEngine;
    3.  
    4. [RequireComponent(typeof(Camera))]
    5. public class HDRAA : MonoBehaviour
    6. {
    7.     public RenderTexture temp = null;
    8.     public bool AA = false;
    9.  
    10.     void OnPreRender()
    11.     {
    12.         var camera = GetComponent<Camera>();
    13.         temp = RenderTexture.GetTemporary(camera.pixelWidth, camera.pixelHeight, 24, RenderTextureFormat.DefaultHDR, RenderTextureReadWrite.Default, AA ?  8 : 1);
    14.         camera.targetTexture = temp;
    15.     }
    16.  
    17.     void OnPostRender()
    18.     {
    19.         GetComponent<Camera>().targetTexture = null;
    20.         Graphics.Blit(temp, (RenderTexture)null);
    21.         RenderTexture.ReleaseTemporary(temp);
    22.     }
    23. }
    24.  
    Just note: This will use more memory for the texture (8xmsaa in this example). I tested with bloom and it seemed to work pretty well :)

    No AA vs AA
     

    Attached Files:

    forestrf, clagrens, Pica and 8 others like this.
  3. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Oh interesting! I guess there is a minor penalty for another blit but other than that, can be used fine?
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Any ideas on getting this technique working with VR? Setting a render target disables stereo VR rendering on cameras.
     
  5. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    :/ I wasn't aware of that and am not a VR guru. I'll ask the vr team.
     
  6. argosy_ops

    argosy_ops

    Joined:
    Dec 27, 2012
    Posts:
    49
    For whatever reason, the notifications for replies in this thread only just appeared, totally missed out on this. Yes, a solution like this for VR would be highly appreciated. On other platforms, I could always go for deferred instead, but having MSAA + HDR in OpenVR would be an amazing step forward :)

    I'm curious: Is there anything beyond the additional memory one should be aware of using this technique? I assume there's a reason for Unity not allowing MSAA + HDR by default? In our case, we are targeting stationary machines with a GTX1080 configuration, so we have 8 gigs of memory available, should be enough :)
     
  7. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    They are not aware of this issue, any chance you could log a bug that they could look into?
     
  8. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    I think the main reason is that this part of our render pipe is a little old. Modern hardware supports HDR msaa (and so do we if you do it manually), but that part of our render pipe hasn't had the engineering love it needs. We have a pretty cool plan to address this that extends the concept of command buffers.
     
    argosy_ops likes this.
  9. Kaspar-Daugaard

    Kaspar-Daugaard

    Unity Technologies

    Joined:
    Jan 3, 2011
    Posts:
    150
    Are you using Unity 5.4? Stereo rendering should work with render targets and image effects, though you may have to make some shader changes for Single-Pass Stereo Rendering. We would appreciate a bug report about cases that don't work.
     
  10. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    The repro case is create an new project, enable VR support (single pass disabled), add that script to a camera, run. The result is a black game view. If you remove the script and add it while in play mode the view "freezes".

    It's even in your own documentation that this is the expected result.
    http://docs.unity3d.com/530/Documentation/Manual/VROverview.html
    The logical flip of that is any camera with a render texture does not get rendered "to your device", to which I assume to mean the front buffer.

    This doesn't cause issues for image effects or separate cameras that are rendering to a render texture as those are being composited into the main camera with at initial render time does not have a render target.
     
    Last edited: Aug 2, 2016
  11. pojoih

    pojoih

    Joined:
    Mar 30, 2013
    Posts:
    226
    Same Problem here, Unity 5.4f3 single pass disabled results in a black screen when using this script, specifically assigning camera.targetTexture.

    Looking further into it, the script seems to work with VR mode disabled, but putting the Bloom Effect (newest version from the Bitbucket repo) onto the camera results in a freezed screen as well (with VR enabled, it freezes the editor completly)
     
  12. amarchal

    amarchal

    Joined:
    Nov 19, 2014
    Posts:
    3
    Yep, as much as I was hoping I could just copy Tim-C's script and drop it into our VR project and have MSAA and HDR...I already knew it would not be that easy. The difference for the project we are working on of having MSAA is huge. How huge? Huge enough that we will drop back from the deferred renderer just to use MSAA...if I have to add glow cards to everything that previously bloomed nicely, I will do it. Out of the box no other AA type can touch MSAA under certain circumstances. Unity engineers, I beseech you to sort this out asap. Others devs and I need HDR support with MSAA....and it needs to work in VR with SPS. So if there is an alter somewhere that I could sacrifice a goat or an offshore account that i could deposit funds into that would speed this up, let me know.
     
  13. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Noted. I'll be cracking some skulls soon about this.
     
  14. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    Also I'll need pics of that goat sacrifice.
     
  15. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    @Tim-C Rather than fixing this I'd rather just have HDR and MSAA work in Unity natively, with some way of defining an inverse tonemapper to be used, along with the Cinematic Tonemapping & Color Grading component supporting it.
     
    argosy_ops likes this.
  16. Tim-C

    Tim-C

    Unity Technologies

    Joined:
    Feb 6, 2010
    Posts:
    2,221
    massive agreedo.
     
    argosy_ops likes this.
  17. argosy_ops

    argosy_ops

    Joined:
    Dec 27, 2012
    Posts:
    49
    Haha, agree! We were exhibiting at gamescom last week and being one of the smaller VR exhibitors at the show, people actually came to us and told us how our graphics looked nicer / crispier than most of what they have seen at the larger companies, who must've looked rather blurry most of the time. So it's really worth choosing MSAA/forward for VR. I'm really looking forward to that HDR fix, as that will ramp up the graphical fidelity even more, especially with emissive surfaces :D
     
  18. Fallc

    Fallc

    Joined:
    Mar 13, 2015
    Posts:
    48
    For those who can't wait... I managed to make it work in VR. It's semi-efficent, but together with single-pass rendering, the performance impact should be rather minor.

    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. [RequireComponent(typeof(Camera))]
    4. public class HDR_MSSA_VR : MonoBehaviour {
    5.  
    6.     private Camera msaaCam;
    7.     private GameObject msaaCamGO;
    8.  
    9.     public LayerMask LayerMask;
    10.     public bool MSSA = true;
    11.  
    12.     public bool didRender = false;
    13.  
    14.     void OnRenderImage(RenderTexture source, RenderTexture destination) {
    15.         RenderTexture rtMSSA_HDR = null;
    16.  
    17.         var srcCam = GetComponent<Camera>();
    18.         if (msaaCamGO == null) {
    19.  
    20.             msaaCamGO = new GameObject("tmpCam");
    21.             msaaCamGO.hideFlags = HideFlags.DontSave;
    22.             msaaCamGO.transform.parent = this.transform;
    23.  
    24.             msaaCam = msaaCamGO.AddComponent<Camera>();
    25.         }
    26.  
    27.         msaaCamGO.SetActive(true);
    28.         msaaCam.CopyFrom(srcCam);
    29.         msaaCam.cullingMask = LayerMask.value;
    30.         msaaCam.depth = -100f;
    31.  
    32.         rtMSSA_HDR = RenderTexture.GetTemporary(source.width, source.height, 24, source.format, RenderTextureReadWrite.Default, MSSA ? 8 : 1);
    33.  
    34.         msaaCam.targetTexture = rtMSSA_HDR;
    35.         msaaCam.Render();
    36.         msaaCam.targetTexture = null;
    37.         msaaCamGO.SetActive(false);
    38.  
    39.         Graphics.Blit(rtMSSA_HDR, destination);
    40.         didRender = true;
    41.  
    42.         if (rtMSSA_HDR != null)
    43.             RenderTexture.ReleaseTemporary(rtMSSA_HDR);
    44.     }
    45. }
    46.  
    Put the script onto your camera, before all post-effect stuff. Also set your camera culling mask to "Nothing", since this script basically renders the scene on a seperate camera and blits it back into the vr-camera post-effect chain. Together with valves lab-renderer, this should be epic! :D
     
  19. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    Just need to figure out how to cram in some basic inverse tone mapping now.
     
  20. Fallc

    Fallc

    Joined:
    Mar 13, 2015
    Posts:
    48
    here you go!
    no tonemap.JPG tonemap.JPG

    It's done by in-place tonemapping, so you have to put "NeutralTonemapping()" (in InverseTonemapping.cginc) right on every output in every pixel shader you use. For surface shaders, it can be done by final color modification, but I didn't testet that. Once the everything is drawn, the script does the inverse tonemapping and everything is ready for futher processing :) . Downside is that it breaks blending, so one may do all particle etc. stuff on a seperate buffer. But since I need this for a project, I may post a solution here soon.

    Interessting: with tonemapping, ARGB2101010 is indistinguishable from ARGBHalf (which even glitches with high values on my R390), even for values beyond 2000.0!

    Credit goes to Brian Karis's blog post: http://graphicrants.blogspot.de/2013/12/tone-mapping.html .
     

    Attached Files:

  21. DennisVH

    DennisVH

    Joined:
    Jul 8, 2012
    Posts:
    48
    This already IS epic on its own. Thank you! Works like a charm.

    Oh and for anyone else using it, don't forget to switch to forward rendering and adding:
    Code (CSharp):
    1.  
    2.     private Camera srcCam;
    3.  
    4.     void Start()
    5.     {
    6.         srcCam = GetComponent<Camera>();
    7.         srcCam.cullingMask = 0;
    8.     }
    9.  
    allows you to still use the camera in-editor when you're not in play mode (otherwise it's black when you set cullingMask to nothing).
     
    Last edited: Oct 4, 2016
  22. DennisVH

    DennisVH

    Joined:
    Jul 8, 2012
    Posts:
    48
    I discovered with single pass stereo rendering the specular lighting of my realtime point lights weren't visible in the right eye. Narrowed it down to this script as the root cause.

    So for now turning off the single pass rendering, but was hoping someone had the same problem and any idea how to solve it.
     
  23. pojoih

    pojoih

    Joined:
    Mar 30, 2013
    Posts:
    226
    Most awesome, you guys!! Finally a working solution for VR of my most annoying problem. And until the new PostPro stack isn't suited for single pass rendering, there's no need for it.

    Little refactoring and speedup:
    Code (CSharp):
    1.  
    2. using UnityEngine;
    3.  
    4. [RequireComponent(typeof(Camera))]
    5. public class HDR_MSAA_VR : MonoBehaviour {
    6.   private Camera msaaCam, srcCam;
    7.   private GameObject msaaCamGO;
    8.   private RenderTexture rtMSAA_HDR;
    9.  
    10.   public enum MSAALevel {
    11.   None = 1,
    12.   Two = 2,
    13.   Four = 4,
    14.   Eight = 8
    15.   };
    16.  
    17.   public MSAALevel antialiasLevel = MSAALevel.Two;
    18.  
    19.   void Start() {
    20.   srcCam = GetComponent<Camera>();
    21.   if (msaaCamGO == null) {
    22.   msaaCamGO = new GameObject("msaaCam");
    23.   msaaCamGO.hideFlags = HideFlags.DontSave;
    24.   msaaCamGO.transform.parent = transform;
    25.   msaaCam = msaaCamGO.AddComponent<Camera>();
    26.   }
    27.   msaaCam.CopyFrom(srcCam);
    28.   msaaCam.tag = "Untagged";
    29.   msaaCam.depth -= 100f;
    30.   srcCam.cullingMask = 0;
    31.   }
    32.  
    33.   void OnRenderImage(RenderTexture source, RenderTexture destination) {
    34.   msaaCam.transform.localPosition = Vector3.zero;
    35.   msaaCam.transform.localRotation = Quaternion.identity;
    36.   msaaCamGO.SetActive(true);
    37.   rtMSAA_HDR = RenderTexture.GetTemporary(source.width, source.height, 24, source.format, RenderTextureReadWrite.Default, (int)antialiasLevel);
    38.   msaaCam.targetTexture = rtMSAA_HDR;
    39.   msaaCam.Render();
    40.   msaaCam.targetTexture = null;
    41.   msaaCamGO.SetActive(false);
    42.   Graphics.Blit(rtMSAA_HDR, destination);
    43.   if (rtMSAA_HDR != null)
    44.   RenderTexture.ReleaseTemporary(rtMSAA_HDR);
    45.   }
    46. }
     
    Last edited: Nov 24, 2016
  24. OlliQueck

    OlliQueck

    Joined:
    Apr 11, 2013
    Posts:
    49
    works, thanks this saved me a lot of time since i was trying to achieve something similar. Just one more thing... the depth buffer is gone after the MSAA applying... is there a quick solution for this?
     
  25. jvo3dc

    jvo3dc

    Joined:
    Oct 11, 2013
    Posts:
    1,520
    This is very interesting. Probably a bit easier to set up than TAA with VR, but that also looks very promising. The nice thing is that in VR you can pretty much disable the jitter on TAA, because the camera is never really static anyway. And that removes those TAA jitter artefacts. The question is what the difference in cost is.
     
  26. j2l2

    j2l2

    Joined:
    Mar 14, 2016
    Posts:
    14
    Wow, great news!
    I'm happy to see a thread building a true solution for all, without useless bashing about Unity.
    Thanks to all.
    I had good results with The Corridor + Lab Renderer + HDR + MSAA, except for most of ParticlesVentSteams.
    Did someone also tried this setup?
     
  27. OlliQueck

    OlliQueck

    Joined:
    Apr 11, 2013
    Posts:
    49
    Using HDR + MSAA breaks the Depthbuffer for me after baking GI, even in 5.6 where it's implemented the depth buffer gets broken after baking GI. It's great that unity promptly included this feature in their new beta though:)
     
  28. OlliQueck

    OlliQueck

    Joined:
    Apr 11, 2013
    Posts:
    49
    my fault, i used DepthTextureMode.DepthNormals on my camera but decoded the sampler2D _CameraDepthTexture instead of sampler2D _CameraDepthNormalsTexture. That way it should work...
     
  29. ekergraphics

    ekergraphics

    Joined:
    Feb 22, 2017
    Posts:
    257
    Well, I'm a newbie so I could not get this to work. Either the stereo got completely screwed up, or nothing showed up due to the culling mask settings. Also, when I tried to revert what I had done, none of my Teleportation Points from SteamVR showed up anymore (although they were still functioning). Had to roll back the entire project.

    Don't mess with stuff you know nothing about, I guess is the lesson learned here. :)
     
  30. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,329
    While I didn't experience anything as bad as needing to roll back the project, I definitely was able to get my project into a state where shadows stopped working until I restarted the editor. I have another attempt at this script using a slightly different technique (one that doesn't require messing with layer masks) that has a one line change which, if commented out on the first run, works with or without it, but breaks stereo if the first time I run it's not commented out and uncommenting doesn't fix it.

    Unity's VR systems are a little fragile when doing this kind of manipulation, so it's nice Unity 5.6 finally has this working out of the box ... though I haven't actually tried using it yet to see if it's doing proper inverse tonemapping or not.
     
  31. Anshul31

    Anshul31

    Joined:
    Dec 8, 2018
    Posts:
    2
    Hi!

    I'm trying to implement the HRD_MSAA_VR script in unity but the left eye displays greyish color and the right eye blank. In the play mode of unity, everything works fine. What could be the issue?
     
  32. RyanKeable

    RyanKeable

    Joined:
    Oct 16, 2013
    Posts:
    62
    Ressurecting this ask if anyone has got this working in Unity URP 2019? Specifically on the Oculus Quest?