Horizon Based Ambient Occlusion - HBAO image effect

If the camera that holds the hbao effect is tagged as "MainCamera", simply do this:

Code (csharp):

1.  
2. using UnityEngine;
3.  
4. public class NewBehaviourScript : MonoBehaviour {
5.  
6.     HBAO_Integrated hbao;
7.  
8.     void Start () {
9.         hbao = Camera.main.GetComponent<HBAO_Integrated>();
10.         hbao.enabled = false;
11.    }
12. }
13.  

 

they [Unity] are really taking their time it seems

 

I've got a pretty critical bug fix that isn't getting fixed until 2017.3... you just gotta wait unfortunately...

 

I'm thinking about that dropdown setting we previously discussed @buttmatrix... That said I'm feeling bad asking the customer to set his HMD device type himself (no to think about what to do if you want to ship your app).

 

I see where you are coming from, but Unity already does this exact thing when they ask you to specify the SDK type in the player settings. It would be a small price to pay for having HBAO working steadily in VR, so I would go for it.

EDIT: I could see it being an inconvenience for shipped apps, but I'm sure there is a workaround

 

Dang, the problem is that my script is JS, apparently JS scripts can't reference C# scripts.

EDIT: Fixed it by placing HBAO in the Plugins folder.

 

Read this:
https://blogs.unity3d.com/2017/08/11/unityscripts-long-ride-off-into-the-sunset/

I can only advise you to try to switch to C# if you can...

 

Ok, I'll try to look at this on this weekend.

 

I know, I know, but I sadly made my entire game in JS and now I'm too lazy to switch.

Well, might start now. There are translators at least.

 

Quick fix - throw it in the plugins folder. Better to learn C# tho imho

 

Inverting the post effect won't look like this as mentionned earlier. However sampling a bit differently and inverting the factor can look like this:


Is it what you were looking for @Cascho01 ?

 

Hello, awesome pack.
I have a question though, I read that speed tree should be artifacts free but I get artifacts on all my speedtree trees.
Is there a way to fix it?

 

You need to set Per Pixel Normals unders AO settings to Reconstruct.

 

If the white is emissive - exactly!
Would appreciate to see it in your next update, especially if I could define the occlusion color, instead of white, by myself e.g. by exposing a custom color !

 

What you're seeing there is the AO only output, with AO being colored navy blue then the factor being inverted at the end.
So this white isn't emissive, as it is finally applied with multiplicative blending (where you see white nothing will change in the final output).

Maybe additive blending or 2x multiplicative is what you're looking for in this case ?

 

@jimmikaelkael got a bug for you (color bleed blurring / dithering around the handle), you may have seen this before. I know this is probably a combination of my settings somehow, but can you point me in the right direction?

 

Well, I'm not sure, I'd like to look at it in a test scene. Is it possible to get a sample scene with your HBAO settings in place and just the door/handle ?

 

Actually, in the process of preparing a sample scene I discovered that it was not HBAO acting poorly, it was my misuse of some lighting options. I apologize for the disruption.

EDIT: I activated a realtime light source in a scene that was fully baked (without enabling shadowmask)

 

Still interesting, thank you very much for feedback as a user reported similar thing in the past and never found what it was...

 

How to get same result as normal AO?
ps:normal AO is bake vertexcolor to mesh use other plugin

myfbx:https://1drv.ms/u/s!Akk8sdGmT1-GjCK-d_Yl1SPz6qjm

 

Try to lower the radius, and raising intensity if required. A medium blur with 8 as sharpness is good too.
That said HBAO being an SSAO solution: it is calculated every frame based on depth/normals and you'll never get the result you can get with an offline raytraced AO tool...

 

current I have some confusing:
if there two technology use together,the results is very dark,
if we don't SSAO, the results is not well as we respect.....
if we don't use offine AO,only use SSAO, the results is not well too...
how to mixed two way?

 

VertexDirt AO being a raytraced solution, it should deliver proper result as I think you can consider it as a ground truth AO.
Make sure your object scale is correct, make sure model normals are correct.

EDIT: Well just quickly checked your model it is 300K tris ?

 

Quickly checked HBAO output (latest 2.5 version, default medium preset) with your model freshly imported in a new Unity project:


This doesn't look correct from your point of view ?

 

@jimmikaelkael
Got a couple of questions about color bleeding:

• Is there a way to subtract emissivity written by shaders to deferred buffers from the color bleeding results? Right now color bleeding picks up all emissive pixels, making objects like streelight glass and lit windows enormously intense sources of color bleeding (revealing very sparse sampling clouds of dithered "fireflies" splatted around every single emissive pixel. Ideally, only albedo + lighting should contribute to color bleeding samples, so I was wondering if you could subtract deferred emission from final composite to solve this.
• Am I correct in assuming that color bleeding implementation is using separate samples from occlusion? Looking at frames where bright emissive surfaces are present, I see some sparse dithered samples scattered around in places not matching the AO itself. With that in mind:
  
• How hard would it be to strip all functionality not related to color bleeding from a derivative version of HBAO class, leaving only color bleeding effect intact? I'd love to be able to use your color bleeding implementation together with new volumetric multi-scale AO effect in Unity stack. If I were to do that, where would be the best way to start?

Here is an example of an issue with color bleeding being influenced badly by emissive materials, causing it to behave exactly like bloom:

• https://i.imgur.com/1ueZMr1.gifv

 

This looks similar to the issue I was facing here, but it did not involve an emissive material

 

Seems like a similar situation since that image depicts a very bright reflection (light source highlight or a probe-captured emissive material).

 

In that case, the situation disappeared after I reconfigured my lighting setup. At the time, I had a realtime light active in a scene that was fully baked, without activating pre-computed realtime in the lighting window or enabling shadowmask in the settings.

 

I'm sorry, but not really... RT0 comes with albedo, and RT3/CameraTarget comes with emission+lighting already mixed, no way to tell how much is emission only (https://docs.unity3d.com/Manual/RenderTech-DeferredShading.html).

Not really, the samples for color bleeding at collected exacly in the same place as occlusion samples, but only half the samples are taken.

Not that hard, but in this case you'll badly hurt performances: to calculate color bleeding we need the AO samples contribution and coordinates. Knowing that it is easy to understand that you make 3/4 of the work just to get color bleeding... All the computation power used to calculate AO will be lost.
It's better to integrate your own color bleeding in MiniEngine AO in this case, which should be fairly easy. And for where to start, look at HBAO_frag.cginc near the end of the fragment you should see how color bleeding is calculated.

The best I could do in HBAO is adding another multiplier so that you fully control how much albedo contributes and how much emission+lighting contributes.

 

@jimmikaelkael
Thanks, it all makes sense! Introducing a setting directly controlling contribution of emission and albedo components might be a very good idea! I had few other indirect ideas based on this limitation of deferred layout:

• Since emission is stored in an HDR format and lit surfaces typically have a very different HDR range compared to emissive surfaces (0.5-1.5 vs. 4-8, for example), it might be a good idea to introduce a factor based on remapping of HDR emission, which starts cancelling out color bleeding result if emission goes above certain level. Something along these lines:

Code (csharp):

1. float emissionBrightness = (emissionSample.x + emissionSample.y + emissionSample.z) / 3; // traditional dot-based luminance calculation is probably a bad idea on HDR color
2. float limitingFactor = saturate ((emissionBrightness - _EmissionLimitStart) * _EmissionLimitRange);
3. emissionSample = min (float3 (0, 0, 0), emissionSample - limitingFactor.xxx);

• Since emissive objects like glowing windows and lights often have an albedo value of 0, it might be a good idea to add an optional value that kills color bleeding taken from pixels with albedo below certain threshold. Something like this:

Code (csharp):

1. float albedoLuminance =  dot (albedo.xyz, float3(0.299, 0.587, 0.114));
2. float limitingFactor = saturate ((albedoLuminance + _AlbedoOffset) * _AlbedoMultiplier);
3. emissionSample = min (float3 (0, 0, 0), emissionSample - limitingFactor.xxx);

Both of those aren't surefire ways to filter out emissive objects while leaving lit objects contributing, but they might be versatile enough for some cases. I'd try them myself, but I'm not sure where this can be performed in HBAO shaders (HBAO_frag only accesses RT0 and RT2, not RT3).

 

Nice ideas! you should look where _MainTex is read.

FYI I already have a fix planned for release to fix a problem the PBR lit AO had (HBAO_Integrated component set with 'BeforeReflections' as integration stage).
The emissive where not cancelling AO and I worked out in a similar way as you suggested:

Code (csharp):

1. o.gbuffer3.rgb *= ao;

became:

Code (csharp):

1. o.gbuffer3.rgb *= lerp(ao, float3(1.0, 1.0, 1.0), dot(o.gbuffer3.rgb, float3(0.299, 0.587, 0.114) * _LuminanceInfluence));

This was not related to color bleeding but it works very well to cancel AO with emissives.

 

Good to know!

That snippet above can be found in HBAO_Deferred on line 24, but editing that area seems to have zero effect on HBAO output. Judging from includes from main HBAO shader, it should be used, so I'm not sure what I'm missing.

By the way, can you explain a bit more what purpose new integrated version is serving? I jumped to 2.5 straight from 1.x yesterday, so I wasn't familiar with it before. At first I thought it was simply a new implementation using command buffers instead of traditional post effect style with render textures; but since old implementation is still shipped, I have to wonder if new "integrated" one should be used only in a certain special case and can't fully replace it. We're developing a DX11-only game utilizing deferred rendering and linear color space - should we move to integrated version?

 

HBAO_Integrated is better in any case, it's a more modern way to integrate ao directly from the rendering pipeline.

EDIT: to answer your last question, yes you should.

HBAO_Integrated component have a new Integration Stage setting which when doing deferred is best set to 'Before Reflections' to get PBR lit AO (as reflections happens before lighting). When in deferred don't hesitate to take advantage too of high quality normals with Per Pixel Normals set to 'GBuffer'.

Finally with integrated component you get more control of how AO or Color Bleeding is injected into GBuffer so you'll be interested to look at HBAO_Deferred.cginc (used for composition of 'Before Reflections' stage) or HBAO_Integrated.cginc (used for composition of other stages). Be careful as HDR versus non HDR can take a different path.

 

A-ha, great, I'll move to it right away! So, I was not seeing any changes to HBAO output after editing HBAO_Deferred because HBAO_Deferred is used by HBAO_Integrated effect, not by legacy HBAO implementation. Got it.

One more thing - is there any downside to changing world space radius limit to, say, 8 meters when max radius in screen space pixels is set to a reasonable value? I mostly use AO as a way to improve dynamically lit voxel geometry of our levels, and since we're making a turn-based game with bird's eye view of locations, 2m limit results in pretty tiny AO bands (especially when voxel scale itself is 3x3x3m). As you can see, HBAO can nicely accentuate the volumes when set to 6-8m radius on a level we observe from a distance:

• https://i.imgur.com/8sHPriI.gifv

I really like world space radius as a setting, it allows you to make a game look consistent no matter the screen resolution (other AO implementations often don't offer such a setting). But I'm guessing it has no direct impact on performance - what actually affects performance is final screen space pixel radius (which world space radius only indirectly affects as one of the factors), right?

P.S.: Even though I'm planning to drop high-radius fast-preset HBAO as our main AO pass and make Unity implementation of MSVO the main AO pass (it's pretty great at shading big chunky volumes we have), your HBAO implementation is unmatched for capturing small detail which MSVO misses, so leaving in an optional fast-preset HBAO pass with low radius feels like a great addition to me.

 

Speaking of the postprocessing stack - are there any performance benefits to be reaped from integrating HBAO with the new (V2) postprocessing stack? There are obvious future proofing benefits (anything using new shader and class structure would be compatible with new scriptable render loops and new rendering modes coming in the future, not just with V2 stack) and some scripting benefits (being able to drive AO settings with postprocessing volumes), but the performance question is most interesting one, I suppose.

 

Yes and notice how debug output switches to BeforeImageEffectOpaque... In the debug output you'll only see raw AO.

Hmmm a bit difficult to explain but with higher radius you'll get higher cache misses and so performances will drop. That said HBAO can be used with higher radius.

You'll be interested by deinterleaved texturing (Detinterleaving setting in General). it separates the kernel and works at lower resolution thus the radius can go as high as 16 without impacting perfs. If you're targetting modern GPU and HD resolution then 4x deinterleaving is a good choice for 1080p to 4K. You'll trade against a few more drawcalls but it have a lot of ther advantages but I do not suggest deinterleaving with resolutions <1080p.

 

Thanks, I was wondering what deinterleaving does, that makes sense.

Can you elaborate a bit more on correct way to edit the combiner for integrated HBAO? I'm not seeing any changes to rendered output no matter what I edit in HBAO_Integrated.cginc or HBAO_Deferred.cginc no matter what integration mode I select. Even doing something visually drastic like gbuffer3.rgb += float3 (10, 0, 0); results in no visible changes anywhere in any mode. What can be causing that?

 

Let me know which integration stage you can want and HDR or not ?

 

BeforeReflections is stage set on HBAO_Integrated effect (since you recommend to use it), and yeah, HDR is enabled on the camera.

 

On the performance side there's really nothing more to expect. An SSAO solution takes most of his time sampling depth buffer, nothing that can be re-used...

 

Then pass 32 is used to combine AO, frag_blend is used fom HBAO_Deferred.cginc.
Color bleeding is combined in pass 35.

Scroll down in HBAO.shader to look at shader passes.

EDIT: take a look around line 325 in HBAO_Integrated.cs you'll see how it is blended.

 

Hmm. For some reason, absolutely nothing I do in CombinerOutput of HBAO_Deferred.cginc has any influence on rendered image. I double checked the camera (it's in HDR mode and deferred path), color space (it's linear), HBAO_Integrated component settings (it's set to BeforeReflections mode and is set to use GBuffer) - but nothing (even commenting out every single operation on gbuffer elements of CombinerOutput as a radical test), seems to affect the resulting image. It's as if this shader code is not actually used in current case, or as if the shader sits in memory unmodified (unlikely).

I'm tempted to start commenting out random passes to see if it's an issue with Unity not recompiling the shader or me not editing the correct shader

 

Be careful, debug outputs won't show anything. Hmmm that's pretty strange make a typo to see if the shader complaints about it

 

A-ha, commenting out very last part of HBAO_Deferred.cginc actually has an effect, AO disappears from composited result. I'm not sure why nothing done in CombinerOutput frag(v2f i) between lines 9 and 31 has any effect, though.

Yep, I'm using normal rendering mode, not debug views.

 

Hey no, sorry, you're in frag, this one is for non HDR...
Look below in frag_blend!

 

Ahh, so the version above was strictly for non-HDR case? I assumed I need to use it because you offered a luminance based fix to gbuffer3 multiplication a few posts before, and frag was the place where I found the line you were referring to.

If so, how should I apply the AO fix you offered previously?

Code (csharp):

1. o.gbuffer3.rgb *= lerp(ao, float3(1.0, 1.0, 1.0), dot(o.gbuffer3.rgb, float3(0.299, 0.587, 0.114) * _LuminanceInfluence));

This snippet seems to belong in the frag function, because you are not performing any tex2D sampling of deferred render targets in vert_blend function, so o.gbuffer3.rgb contains nothing at all.

 

In HDR, there's a special case.

from https://docs.unity3d.com/Manual/RenderTech-DeferredShading.html

Hence, it is handled accordingly both CPU and GPU side.

 

Ah, so that's where emission buffer was disappearing! Thanks, that's actually something that was puzzling to me when I worked with deferred decals, this is a huge help!

So, to apply the luminance-based fix you offered before to frag_blend:

What should I sample in frag_blend instead of _rt3Tex? Is _MainTex the camera render target? (Unless I misunderstood the subject of the fix and there is no need for it in HDR case - then I'm hopping off to explore color bleeding correction through pass 35).

 

Hehe, let me repair my mistake.

frag_blend will become:

Code (csharp):

1.  
2.     CombinerOutput frag_blend(v2f i) {
3.        half4 occ = FetchOcclusion(i.uv2);
4.        half3 ao = lerp(_BaseColor.rgb, half3(1.0, 1.0, 1.0), occ.a);
5.  
6. #if UNITY_SINGLE_PASS_STEREO
7.        float2 uv = UnityStereoTransformScreenSpaceTex(i.uv2);
8.        half3 rt3 = tex2D(_rt3Tex, uv);
9. #else
10.        half3 rt3 = tex2D(_rt3Tex, i.uv2);
11. #endif
12.        CombinerOutput o;
13.        o.gbuffer0 = half4(1, 1, 1, occ.a);
14.        o.gbuffer3 = half4(lerp(ao, float3(1.0, 1.0, 1.0), dot(rt3, float3(0.299, 0.587, 0.114) * _LuminanceInfluence)), 0);
15.        return o;
16.     }
17.  

But as you can see you need to send CameraTarget to rt3, and so the code in HBAO_Integrated.cs near line 325 become:

Code (csharp):

1.  
2. if (_renderTarget.hdr)
3. {
4.     var rt3TexId = new RenderTargetIdentifier(ShaderProperties.rt3Tex);
5.     _hbaoCommandBuffer.GetTemporaryRT(ShaderProperties.rt3Tex, _renderTarget.fullWidth, _renderTarget.fullHeight, 0, FilterMode.Point, RenderTextureFormat.ARGBHalf);
6.     _hbaoCommandBuffer.Blit(BuiltinRenderTextureType.CameraTarget, rt3TexId);
7.  
8.     _hbaoCommandBuffer.SetRenderTarget(mrt, BuiltinRenderTextureType.CameraTarget);
9.     _hbaoCommandBuffer.DrawMesh(quadMesh, Matrix4x4.identity, _hbaoMaterial, 0, Pass.Combine_Deffered_Multiplicative); // AO blend pass (multiplicative)
10.     if (colorBleedingSettings.enabled)
11.         _hbaoCommandBuffer.Blit(BuiltinRenderTextureType.CameraTarget, BuiltinRenderTextureType.CameraTarget, _hbaoMaterial, Pass.Combine_ColorBleeding); // Color Bleeding blend pass (additive)
12.  
13.     _hbaoCommandBuffer.ReleaseTemporaryRT(ShaderProperties.rt3Tex);
14. }
15.  

 

Forgot to answer this.

Once you have made the changes I've shown above, on GPU side _rt3Tex is available and contains CameraTarget, and guess what's CameraTarget in HDR ? yes, RT3.

Now in pass 32 as well as pass 35 you have access to RT3:

Code (csharp):

1. half3 rt3 = tex2D(_rt3Tex, i.uv2);