HDR The Bungie Way featuring Bloom, Flares and Lighting

Hey guys, back again with some more basic stuff. I need help getting a bloom curve, from the presentation HDR The Bungie Way by Chris Tchou. I dont like the way my bloom looks in its current state. I may look at some different techniques for bloom because currently there aren't many that looks good in my opinion besides the few listed below. I've already implemented Kawase's Bloom Filter, now i'm trying out this one. But the key component I don't understand how to achieve is the bloom curve. I've uploaded just the slides, however some of the slides are missing pictures which hinder slightly my ability to understand. But a few of the pictures can also be found below, along with the current code I have. So what I ask is that someone help me with implementing a bloom curve to the HDR Lighting from [Bungie06], that will allow me to have bloom that looks gorgeous . (Bloom Curve starts at slide 64).

I've extended this post to ask for help with other post processing matters on how to achieve better bloom/glare/ghosting effects with any tips or slides that you guys may have found. I wish to improve the quality of my lighting in games as well by making the lighting more "physically accurate".

Bloom Curve (Taken from Bungie's slides, but missing in the powerpoint, only found on the web):

Bloom Curve:

No Bloom Curve:


Examples of bloom/flares that I like and want to acheive something like:



I know I need some ghost samples for my bloom to also make it look "prettier":


And I need streaking, I already have streaking but Its not as good as these other implementations like Yebis 2 or iCEnhancer the GTAIV ENB Series mod. Here is my implementation of streaking:
https://www.youtube.com/watch?v=5qH87cAp2QU

Lastly I want to improve my lighting , like Cody has done here, with Physically accurate sunlight and ambient colors:


Current Code:
Shader:

Code (csharp):

1. Shader "Custom/HDRLighting2"
2. {
3.     Properties
4.     {
5.         _MainTex ("Base (RGB)", 2D) = "white" {}
6.         _BloomTex ("Base (RGB)", 2D) = "white" {}
7.     }
8.        
9.         CGINCLUDE
10.         #include "UnityCG.cginc"
11.  
12.         uniform sampler2D _MainTex;
13.         uniform sampler2D _OrignMaterial;
14.         uniform sampler2D _BloomTex;
15.         uniform sampler2D _BlurTex;
16.         uniform sampler2D _Curve;
17.         uniform float4 _BloomColor;
18.         uniform float _BlurSize;
19.         uniform float _RangeScale;
20.         uniform float _BloomIntensity;
21.         uniform float _BloomThreshold;
22.        
23.         float4 offsets;
24.        
25.         struct v2f
26.         {
27.             float4 pos : POSITION;
28.             float2 uv : TEXCOORD0;
29.    
30.             float4 uv01 : TEXCOORD1;
31.             float4 uv23 : TEXCOORD2;
32.             float4 uv45 : TEXCOORD3;
33.         };
34.        
35.         v2f vert (appdata_img v)
36.         {
37.             v2f o;
38.             o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
39.    
40.             o.uv.xy = v.texcoord.xy;
41.    
42.             o.uv01 =  v.texcoord.xyxy + offsets.xyxy * float4(1,1, -1,-1);
43.             o.uv23 =  v.texcoord.xyxy + offsets.xyxy * float4(1,1, -1,-1) * 2.0;
44.             o.uv45 =  v.texcoord.xyxy + offsets.xyxy * float4(1,1, -1,-1) * 3.0;
45.    
46.             return o;  
47.     }
48.        
49.         float BloomCurve(float fBright)
50.         {
51.             fBright = max(tex2D(_Curve, float2(_RangeScale, 0.5)).x, 2.00f);
52.             return fBright;
53.         }
54.        
55.         float4 Tonemap(v2f_img i)  : COLOR
56.         {
57.             float4 vColor = tex2D(_MainTex, i.uv);
58.             vColor.rgb *= 5.199f;
59.             vColor.rgb = vColor.rgb / (1.0f + vColor.rgb);
60.  
61.             vColor.rgb = pow(vColor.rgb, float3(1.0f / 0.550f, 1.0f / 0.552f, 1.0f / 0.549f));
62.             vColor.rgb *= 1.10f;
63.            
64.             return float4(vColor.rgb, 1.0f);
65.         }
66.        
67.         float4 Threshold(v2f_img i) : COLOR
68.         {
69.             float4 vSample = tex2D(_MainTex, i.uv);
70.            
71.             return saturate((vSample - _BloomThreshold) / (1 - _BloomThreshold));
72.         }
73.        
74.         float4 Downsample(v2f_img i) : COLOR
75.         {
76.             float2 InputSize = float2( 1.0f / 1280.0f, 1.0f / 720.0f);
77.        
78.             float4 vSample = tex2D(_MainTex, i.uv);
79.            
80.             vSample += tex2D(_MainTex, i.uv+InputSize * float2(1.0f, 1.0f));
81.             vSample += tex2D(_MainTex, i.uv+InputSize * float2(-1.0f, 1.0f));
82.             vSample += tex2D(_MainTex, i.uv+InputSize * float2(1.0f, -1.0f));
83.             vSample += tex2D(_MainTex, i.uv+InputSize * float2(-1.0f, -1.0f));
84.            
85.             vSample /= 4.0f;
86.            
87.             return vSample;
88.        
89.         }
90.        
91.         float4 Bloom(v2f_img i) : COLOR
92.         {
93.             float4 vSample = tex2D(_MainTex, i.uv);
94.             float fIntensity = max(dot(vSample.rgb, float3(0.3f, 0.3f, 0.3f)), 0.0001f);
95.            
96.             //_BloomIntensity = BloomCurve(fIntensity);
97.             float3 fBloomColor = vSample.rgb * float3(_BloomIntensity, _BloomIntensity, _BloomIntensity) / float3(fIntensity, fIntensity, fIntensity);
98.            
99.             return float4(fBloomColor, 1.0f);
100.            
101.         }
102.        
103.         float4 AccumBlur(v2f_img i)  : COLOR
104.         {
105.             float4 vBloom = tex2D(_BloomTex, i.uv);
106.             float4 vBlur = tex2D(_BlurTex, i.uv);
107.        
108.             float4 vFinal = vBloom + vBlur;
109.            
110.             return vFinal;
111.         }
112.        
113.         float4 Blur(v2f i)  : COLOR
114.         {
115.             float4 color = float4 (0,0,0,0);
116.  
117.             color += 0.40 * tex2D (_MainTex, i.uv);
118.             color += 0.15 * tex2D (_MainTex, i.uv01.xy);
119.             color += 0.15 * tex2D (_MainTex, i.uv01.zw);
120.             color += 0.10 * tex2D (_MainTex, i.uv23.xy);
121.             color += 0.10 * tex2D (_MainTex, i.uv23.zw);
122.             color += 0.05 * tex2D (_MainTex, i.uv45.xy);
123.             color += 0.05 * tex2D (_MainTex, i.uv45.zw);   
124.            
125.             return color;
126.         }
127.        
128.         float4 Composite(v2f_img i)  : COLOR
129.         {
130.             float4 vSample = tex2D(_OrignMaterial, i.uv);
131.             float4 vBloom = tex2D(_BloomTex, i.uv);
132.             vBloom *= _BloomColor;
133.        
134.        
135.             float4 vFinal = vBloom + vSample;
136.        
137.             return vFinal;
138.         }
139.  
140.         ENDCG
141.  
142. Subshader {
143.  
144.     ZTest Off
145.     Cull Off
146.     ZWrite Off
147.     Fog { Mode off }
148.    
149. //Pass 0: Threshold
150.  Pass
151.  {
152.  Name "Threshold"
153.  
154.       CGPROGRAM
155.       #pragma fragmentoption ARB_precision_hint_fastest
156.       #pragma vertex vert_img
157.       #pragma fragment Threshold
158.       ENDCG
159.   }
160.  
161. //Pass 1: Downsample
162.  Pass
163.  {
164.  Name "Downsample"
165.  
166.       CGPROGRAM
167.       #pragma fragmentoption ARB_precision_hint_fastest
168.       #pragma vertex vert_img
169.       #pragma fragment Downsample
170.       ENDCG
171.   }
172.  
173.   //Pass 2: Bloom
174.  Pass
175.  {
176.  Name "Bloom"
177.  
178.       CGPROGRAM
179.       #pragma vertex vert_img
180.       #pragma fragment Bloom
181.       ENDCG
182.   }
183.     //Pass 3: AccumBlur
184.  Pass
185.  {
186.  Name "AccumBlur"
187.  
188.       CGPROGRAM
189.       #pragma vertex vert_img
190.       #pragma fragment AccumBlur
191.       ENDCG
192.   }
193.  
194.     //Pass 4: Blur
195.  Pass
196.  {
197.  Name "Blur"
198.  
199.       CGPROGRAM
200.       #pragma vertex vert
201.       #pragma fragment Blur
202.        #pragma fragmentoption ARB_precision_hint_fastest
203.       ENDCG
204.   }
205.    
206.    //Pass 5: Tonemap
207.   Pass
208.  {
209.  Name "Tonemap"
210.  
211.       CGPROGRAM
212.       #pragma vertex vert_img
213.       #pragma fragment Tonemap
214.       ENDCG
215.   }
216.  
217.    //Pass 6: Composite
218.  Pass
219.  {
220.  Name "Composite"
221.  
222.       CGPROGRAM
223.       #pragma vertex vert_img
224.       #pragma fragment Composite
225.       ENDCG
226.   }
227. }
228.  
229. Fallback off
230.    
231. }
232.  

C# (CPU code)

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. [ExecuteInEditMode]
5. public class HDRLighting2 : MonoBehaviour {
6.  
7.     #region Variables
8.     public Shader HDRLightingShader;
9.     public float BloomAmount = 1.0f;
10.     public float BloomThreshold = 1.0f;
11.     public Color BloomColor;
12.     public float Blur = 1.0f;
13.     private Material HDRLightingMaterial;
14.     private RenderTexture BrightPass;
15.     private RenderTexture DownSample_0;
16.     private RenderTexture DownSample_1;
17.     private RenderTexture DownSample_2;
18.     private RenderTexture DownSample_3;
19.     private RenderTexture HDRBloom;
20.     private RenderTexture HDRBlur;
21.     private RenderTexture LDRTonemap;
22.     private RenderTexture UpSample_0;
23.     private RenderTexture UpSample_1;
24.     private RenderTexture UpSample_2;
25.     private RenderTexture UpSample_3;
26.     private RenderTexture Compose;
27.     private RenderTextureFormat HDR = RenderTextureFormat.ARGBHalf;
28.     private RenderTextureFormat LDR = RenderTextureFormat.ARGB32;
29.     #endregion
30.    
31.     #region Properties
32.     Material material
33.     {
34.         get
35.         {
36.             if(HDRLightingMaterial == null)
37.             {
38.                 HDRLightingMaterial = new Material(HDRLightingShader);
39.                 HDRLightingMaterial.hideFlags = HideFlags.HideAndDontSave; 
40.             }
41.             return HDRLightingMaterial;
42.         }
43.     }
44.     #endregion
45.     // Use this for initialization
46.     void Start ()
47.     {
48.         if(!SystemInfo.supportsImageEffects)
49.         {
50.             enabled = false;
51.             return;
52.         }
53.        
54.         //HDRLightingShader = Shader.Find("HDRLighting2");
55.        
56.     }
57.    
58.     void SetConstants()
59.     {
60.         material.SetFloat("_BloomIntensity", BloomAmount);
61.         material.SetFloat("_BloomThreshold", BloomThreshold);
62.         material.SetColor("_BloomColor", BloomColor);
63.     }
64.    
65.     void BrightPassFilter(RenderTexture source)
66.     {
67.         BrightPass = RenderTexture.GetTemporary(source.width, source.height, 0, HDR);
68.        
69.         Graphics.Blit(source, BrightPass, material, 0);
70.     }
71.    
72.     void Downsample(RenderTexture source)
73.     {
74.             DownSample_0 = RenderTexture.GetTemporary(source.width / 2, source.height / 2, 0, HDR);
75.             DownSample_1 = RenderTexture.GetTemporary(source.width / 4, source.height / 4, 0, HDR);
76.             DownSample_2 = RenderTexture.GetTemporary(source.width / 8, source.height / 8, 0, HDR);
77.             DownSample_3 = RenderTexture.GetTemporary(source.width / 16, source.height / 16, 0, HDR);
78.        
79.             Graphics.Blit(BrightPass, DownSample_0, material, 1);
80.             RenderTexture.ReleaseTemporary(BrightPass);
81.             Graphics.Blit(DownSample_0, DownSample_1, material, 1);
82.             RenderTexture.ReleaseTemporary(DownSample_0);
83.             Graphics.Blit(DownSample_1, DownSample_2, material, 1);
84.             RenderTexture.ReleaseTemporary(DownSample_1);
85.             Graphics.Blit(DownSample_2, DownSample_3, material, 1);
86.             RenderTexture.ReleaseTemporary(DownSample_2);
87.        
88.     }
89.    
90.     void Bloom(RenderTexture source)
91.     {
92.         HDRBloom = RenderTexture.GetTemporary(source.width / 16, source.height / 16, 0, HDR);
93.         Graphics.Blit(DownSample_3, HDRBloom, material, 2);
94.         RenderTexture.ReleaseTemporary(DownSample_3);
95.     }
96.    
97.     void AccumulateBlur(RenderTexture source)
98.     {
99.         HDRBlur = RenderTexture.GetTemporary(source.width / 16, source.height / 16, 0, HDR);
100.        
101.         float WidthOverHeight = (source.width) / (source.height);
102.         float OneOverBase = 1.0f / 512.0f;
103.        
104.         //Seperable Blur (Unitys approach)
105.         for(int i = 0; i < 2; i++)
106.         {
107.             material.SetVector ("offsets", new Vector4 (0.0f, Blur * OneOverBase, 0.0f, 0.0f));
108.             Graphics.Blit (HDRBloom, HDRBlur, material, 4);
109.             material.SetVector ("offsets", new Vector4 (Blur * OneOverBase / WidthOverHeight, 0.0f, 0.0f, 0.0f));
110.             Graphics.Blit (HDRBlur, HDRBloom, material, 4);
111.         }
112.        
113.         UpSample_0 = RenderTexture.GetTemporary(source.width / 16, source.height / 16, 0, HDR);
114.         UpSample_1 = RenderTexture.GetTemporary(source.width / 8, source.height / 8, 0, HDR);
115.         UpSample_2 = RenderTexture.GetTemporary(source.width / 4, source.height / 4, 0, HDR);
116.         UpSample_3 = RenderTexture.GetTemporary(source.width / 2, source.height / 2, 0, HDR);
117.        
118.         Graphics.Blit(HDRBloom, UpSample_0, material, 3);
119.         RenderTexture.ReleaseTemporary(HDRBloom);
120.        
121.         material.SetTexture("_BlurTex", HDRBlur);
122.        
123.         for(int i = 0; i < 2; i++)
124.         {
125.             material.SetVector ("offsets", new Vector4 (0.0f, Blur * OneOverBase, 0.0f, 0.0f));
126.             Graphics.Blit (UpSample_0, HDRBlur, material, 4);
127.             material.SetVector ("offsets", new Vector4 (Blur * OneOverBase / WidthOverHeight, 0.0f, 0.0f, 0.0f));
128.             Graphics.Blit (HDRBlur, UpSample_0, material, 4);
129.         }
130.        
131.         Graphics.Blit(UpSample_0, UpSample_1, material, 3);
132.         RenderTexture.ReleaseTemporary(UpSample_0);
133.        
134.         for(int i = 0; i < 2; i++)
135.         {
136.             material.SetVector ("offsets", new Vector4 (0.0f, Blur * OneOverBase, 0.0f, 0.0f));
137.             Graphics.Blit (UpSample_1, HDRBlur, material, 4);
138.             material.SetVector ("offsets", new Vector4 (Blur * OneOverBase / WidthOverHeight, 0.0f, 0.0f, 0.0f));
139.             Graphics.Blit (HDRBlur, UpSample_1, material, 4);
140.         }
141.        
142.         Graphics.Blit(UpSample_1, UpSample_2, material, 3);
143.         RenderTexture.ReleaseTemporary(UpSample_1);
144.        
145.         for(int i = 0; i < 2; i++)
146.         {
147.             material.SetVector ("offsets", new Vector4 (0.0f, Blur * OneOverBase, 0.0f, 0.0f));
148.             Graphics.Blit (UpSample_2, HDRBlur, material, 4);
149.             material.SetVector ("offsets", new Vector4 (Blur * OneOverBase / WidthOverHeight, 0.0f, 0.0f, 0.0f));
150.             Graphics.Blit (HDRBlur, UpSample_2, material, 4);
151.         }
152.        
153.         RenderTexture.ReleaseTemporary(HDRBlur);
154.        
155.         Graphics.Blit(UpSample_2, UpSample_3, material, 3);
156.         RenderTexture.ReleaseTemporary(UpSample_2);
157.        
158.        
159.     }
160.     void OnRenderImage (RenderTexture sourceTexture, RenderTexture destTexture)
161.     {
162.         if(HDRLightingShader != null)
163.         {
164.             SetConstants();
165.            
166.             BrightPassFilter(sourceTexture);   
167.             Downsample(sourceTexture);
168.             Bloom(sourceTexture);
169.             AccumulateBlur(sourceTexture);
170.            
171.             Compose = RenderTexture.GetTemporary(sourceTexture.width, sourceTexture.height, 0, LDR);
172.            
173.             material.SetTexture("_BloomTex", UpSample_3);
174.             material.SetTexture("_OrignMaterial", sourceTexture);
175.             Graphics.Blit(UpSample_3, Compose, material, 6);
176.             RenderTexture.ReleaseTemporary(UpSample_3);
177.             Graphics.Blit(Compose, destTexture, material, 5);
178.            
179.             RenderTexture.ReleaseTemporary(Compose);
180.            
181.         }      
182.     }
183.    
184.     // Update is called once per frame
185.     void Update ()
186.     {
187.         BloomAmount = Mathf.Clamp(BloomAmount, 0.0f, 5.0f);
188.         BloomThreshold = Mathf.Clamp(BloomThreshold, 0.0f, 1.0f);
189.     }
190.    
191.     void OnDisable ()
192.     {
193.         if(HDRLightingMaterial)
194.         {
195.             DestroyImmediate(HDRLightingMaterial); 
196.         }
197.        
198.     }
199.    
200.    
201. }

[Bungie06] https://www.dropbox.com/s/t2m4aktvj591q3y/HDR The Bungie Way.ppt

Thanks for your help guys, I will be back soon with help on auto-exposure without reinhard's tonemap operator, just a pure custom auto-exposure.

 

Also forgot to mention, here is what I currently have for the ambient lighting but i'm sure lots of things can be improved:

 

For the bloom curve, the idea is to use the overall intensity of the pixel to determine how much to apply the effect instead of using each color channel individually, which would result in any differences in color becoming exaggerated.

Bungie is using dot(color, float3(0.3,0.3,0.3)) to get the intensity, but you may want to try dot(color, float3(0.212671, 0.71516, 0.072169)) instead. That will get you the luminance, which is essentially the intensity weighted by the sensitivity of our eyes to particular wavelengths of light. In theory it should give a more realistic result, but to be honest I don't know enough about photometry to be sure that it's correct to use in this situation. It might just make everything look worse. It's also incorrect for use in low-light scenes, but I don't have the matrix for those to hand at the moment.

The function for the bloom curve graph appears to be something along the lines of x*0.4+max(0,x-1.25)*0.6, but it's hard to know for sure (label your axes, Bungie!) You could use pretty much anything here, it just takes the image intensity and outputs the bloom intensity. x*x/3.2 will give you something similar but with a smoother distribution, or max(0,x-1)^2 will only cause intensities over 1.0 to bloom. It all depends on what sort of look you're going for.

After that, the bloom_color = color * bloom_intensity / intensity line divides out the old intensity and multiplies in the new one, then you can do the usual resampling, blurring, and additive blending to get the final image.

 

Your post is extremely helpful . Each of the samples worked flawlessly. I was going to use those "common" luminance coefficients, but I was afraid it would affect the color of the bloom somehow, the slides kept mentioning the key is to reserve the color, so i tried not to veer off too far from their code.

I guess another issue is preserving color, I'm seeing the horrible reddish bloom from orange glowing objects, and the darker blues in the bloom than the objects actually are. I assume this has something to deal with the thresholding, when I threshold the image I get colors that aren't the same as the original image. I read some where that you can use a tone map operator to threshold the image so it preserves color and only extracts the bright parts. the D3D Book Im pretty sure talks about this. But It has reinhards tonemap written all over it.

I did notice something though, I see some bloom techniques where they don't just use a thresholding main image but using patterns and code that makes shapes and they use those to draw the bloom I think . I'm not entirely sure how to get that effect working but it seems like an interesting approach.

But I'll figure something out for that soon. I plan on adding blue shifting after I figure an auto exposure without reinhard's tonemap! Then the rest of the glare/flare, streaking etc.

Any more tips or tricks guys? Feedback is welcome!

 

WGermany

Please - Finish This!!!!!!!!!!! And I'm your first customer. I tried and looks beautify, but needs auto exposure. (As an opportunity to introduce support dirty texture - is add realistics).

 

I've made a bunch more changes to this, I've added dirty texture support a long time ago, fixed some bugs and I'm adding auto exposure now. This wasn't a release though, it was a thread asking for help.