True Volumetric Lights (Now Open Source)

I decided to make this little technology open source. Contributions are welcome. Or just fork it and completely rewrite it. Also be warned, it is a hobby project turned into open source. You'll get what you pay for as they say.

Be sure to let me know if you use it in your project.
Here it is.

Original post follows
Hey,

I have experimented with command buffer in my spare time recently and I have decided to expand unity lights to make them volumetric. And since people really seem to like the results I decided to show you guys as well. YouTube and Vimeo links. Please ignore poor image quality. Compression wasn't very kind to it. Vimeo link is probably slightly better.

Enjoy

 

Like it very much!

Performance?
Webplayer?
AssetStore?

 

Thanks!
Performance: Not sure. I didn't profile it yet. But it is more or less the same technique as the one used in Killzone Shadow Fall, so I would say it is ok for Next Gen consoles, mid range PCs and up.
Webplayer: Theoretically yes, but I never used webplayer. There may be some limitation I'm not aware of.
AssetStore: Very unlikely. I couldn't be bothered to maintain it. I already have a day job and as a game developer I don't have too much free time. I did it just for fun. However, I will consider making it open source.

 

Very nice! count me as interested Sorry for shooting you a PM before, I didn't see this thread. I'd love to see it open source, probably could throw a donate button up or something?

In any case, great work, how is it done?

 

No problem with that PM. I wrote a quick explanation here. I'll come here with more details when I have time. It requires deferred renderer, it reconstructs position from the gbuffer. That means it doesn't work with transparent geometry (cutout is ok).

 

So, afters years of silence on the subject, TWO solutions pop up at the same time. Typical

This is looking great! Would love it to see it open-source too - and so would my game:

 

Sorry, I'm kind of new around here Also, is that a rabbit in that car? It looks amazing.

Edit: Ok, I just noticed your avatar. Coincidence? I don't think so

 

This is intereting, i've been playing with raymarching for volumetric light too but only manage to make directional to work with it via command buffer and haven't continue it again. Would love to see your code if you don't mind.
+1 to make it open source put it to github or something would be cool

 

Thank you Yeah, it's an evolved rodent from the far far future (driving an under-evolved car from the past )

 

Good news. I decided to make it open source. It will happen within few weeks. Hopefully by the end of the month. I'll come here with more info about it, its requirements and limitations before the release. Stay tuned...

 

After seeing the volumetric fog on gdc i was hoping to play with something similar, thanks
If you need any testers let me know!

 

That's great news! Looking forward to read the requirements and limitations. Should you need testing this in a large open-world setting, I'm ready

 

Please bear with me. One of the games I'm working on is close to release. My current plan is to give you more info + my volumetric demo for testing at the end of this week. I'll hopefully release source code some time next week. I don't plan to do any real testing. It's open source, it doesn't have to work for everyone out of the box. We can iron it out as we go..

 

Extremely cool work Michal, cant wait to play around with the source!

I have a question, in the other thread you mention using LightEvent.AfterShadowMap to fetch the shadow map and to render the light volume.
Im assuming you have to change the RenderTarget before using DrawMesh to render the light volume, which RenderTarget do you set it to?

 

It would be a new temporary render texture created before the lights start rendering. At least that's the way I do it.

 

@Lexie is right. I use temporary render target. Let me explain in more detail.

First of all it was a hobby project. You should see it more as a starting point then a production ready plugin. Could be good enough for some use cases though.

Technique Overview

• Create render target for volumetric light (volume light buffer)
• Use CameraEvent.BeforeLighting to clear volume light buffer
• For every volumetric light render light's volume geometry (sphere, cone, full-screen quad)
• Use LightEvent.AfterShadowMap for shadow casting lights​
• Use CameraEvent.BeforeLighting for non-shadow casting lights​
• Perform raymarching in light's volume​
• Dithering is used to offset ray origin for neighbouring pixels​
• Use CameraEvent.AfterLighting to perform depth aware gaussian blur on volume light buffer
• Add volume light buffer to Unity's built-in light buffer

Half resolution rendering
Common optimization is to render the volumetric effect in lower resolution. Usually half or quarter resolution. The only real difference is that instead of simply adding volume light buffer to Unity's light buffer in the last step, bilateral upscale is used.
I can't get it to work properly for some reason. I believe it is a bug in Unity. I'll try to figure it out before release but no promises.
It is worth noting that using half resolution + bilateral upscale would affect image quality. Especially in motion. Some sort of temporal reprojection is needed to reduce artifacts but I'm not going to add that.

Limitations

• DirectX 11 only. OpenGL should be easy to do. It should differ only in projection matrices (different clip space, left/right handed system, flipped uv)
• HDR only. Again, should be easy to add LDR support.
• Deferred renderer only.
• Doesn't render transparent geometry correctly (cutout is ok). There are ways to handle transparent objects but they are either difficult or impossible to do with Unity's public API.

Performance
As I said before, it was a hobby project. I didn't care about performance much. The demo runs 25-30 fps on my notebook with low end GeForce 750 (volumetric light computed in full resolution, 1080p). And it runs several hundred fps on my desktop. So I felt no need to try to optimize it more. There is definitely room for improvement. With that being said, there are currently several ways how you can trade image quality for performance.

• Perform raymarching in lower resolution. You can choose half resolution (quarter pixel count) for big performance savings.
• Every light source has several parameters that affect performance
• Number of raymarching steps​
• Shadows​
• Cookie​
• Volumetric Noise (currently 3d texture)​

 

Update. Just figured out why half resolution rendering didn't work. For some reason Unity doesn't like when render target and depth buffer have different resolutions. Even though it is perfectly legal in DirectX and it runs correctly in standalone player. Only Unity editor is having problems. Another workaround then

That brings me to interesting question. I simulate Z Test in pixel shader when I render in lower resolution. That is not exactly ideal. Did anyone find a way how to downsize existing depth buffer and then use as depth buffer for further rendering? It is a simple thing to do in DirectX at least. But I didn't find how to do it in Unity. @Lexie, how do you handle ZTest in lower resolution?

 

Hi in this project you can see how to downscale depth (im not the author). In that project the shader that does the depth downscaling is this

Code (CSharp):

1. Shader "Custom/DownscaleDepth" {
2.  
3.     CGINCLUDE
4.  
5. #include "UnityCG.cginc"
6.  
7.     struct v2f
8.     {
9.         float4 pos : SV_POSITION;
10.         float2 uv : TEXCOORD0;
11.     };
12.  
13.     sampler2D _CameraDepthTexture;
14.     float4 _CameraDepthTexture_TexelSize; // (1.0/width, 1.0/height, width, height)
15.  
16.     v2f vert(appdata_img v)
17.     {
18.         v2f o = (v2f)0;
19.         o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
20.         o.uv = v.texcoord;
21.  
22.         return o;
23.     }
24.  
25.     float frag(v2f input) : SV_Target
26.     {
27.         float2 texelSize = 0.5 * _CameraDepthTexture_TexelSize.xy;
28.         float2 taps[4] = { float2(input.uv + float2(-1,-1)*texelSize),
29.             float2(input.uv + float2(-1,1)*texelSize),
30.             float2(input.uv + float2(1,-1)*texelSize),
31.             float2(input.uv + float2(1,1)*texelSize) };
32.  
33.         float depth1 = tex2D(_CameraDepthTexture, taps[0]);
34.         float depth2 = tex2D(_CameraDepthTexture, taps[1]);
35.         float depth3 = tex2D(_CameraDepthTexture, taps[2]);
36.         float depth4 = tex2D(_CameraDepthTexture, taps[3]);
37.  
38.         float result = min(depth1, min(depth2, min(depth3, depth4)));
39.  
40.         return result;
41.     }
42.  
43.         ENDCG
44.         SubShader
45.     {
46.         Pass
47.         {
48.             ZTest Always Cull Off ZWrite Off
49.  
50.             CGPROGRAM
51. #pragma vertex vert
52. #pragma fragment frag
53.             ENDCG
54.         }
55.     }
56.     Fallback off
57. }

You just blit a downsampled rendertexture to the downscale shader's material

Code (CSharp):

1.         //----DEPTH
2.         RenderTextureFormat formatRF32 = RenderTextureFormat.RFloat;
3.         int lowresDepthWidth = source.width / 4;
4.         int lowresDepthHeight = source.height / 4;
5.  
6.         RenderTexture lowresDepthRT = RenderTexture.GetTemporary(lowresDepthWidth, lowresDepthHeight, 0, formatRF32);
7.  
8.         //downscale depth buffer to quarter resolution
9.         Graphics.Blit(source, lowresDepthRT, DownscaleDepthMaterial);

Edit: Then you pass the downsampled depth to the raymarching shader
e.g.

Code (CSharp):

1. FogMarchedMaterial.SetTexture("LowResDepth", lowresDepthRT);

and in the raymarching shader:

Code (CSharp):

1. // read low res depth and reconstruct world position
2.         float depth = SAMPLE_DEPTH_TEXTURE(LowResDepth, i.uv);
3.    
4.  
5.         //linearise depth      
6.         float lindepth = Linear01Depth(depth);
7.  
8.         //get view and then world positions      
9.         float4 viewPos = float4(i.cameraRay.xyz * lindepth,1);
10.         float3 worldPos = mul(InverseViewMatrix, viewPos).xyz;
11.  
12.         //get the ray direction in world space, raymarching is towards the camera
13.         float3 rayDir = normalize(_WorldSpaceCameraPos.xyz - worldPos);
14.         float rayDistance = length(_WorldSpaceCameraPos.xyz - worldPos);
15.  
16. .......
17.  

Just copied most relevant stuff in case you were bored to check the whole project

 

Thanks, but this is exactly what I do. The problem is that the raymarching is executed even for pixel that are behind existing geometry. Imagine point light behind a wall. It shouldn't be visible at all. I have to discard those pixels in pixel shader based on their depth. Better approach would be to downsample native zbuffer and use it as native zbuffer for further rendering. This way pixels behind existing geometry would be discarded by ZTest before pixel shader and ray marching. I just don't know how to downsample native z buffer in Unity...

 

Awesome, thank you for the detailed explanation of your approach!

 

Here is the demo as promised. You can test it for yourself. Half resolution rendering still has its issues and there is room for improvement. The motion shimmering will never really disappear though. Not without temporal reprojection.
The scene is taken from Unity's Blacksmith demo. There are two dynamic lights and dynamic GI. Apart from the volumetric lights, there is only custom tone mapping + eye adaptation + bloom. No AA.

I plan to release the source code in the matter of days. Enjoy

 

On an ati 5870 I got very good frame rates at half resolution.
Around 60 fps with vol. lights on and 66 fps without.

Well done!

 

From the testing that I've done on my volumetric rendering, I've found that you have to render at quarter resolution for it to be viable for mid to high end hardware if you want to render more then 1-4 lights. It also means you have to have a better down sampling, blur and up sampling shaders to hide all the awful artifacts that quarter/half resolution gives you.

The downside of having more complicated sampling methods is that its slower at rendering a few lights, but the upside is that you can render a lot more and still have acceptable results to use in a game.

 

Yeah, shadow casting lights are pretty expensive. Lights without shadows on the other hand require only 2 or 3 ray-marching samples per pixel and are relatively cheap. I guess it depends on what kind of game you're making.
Do you think quarter resolution would work in this demo where the light shines through small cracks in the wall? I'll have to try someday.

I'd be curious to know how you make quarter resolution look good. How do you make it stable in motion? Would you share with us (or PM me)? I use min or max depth buffer downsampling, depth-aware gaussian blur and bilateral upsampling (nearest depth for edges, bilinear for non-edges or lerp between the two)

 

@Michal_ some context about the system of @Lexie http://forum.unity3d.com/threads/dynamic-volumetric-lighting-replacement.379041/

 

Only just discovered this thread, which is surprising considering I have a huge thing for volumetric fog. That you plan to share your source code is just great, thanks so much for going down that path.

Off to test the demo now. Cheers.

 

Demo is lovely. When you release the code I plan to do some custom stuff by (mis)using the 3d noise texture - all will be revealed

 

Here is a demo of the same building using my volumetric lighting. Default is quarter resolution, you can change the settings with the buttons in the top corner. Hold right mouse button to enable fly cam.

I've disabled all post processing and also the phase function on the light as well as 3D noise texture, Just so there isn't any extra stuff hiding the artifacts. Chances are mine is slower at rendering one single light but the results of my half resolution and quarter resolution are a lot nicer then the simpler method of low res rendering. The goal of the system is to offset the extra cost of the more advanced sampling with the speed increase of rendering multiple lights at quarter/half resolution.

I'm still thinking about selling this on the asset store, I've been working on making volumetric lighting for many lights feasible for some time now, so I'm not sure if ill open source it or not. Would be nice to get something back for this time spent.

 

I've rewritten how the half resolution rendering works to make it look better. Here's updated version. It is more expensive now but it looks almost identical to the full resolution version. It is surprisingly stable in motion too. Even without temporal filter. This new technique should work reasonably well with quarter resolution as well. But I'll probably release it without it to get it out of the door as fast as possible.
And I'll make a list of things that I know could be improved.

 

No problem. I understand. Hey, I'll buy it just to take look!

The demo looks impressive. I like how the image is clean even with very low resolution. No bleeding, no halos. Great work. I don't like how soft it all looks, but I understand that it could be necessary if you need better performance. Anyway, your upsampling is superb.

 

Thanks! Yeah the downside is the softness, half resolution isn't as bad and there is a setting to make it less soft but then artifacts from the low sample rate of the frequency start to show. So you can choose what is more important to your scene.

Here is an example with the setting off that makes it a lot sharper, this is also at quarter resolution.
Although Off looks way better, on hides a lot of artifacts when you have small beams of light.

 

Keep in mind that Unity will be releasing their own volumetric fog sometime after Unite Europe (source http://unity3d.com/pages/adam)... In your case, faster/better lighting/shadows would better justify an Assetstore release, IMHO.

Also, even if Unity releases their own solution, impatient unity users might very well buy your's (a phenomenon that can be observed in the very entertaining Candela SSRR threads, for example).

 

@Michal_ Played around with your demo and I'm getting approx. 0.7ms difference between volumetrics on/off. Around 0.4ms with half res. (4790K + GTX770). I'm really curious now, how the performance will scale with more lights.

Looks gorgeous Can't wait to try it in my project!

 

I'm way too excited about this, need to stay calm and patient

 

Oh, man. I didn't know about it. This is what I ultimately wanted to do. I started with this traditional ray marching approach because it is very straightforward and relatively easy to do, but I really wanted to implement this compute shader + 3d texture based variant. Oh, well.

It is not really about number of lights. You're paying for every lit pixel. So, it is more about number of overlapping lights (overlapping in view space). 10 small non-overlapping lights should give you better performance then one fullscreen light. Speaking only about the volumetric effect here. But yeah, you should be fine with GTX770. Same technique + half resolution rendering was used in several AAA console games in the past.


I have finally found the time to put it on GitHub. I'll update the first post to make easier for newcomers.

 

If you sign up (for free) then you can also get access to nvidia gameworks volumetric fog source code on github, which might be of use to those learning and experimenting with techniques. Their implementation is a standalone example right now, not integrated into Unity or UE4.

 

It suppose this is part of the gameworks program where you have to slam the nvidia plash screen on your product?

 

Probably, I haven't checked the license yet. I mention it here simply on the off chance that techniques used in it are of interest to those developing their own volumetric fog/lighting solutions.

 

Sorry, didn't mean to spoil your day it's been announced at GDC16'

Did some quick testing. I'm getting some black squares popping in and out at random screen positions, except when using quarter res. It's really nice that the fog gets reflected by SSR! It does not to seem to get affected by GlobalFog though (I know, fog affected by fog ) so at least I've made the density decrease, based on the light's distance from the camera, and tried to mimic fog using the directional light's vol-fog, but it's way too bright. Performance-wise: heavy, but I haven't played around enough. First screen :

 

Also checked it out in our project (and also made a script which fades out based on a distance).
Looks great and the performance drop is not that big (considering the effect we get).

I also see some random black squares on the screen sometimes.
I get these a lot if I use spot lights so. But can't get what is causing it (using spot lights without volumetric lighting does not produce these artefacts).

But all in all nothing is completly broken. Maybe we can sort these artefacts out.

One note so:
If I switch to full resolution on the camera, I get the following error (and a lot of stuttering):
Dimensions of color surface does not match dimensions of depth surface
UnityEditor.DockArea:OnGUI()

But not that I want to run it with this setting anyway

EDIT:
Screenshot of the artefacts with spot lights

 

Do you have a link? I can't seem to find it.

I don't know that but from my experience NVidia can be reasoned with. Nvidia logo somewhere in options was sufficient last time we used their tech. And of course they can use your game to promote GameWorks. But I see that as an win-win situation.

That's ok. I think I'll do super realistic atmospheric scattering instead. Wanted to do that for some time now.

I don't know what exactly is the cause of this but I made quick fix to prevent it. Let me know if it works in latest version.

Yeah, global fog affects only geometry that writes into zbuffer. But the fog could theoretically be added during the ray marching. I'll fix the directional light if I have time (no ETA). I think it needs proper light scattering instead of this crude approximation.

 

https://developer.nvidia.com/VolumetricLighting

 

Hi @Michal_ , thanks a lot for the source code. I added your volumetric lights in a scene im working on. Its like a metro tunnel. When the camera is inside the "tunnel" everything seems to work ok, but if i move the camera outside the tunnel i can see that blue circular artifact from one of the spotlights (i highlighted red in the bottom two images). The walls are 1 meter thick (so i dont think it has to do with zdepth issues) and use the standard shader. Have you come across anything like this?

 

In the GDC event that they revealed Adam and did a quick overview of the tech they actually explicitly stated that the light haze used in it wasn't going to be released. It, along with real time area lighting (which also interacts with the fog), was done by an external group so it wasn't something they could release publicly.

Now that doesn't preclude a different light fog from being released or the group that made it from releasing it, but the one in Adam isn't owned by Unity and is unlikely to be released for free by them.

 

Did a quick test with the change in the last revision and it did not fix the problem. It's still visible as it was before. I also tried clamping the values, but had no result with this too.

EDIT:
It's most visible in Quarter and Full Resolution. On Half Resolution it's not visible that often (at least for me), or at least on different camera angles. Don't know how to describe that properly...

EDIT2:
2 more infos for you (want to help as much as I can). The black spots get stronger, the more I increase the sample count. So on 64 it's most visible (but they are also visible on a sample count of 1, but not that much). Also they are visible, even if the density is down to 0.

 

Many thanks! The latest version fixed the artifacts (+ the negative light issues I had with spotlights). It was a different issue than Beinscheibe is having, judging from the pictures. This is how it looked:

If you could hint, how to add global fog, that would be great

 

Thats simply not true.

I assume you've got the wrong end of the stick because in the presentation they said that the volumetric lighting effect and area lights stuff was not part of the Unity Cinematic Effects bundle. Indeed it isn't, and so its not released as part of that particular frequently updated modern effects package. But its still made by people who work for Unity and they've already said they plan to place these effects in our hands after Unite Europe.

Look what the Adam demo page says: ( http://unity3d.com/pages/adam )

 

This might sound over the top but your generous release of the source code caused me to hit a target I've been dreaming of for many years. The moods that can be evoked with well lit fog are what I've long been seeking, and despite any limitations that currently exist in your implementation it's already looking great and has triggered my imagination. So many thanks

 

volumetric fog: https://twitter.com/robertcupisz/status/700446188638773248
tube/area lights: https://twitter.com/robertcupisz/status/712663264069554176

Same here. Years. Since 2011, to be exact

 

@Beinscheibe and @moure, any chance you could create simple repro projects? Also can you test if full/half resolution and shadows on/off have any effect?

I'll think about it. But I'm not sure how one fog should affect another . What are you trying to achieve? Dim volumetric lights by global fog?

Thanks guys, I really appreciate it.