Dynamic Volumetric Lighting replacement

I was really unhappy with the quality and performance of Unity's shadows. I tried a few ways to speed it up but they all relied on unity's lighting in the back end.

So I've completely replaced Unity's dynamic lighting.

EDIT: Test Build (updated 3/9/2016)

Features

• Out performs unity's shadows
• Real time Volumetric lighting
• Better smooth shadows
  
• Way less light bleeding, shadow acne, peter panning
• Smooth Shadow distance fall off (No more pop in shadows)
• Coloured Light cookies
• Multi Camera friendly(reuses shadow maps from other cameras)
• Can handle Higher res shadow maps (not recommended for most GPU's).
• Shadow quality can be changed at run time with out needing to change settings profile (Why unity)
• Currently supports point, spot lights and Directional light
• shadow caster exclusion list (great for torches)

To do

• Area lights
• Soft shadow on directional light.
• Cascade Directional light.
• LOD friendly

Downsides

• Doesn't support transparency (supports alpha cut out and alpha overlay though)
• Only supports Standard shader family
• Doesn't interface with light probs or enlighten
• Edit: Currently Only works in deferred shading path.
• Edit: Uses VRAM to increase performance (size set by user) Still out performs unity if buffer size is 0

Some of these downside could be fixed but I'd need Unity source or their Render API improved.
The game I'm working on doesn't need transparency and is procedurally generated, So most of these downside don't apply to me.

Going to finish up directional lighting this week and hopefully have a build ready for testing soon.
If anyone has any feature requests, I'd love to hear it!

 

Very interesting, any chance you could show a greater variety of settings, such as a forested scene, that sort of thing?

 

Yeah, i need to work on some example scenes. I might just buy a premade one off the store, probably easier then wasting a day.

 

This looks awesome. I'd like to see a demo with much stronger volumetric effects - to see how far it can be pushed. I'd also like to see how it interacts with enlighten, and how it looks when two different colored light volumetrics interact/intersect. I'm really stoked about this, as I'm having endless shadow gap issues. You also mention that directional and area lights are to do - does that mean that this only works with point and spot? Or only point?
Thanks in advance!

 

PINNED

And agreed with the previous posters.
What do you think you'll set as start price?

 

Fixing shadow acne looks amazing. We've had a lot of trouble with that in our procedural game.

The limitations do seem a little hard (only working with standard shaders and no transparency). Depending on price it's definitely something I'd be tempted to look into and research with to see if it fits our game.

 

Interior would be good to have a look at also. An interior with a bit more substance than what the video shows (on the light bleeding part). Thanks!

 

Right now it works with spot and point lights. I should have directional done by the end of the week. Area lights will probably be the last thing I add. Unfortunately my lights don't interact with enlighten, I'd need unity's source to get that working. you might be able to fake it by having a unity light in the same place during the baking process, but I haven't tested it.

Here's a picture of stronger scattering and colour mixing, Ill add more scatter functions soon, right now it scatters in all directions with the same strength

If i sell it, probably around 50-60, but ill have to do some more research first.

Ill include a few scenes in the demo, its hard to show off this effect with video and screen shots. If anyone has any recommendation on scenes, id love to hear it.

 

Want.

 

Nearly got cascade shadow maps generating today. Couldn't find a way to limit the render area in a command buffer so i had to make a custom blit shader to copy 4 textures into one.

Note: color is compressed in the below image

 

@Jesper Mortensen or @KEngelstoft have some idea of how to render lighting to enlighten without unity lights or tip for integrating other lighting systems such as this one?

 

Good one. I have lots of issues with shadow acne, a.k.a banding in my game, also (to less degree) peter panning. Too bad you can't make this work with legacy diffuse shader (which is what I'm using for my game to get oldschool flatshaded look), otherwise it'd be an instant buy from me (don't care about transparency either or interfacing with probes).

 

Those red and blue screenshots look absolutely awesome.

 

You can modify the internal standard shader to get flat shading, we do it for our game.

 

So this becomes useless if it can't mix smoothly with Enlighten GI.
I mean this will work great with only real time lightening games.

 

Amazing work dude, fully credit to you on trying to do something on your own and improve on things where a large company has.

My question is, how is mobile performance?, will it be suitable to use on mobile devices.

Also I feel Unity should help you with what you need, the sources etc.

 

So if I understand correctly, in essence it's similar to my own implementation of lights:
http://forum.unity3d.com/threads/procedural-static-point-lights-with-source.349155/

Sounds like an increase in shadowmap precision, probably R16->R32, doubling memory use. Atleast, for point lights as those don't support dedicated depth formats.

So you have a unique shadowmap for each light? Your memory requirements would then scale with the number of lights, while Unity lights have a constant requirement.

I may be wrong here, but this is due to keeping shadows over multiple frames? As that's the only way I've found to increase the performance of unity lights significantly, and the only other way I found was clipping pixels covered if they're not in range of the light. You've also disabled to 'wiggling' of torches in the viking village demo, further increasing the chances of this being correct.

Internally Unity only has 1 shadowmap of each required quality, and draws over the content just before it renders the light onto the screen. This is why there isn't a update-per-script option like on reflection probes, there is no per-light buffer you can keep.

It would be nice if you can include the downsides of these things in the OP:

1) Your lights use a lot more memory than Unity's, especially if you have more of them
2) Your lights only outperform Unity's if the shadow casters don't move and you therefore don't update the shadowmap
3) This only works in deferred.
4) A related problem to 2); As the code is slower but runs less, if something happens that has to update multiple lights you'll get a frame spike that may be higher than what you'd get every frame with builtin lights.

My suggestions:
1) LOD support. it improved performance for me a LOT; from ~70% of builtin ms to ~20% or so iirc.
2) I don't know how you're rendering into your shadowmap that makes this standard shader only but you could use Camera.RenderToCubemap with a replacement shader to generate the shadowmap, removing the standard shader requirement.

I'd love to be proven wrong, and that this isn't what I described above as that'd mean there are lots of further improvements to be made on top of a good base.

Great job at the volumetric lighting though, looks awesome! Next step is voxelizing the world in range of the point light and conetrace some GI through it

 

Hello Lexie,

I've been following your progress on Hitbox Team - this is looking really promising. Definitely interested ! Especially the volumetrics. Though I would need it to play along with light probes and Enlighten.

So far, I've managed to create a sort of fake lit-fog with particles which look OK at far distances:


...but it doesn't look good when close to camera/light sources, as you can see in this vid:

Your implementation would really bring my project to a whole new level.
Keep up the excellent work!

 

This looks pretty good!

A must have!

 

I haven't done any research into making it work with enlighten yet. My game is procedurally generated so figuring out if i can get it working with light probes and enlighten is at the end of my list. My guess is I'll need some enlighten API opened up.

 

I haven't check out performance on mobile yet, I don't have a phone. I'll borrow one from a friend once its done and test it out.

Maybe i should pull the volumetric stuff out and get it working with unity's lighting as well. I haven't seen any plug-ins for point and spot lights that handle volumetric lighting well/fast enough for use in a game.

 

So looking over your implementation it looks very similar to my first attempt at improving unity's lighting. it would basically use a camera and render out depth texture/cubemap. It worked great for static lights but then as soon as they needed updating I ran into problems. I made a system that would switch out my light for a unity dynamic light and slowly rebake all the lights that become "dirty" over many frames. It worked ok but performance was still too slow if anything moved. that method had all the above issues you pointed out, that's why i dropped it and moved onto my new lighting method.

here are answers to your questions/statements.

1) Your lights use a lot more memory than Unity's, especially if you have more of them
A buffer size is set by the user, if their are no textures left to use, the light will be rendered as completely dynamic. dynamic lights are similar to the method unity uses but are slightly faster due to a improved shadow gather step. (the buffer textures are 16-bit btw)

2) Your lights only outperform Unity's if the shadow casters don't move and you therefore don't update the shadowmap
There are four modes my lights can be in, Edit: They switch between them automatically.
Dynamic - slightly out performs unity lighting due to faster shadow gather step
Stationary - massive speed improvements as you said due to saving data between frames.
Hybrid - Moving objects are rendered to a temp texture while stationary objects are saved between frames (depends on the ratio of stationary and moving objects, but generally massively outperforms unity.
no shadows - if no shadow casters are found in the area. massive improvements

3) This only works in deferred.
Correct. I'll update my post, Didn't think standard shader worked in forward.

4) A related problem to 2); As the code is slower but runs less, if something happens that has to update multiple lights you'll get a frame spike that may be higher than what you'd get every frame with builtin lights.
I think I've answered this question already based on the above questions. The code is faster then unity's. If i switch it to always dynamic it will out perform unitys shadows. The cost of "baking" stationary lights is the same cost as rendering a fully dynamic one, and a fully dynamic one is faster then unity's


Your suggestions:
1) LOD support. it improved performance for me a LOT; from ~70% of builtin ms to ~20% or so iirc.
do you mean LOD of meshs or LOD of light texture size? I already change the texture sized used for shadow maps based of light size and distance to the current camera.

2) I don't know how you're rendering into your shadowmap that makes this standard shader only but you could use Camera.RenderToCubemap with a replacement shader to generate the shadowmap, removing the standard shader requirement.
I don't use a camera to render my shadows, that is the slowest way to gather shadows, you end up rendering a bunch of shadows that don't intersect the current view frustum and the overhead of doing a camera.render is crazy. I'm still looking into expanding this to other shader/render paths. but the G-buffer that standard shader gives speeds up lighting a lot.

I think that's everything.

 

Not sure how this would have anything to do with changing the shadow quality with out having to change the quality profile.

I've already made my own GI solution for my game, It uses 3D light volumes. Although its not realtime, it runs a bake on level load, takes < 1-2 seconds (my game is procedural). Probably wont expand on the system because i have a game to make. I'll leave realtime GI up to more talented people (@sonicether).

 

Yes, that would be nice. I'm sure there would be demand for such a plug-in.
Also, this has been announced by Robert Cupisz from Unity, but no news since then. Some people say, he is now pursuing a career as a professional kite-boarder

 

Great to hear my implementation is a naive one, as I'm not all too happy with it as is for the same reasons you listed. The only thing I could think of at the time was using a camera, as I didn't make the connection between commandbuffer.setrendertarget and commandbuffer.drawmesh combined being the essence of a camera draw until, well, just now while writing this. Graphics/GL/Material classes together probably provide the same but only as an image effect.

3 new questions:
1) Would this asset come with all source code? (I'd probably just buy this and save myself a week or 2 fixing my own)
2) What kind of license would apply to this asset?
3) How are you keeping track of which meshes to draw into the shadowmaps?

Thanks for answering the questions, your product is more promising then I thought.

P.S: The standard shader requirement is then due to setting the pass to the default shadow caster pass?

 

1) Yes it would come with source.
2) What ever the opposite of per seat/game is (can't stand that)
3) I use a custom octree that has all the shadow casters in it. This also means that every object you want to cast a shadow needs to have a script on it.
4) I had to make my own depth shader, I couldn't find a way to find the shader pass number for the depth pass. right now the standard shader requirement is because that's all I use for my game. I'll look at expanding it into other render pipelines later.

 

I got directional lights casting shadows today. I Need to improve the occlusion step as its too slow but its looking promising. if i can beat unity's performance ill move onto volumetric and cascade shadows.

Already seeing a big improvement on shadow bias and quality of shadows when soft shadows are applied. seems to be a trend.

Screen shot of custom directional light with hard shadows, I have 2x2 and 4x4 PCF working as well.

 

This can make the workflow slow if we must attach a script on each object we drop in a scene we are making.

 

Not really, its just to register the object with the lighting system.

 

Sorry I read this wrong, I have a menu option that can go through the whole scene and adds the script automatically, or can go through your project folders and add it to the prefab, also their is an option to find all shadow casters on level load and add the script there. This is generally an issue with more advanced plugin from what I've seen. i wish unity allowed you to extend built in classes, that way we wouldn't have to resort to these methods.

 

Does it work in VR?

 

I know it's not something you're planning / needing for your own game but is there any chance you would be willing to support transparency for this asset? Is it possible with the approach you've taken? I would be very interested in it if so.

 

Lighting transparency and casting transparent shadows is a lot of work. honestly i think its outside the scope of this plugin. I don't have access to unity's source so its hard working on stuff like this in the sandbox that unity gives us.

I haven't tested, but it works with multiple cameras. shouldn't be to hard to get it working with VR.

 

Beta version of my lighting replacement for Unity3D.

Download link (windows only)

Press space to switch between Unity's lighting and mine. Press V to toggle volumetric lighing.

• Out performs unity lighting
• Works with Gbuffer (PBS)
• Real time Volumetric lighting
• Real time shadows.
• Way less light bleeding, shadow acne, peter panning
• Smooth Shadow distance fall off (No more pop in shadows)
• Better smooth shadows
• Light cookies (works but not supported till unity fixes a bug)
• Multi Camera friendly(reuses shadow maps from other cameras)

Hold Left mouse + arrow keys to fly around.
All the settings can be changed in game.

Works best on high settings and below. (buffer size not set for very high)

Currently supports point and spot lights, still working on a directional light.

I've been working on this for a few months now, our game is procedurally generated so we need fast dynamic shadow casting lights.
Let me know what you think, and let me know if you get any errors.

Edit: Im still working on directional lights.

 

This is ridiculously impressive.

 

I'm loving the demo. Great job. Even without transparency I'm sure I'd find a use for it. Great job.

 

For comparison

Unity



Custom without Volumetric



Custom with Volumetric



And my specs...

Geforce GTX 980
i7-5820K

So great performance - also the quality when in close is excellent.

EDIT: Updated with new screenshots on a newer build

 

I Linked the wrong build, its updated now, should run even faster on very high settings. and has volumetric enabled by default.

 

I'll download and update my post =)

 

Even better performance (and I love the volumetric too!)

 

It's preferable for VR to use forward rendering rather than deferred as temporal aliasing is much more apparent in VR and MSAA doesn't work in Unity's deferred renderer. Post process anti-aliasing like FXAA don't handle temporal aliasing, only pixel aliasing. Until Unity's temporal SMAA is fully fleshed out to near the quality of UE4's TAA (possibly never) people shouldn't be using Unity's deferred rendering path for VR. Sadly that means these shadows don't really work for VR.

That said people will certainly try, I just wouldn't spend too much time on it. Though if you ever got shadow casting on transparency working that's the most of the same work needed to get it working in forward.

 

Yea this looks pretty great.

Feature for feature, Unity vs. Custom I get a difference of about 12 - 15% speed increase with Custom. Yay!

-Steven

 

Wow! What kind of sorcery this is?!?
Looking really gorgeous.
I want to ask a few questions :

-In most of the demo, I'm getting better fps with custom shadows but at the starting place (interior) Unity's shadows has higher FPS. I was wondering what does effect FPS?

-"Doesn't support transparency" but "alpha cut" is OK
Does that mean it doesn't working with "Fade" and "Transparency" Rendering Modes? Any possibility of a workaround?

-Is it going to be easy to make custom shaders work with your system?

-When do you think you'll be submitting the asset?

-Can you explain "Hybrid" and "Dynamic Only" modes in detail? Is there a specific place in the demo to see its effect?

-Did you had any chance to test your system with Post Effects?

 

1) This is not the case for me, can you send me screen shots so i can see this in action? If you turn off volumetric does it help? The volumetric is extra on top of the lighting. there can be times that the speed increase of my shadows doesn't offset the extra cost of the volumetric pass. Also you might have disabled the hybrid mode (M)

2.) correct it doesn't work with those modes. still working on a fix, but i think i'd need the source for that. I might do what most games do and only render every x pixel to
imitate transparency. That's the only easy work around.

3.) probably really hard and a lot of work to get custom shaders that dont render into the G buffer working.

4.) hybrid mode will reuse shadow data for stationary lights from the previous frame. Stationary objects are render and saved out, while dynamic objects are rendered every frame. think of it like runtime dual light maps. it has a Vram cost though, You set a buffer size to dedicated to this mode. dynamic mode is similar to unity's dynamic shadows (not saving data between frames).

5.) should work with most if not all screen effects.

 

Screen door sounds good. Only worrying part of this system is no enlighten. Shame it can't support it.

 

surprisingly impressive the performance and quality are far better rather than built in solution.
ooh and i stay in quarter resolution for the volumetric just fyi

 

Thanks, quarter resolution is best, I included the other res just in case someone has a good computer. There are some artifacts from using quarter resolution but they aren't noticable unless you turn up the volumetric density by a lot.

 

Yep i notice the artifact actually, but honestly it's not too bad. Maybe you can some settings to use noise sampling and Blur the volumetric effect if user decide to use low resolution buffer that should help.
Oh forgot to ask this but how many raymarch samples for the volumetric effect that you used?

 

It does exactly that. its uses 16 samples per pixel and an 8x8 tile offset for noise, that results in 1024 samples per tile on full res. The sample count can be increased to what ever you want, although anything less then 16 looks bad and anything over 32 isn't that much more of a change in this scene. It then blurs the texture using a depth aware blur.

 

Hmm odd, that's should at least reduce the low resolution artifact or make the volumetric softer, but it's still noticeable when the volumetric intensity are high.
nevermind it's still in development so i believe you gonna add some improvement .
Great job