Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Dynamic Volumetric Lighting replacement

Discussion in 'Works In Progress' started by Lexie, Jan 11, 2016.

  1. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I was really unhappy with the quality and performance of Unity's shadows. I tried a few ways to speed it up but they all relied on unity's lighting in the back end.

    So I've completely replaced Unity's dynamic lighting.


    EDIT: Test Build (updated 3/9/2016)

    Features
    • Out performs unity's shadows
    • Real time Volumetric lighting
    • Better smooth shadows
    • Way less light bleeding, shadow acne, peter panning
    • Smooth Shadow distance fall off (No more pop in shadows)
    • Coloured Light cookies
    • Multi Camera friendly(reuses shadow maps from other cameras)
    • Can handle Higher res shadow maps (not recommended for most GPU's).
    • Shadow quality can be changed at run time with out needing to change settings profile (Why unity)
    • Currently supports point, spot lights and Directional light
    • shadow caster exclusion list (great for torches)
    To do
    • Area lights
    • Soft shadow on directional light.
    • Cascade Directional light.
    • LOD friendly
    Downsides
    • Doesn't support transparency (supports alpha cut out and alpha overlay though)
    • Only supports Standard shader family
    • Doesn't interface with light probs or enlighten
    • Edit: Currently Only works in deferred shading path.
    • Edit: Uses VRAM to increase performance (size set by user) Still out performs unity if buffer size is 0

    Some of these downside could be fixed but I'd need Unity source or their Render API improved.
    The game I'm working on doesn't need transparency and is procedurally generated, So most of these downside don't apply to me.

    Going to finish up directional lighting this week and hopefully have a build ready for testing soon.
    If anyone has any feature requests, I'd love to hear it!
     
    Last edited: Mar 10, 2016
  2. jc_lvngstn

    jc_lvngstn

    Joined:
    Jul 19, 2006
    Posts:
    1,507
    Very interesting, any chance you could show a greater variety of settings, such as a forested scene, that sort of thing?
     
  3. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Yeah, i need to work on some example scenes. I might just buy a premade one off the store, probably easier then wasting a day.
     
  4. Howard-Day

    Howard-Day

    Joined:
    Oct 25, 2013
    Posts:
    125
    This looks awesome. I'd like to see a demo with much stronger volumetric effects - to see how far it can be pushed. I'd also like to see how it interacts with enlighten, and how it looks when two different colored light volumetrics interact/intersect. I'm really stoked about this, as I'm having endless shadow gap issues. You also mention that directional and area lights are to do - does that mean that this only works with point and spot? Or only point?
    Thanks in advance!
     
  5. Chaosgod_Esper

    Chaosgod_Esper

    Joined:
    Oct 25, 2012
    Posts:
    295
    PINNED

    And agreed with the previous posters.
    What do you think you'll set as start price?
     
  6. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    Fixing shadow acne looks amazing. We've had a lot of trouble with that in our procedural game.

    The limitations do seem a little hard (only working with standard shaders and no transparency). Depending on price it's definitely something I'd be tempted to look into and research with to see if it fits our game.
     
  7. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    Interior would be good to have a look at also. An interior with a bit more substance than what the video shows (on the light bleeding part). Thanks!
     
  8. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Right now it works with spot and point lights. I should have directional done by the end of the week. Area lights will probably be the last thing I add. Unfortunately my lights don't interact with enlighten, I'd need unity's source to get that working. you might be able to fake it by having a unity light in the same place during the baking process, but I haven't tested it.

    Here's a picture of stronger scattering and colour mixing, Ill add more scatter functions soon, right now it scatters in all directions with the same strength

    If i sell it, probably around 50-60, but ill have to do some more research first.

    Ill include a few scenes in the demo, its hard to show off this effect with video and screen shots. If anyone has any recommendation on scenes, id love to hear it.
     
    TwiiK, Howard-Day and FreakForFreedom like this.
  9. Schneider21

    Schneider21

    Joined:
    Feb 6, 2014
    Posts:
    2,960
  10. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Nearly got cascade shadow maps generating today. Couldn't find a way to limit the render area in a command buffer so i had to make a custom blit shader to copy 4 textures into one.

    Note: color is compressed in the below image

     
    Last edited: Jan 13, 2016
    mwituni and Howard-Day like this.
  11. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,726
    @Jesper Mortensen or @KEngelstoft have some idea of how to render lighting to enlighten without unity lights or tip for integrating other lighting systems such as this one?
     
    MrEsquire, AcidArrow and Howard-Day like this.
  12. darkhog

    darkhog

    Joined:
    Dec 4, 2012
    Posts:
    2,219
    Good one. I have lots of issues with shadow acne, a.k.a banding in my game, also (to less degree) peter panning. Too bad you can't make this work with legacy diffuse shader (which is what I'm using for my game to get oldschool flatshaded look), otherwise it'd be an instant buy from me (don't care about transparency either or interfacing with probes).
     
  13. TwiiK

    TwiiK

    Joined:
    Oct 23, 2007
    Posts:
    1,678
    Those red and blue screenshots look absolutely awesome. :p
     
  14. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    You can modify the internal standard shader to get flat shading, we do it for our game.
     
  15. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    So this becomes useless if it can't mix smoothly with Enlighten GI.
    I mean this will work great with only real time lightening games.
     
  16. MrEsquire

    MrEsquire

    Joined:
    Nov 5, 2013
    Posts:
    2,712
    Amazing work dude, fully credit to you on trying to do something on your own and improve on things where a large company has.

    My question is, how is mobile performance?, will it be suitable to use on mobile devices.

    Also I feel Unity should help you with what you need, the sources etc.
     
    Last edited: Jan 13, 2016
    zenGarden likes this.
  17. Zuntatos

    Zuntatos

    Joined:
    Nov 18, 2012
    Posts:
    557
    So if I understand correctly, in essence it's similar to my own implementation of lights:
    http://forum.unity3d.com/threads/procedural-static-point-lights-with-source.349155/

    Sounds like an increase in shadowmap precision, probably R16->R32, doubling memory use. Atleast, for point lights as those don't support dedicated depth formats.
    So you have a unique shadowmap for each light? Your memory requirements would then scale with the number of lights, while Unity lights have a constant requirement.
    I may be wrong here, but this is due to keeping shadows over multiple frames? As that's the only way I've found to increase the performance of unity lights significantly, and the only other way I found was clipping pixels covered if they're not in range of the light. You've also disabled to 'wiggling' of torches in the viking village demo, further increasing the chances of this being correct.
    Internally Unity only has 1 shadowmap of each required quality, and draws over the content just before it renders the light onto the screen. This is why there isn't a update-per-script option like on reflection probes, there is no per-light buffer you can keep.

    It would be nice if you can include the downsides of these things in the OP:

    1) Your lights use a lot more memory than Unity's, especially if you have more of them
    2) Your lights only outperform Unity's if the shadow casters don't move and you therefore don't update the shadowmap
    3) This only works in deferred.
    4) A related problem to 2); As the code is slower but runs less, if something happens that has to update multiple lights you'll get a frame spike that may be higher than what you'd get every frame with builtin lights.

    My suggestions:
    1) LOD support. it improved performance for me a LOT; from ~70% of builtin ms to ~20% or so iirc.
    2) I don't know how you're rendering into your shadowmap that makes this standard shader only but you could use Camera.RenderToCubemap with a replacement shader to generate the shadowmap, removing the standard shader requirement.

    I'd love to be proven wrong, and that this isn't what I described above as that'd mean there are lots of further improvements to be made on top of a good base.

    Great job at the volumetric lighting though, looks awesome! Next step is voxelizing the world in range of the point light and conetrace some GI through it :p
     
    AcidArrow likes this.
  18. Soul-Challenger

    Soul-Challenger

    Joined:
    Dec 30, 2010
    Posts:
    152
    Hello Lexie,

    I've been following your progress on Hitbox Team - this is looking really promising. Definitely interested :) ! Especially the volumetrics. Though I would need it to play along with light probes and Enlighten.

    So far, I've managed to create a sort of fake lit-fog with particles which look OK at far distances:
    TRG_indieDB_jan16_B.jpg

    ...but it doesn't look good when close to camera/light sources, as you can see in this vid:


    Your implementation would really bring my project to a whole new level.
    Keep up the excellent work! :)
     
    Last edited: Jan 13, 2016
  19. dogzerx2

    dogzerx2

    Joined:
    Dec 27, 2009
    Posts:
    3,873
    This looks pretty good!

    A must have!
     
  20. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I haven't done any research into making it work with enlighten yet. My game is procedurally generated so figuring out if i can get it working with light probes and enlighten is at the end of my list. My guess is I'll need some enlighten API opened up.
     
  21. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I haven't check out performance on mobile yet, I don't have a phone. I'll borrow one from a friend once its done and test it out.

    Maybe i should pull the volumetric stuff out and get it working with unity's lighting as well. I haven't seen any plug-ins for point and spot lights that handle volumetric lighting well/fast enough for use in a game.
     
  22. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    So looking over your implementation it looks very similar to my first attempt at improving unity's lighting. it would basically use a camera and render out depth texture/cubemap. It worked great for static lights but then as soon as they needed updating I ran into problems. I made a system that would switch out my light for a unity dynamic light and slowly rebake all the lights that become "dirty" over many frames. It worked ok but performance was still too slow if anything moved. that method had all the above issues you pointed out, that's why i dropped it and moved onto my new lighting method.

    here are answers to your questions/statements.

    1) Your lights use a lot more memory than Unity's, especially if you have more of them
    A buffer size is set by the user, if their are no textures left to use, the light will be rendered as completely dynamic. dynamic lights are similar to the method unity uses but are slightly faster due to a improved shadow gather step. (the buffer textures are 16-bit btw)

    2) Your lights only outperform Unity's if the shadow casters don't move and you therefore don't update the shadowmap
    There are four modes my lights can be in, Edit: They switch between them automatically.
    Dynamic - slightly out performs unity lighting due to faster shadow gather step
    Stationary - massive speed improvements as you said due to saving data between frames.
    Hybrid - Moving objects are rendered to a temp texture while stationary objects are saved between frames (depends on the ratio of stationary and moving objects, but generally massively outperforms unity.
    no shadows - if no shadow casters are found in the area. massive improvements

    3) This only works in deferred.
    Correct. I'll update my post, Didn't think standard shader worked in forward.

    4) A related problem to 2); As the code is slower but runs less, if something happens that has to update multiple lights you'll get a frame spike that may be higher than what you'd get every frame with builtin lights.
    I think I've answered this question already based on the above questions. The code is faster then unity's. If i switch it to always dynamic it will out perform unitys shadows. The cost of "baking" stationary lights is the same cost as rendering a fully dynamic one, and a fully dynamic one is faster then unity's


    Your suggestions:
    1) LOD support. it improved performance for me a LOT; from ~70% of builtin ms to ~20% or so iirc.
    do you mean LOD of meshs or LOD of light texture size? I already change the texture sized used for shadow maps based of light size and distance to the current camera.

    2) I don't know how you're rendering into your shadowmap that makes this standard shader only but you could use Camera.RenderToCubemap with a replacement shader to generate the shadowmap, removing the standard shader requirement.
    I don't use a camera to render my shadows, that is the slowest way to gather shadows, you end up rendering a bunch of shadows that don't intersect the current view frustum and the overhead of doing a camera.render is crazy. I'm still looking into expanding this to other shader/render paths. but the G-buffer that standard shader gives speeds up lighting a lot.

    I think that's everything.
     
    Last edited: Jan 13, 2016
    MrEsquire, AcidArrow, hopeful and 2 others like this.
  23. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Not sure how this would have anything to do with changing the shadow quality with out having to change the quality profile.

    I've already made my own GI solution for my game, It uses 3D light volumes. Although its not realtime, it runs a bake on level load, takes < 1-2 seconds (my game is procedural). Probably wont expand on the system because i have a game to make. I'll leave realtime GI up to more talented people (@sonicether).
     
  24. Soul-Challenger

    Soul-Challenger

    Joined:
    Dec 30, 2010
    Posts:
    152
    Yes, that would be nice. I'm sure there would be demand for such a plug-in.
    Also, this has been announced by Robert Cupisz from Unity, but no news since then. Some people say, he is now pursuing a career as a professional kite-boarder :D
     
  25. Zuntatos

    Zuntatos

    Joined:
    Nov 18, 2012
    Posts:
    557
    Great to hear my implementation is a naive one, as I'm not all too happy with it as is for the same reasons you listed. The only thing I could think of at the time was using a camera, as I didn't make the connection between commandbuffer.setrendertarget and commandbuffer.drawmesh combined being the essence of a camera draw until, well, just now while writing this. Graphics/GL/Material classes together probably provide the same but only as an image effect.

    3 new questions:
    1) Would this asset come with all source code? (I'd probably just buy this and save myself a week or 2 fixing my own)
    2) What kind of license would apply to this asset?
    3) How are you keeping track of which meshes to draw into the shadowmaps?

    Thanks for answering the questions, your product is more promising then I thought.

    P.S: The standard shader requirement is then due to setting the pass to the default shadow caster pass?
     
    hopeful likes this.
  26. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    1) Yes it would come with source.
    2) What ever the opposite of per seat/game is (can't stand that)
    3) I use a custom octree that has all the shadow casters in it. This also means that every object you want to cast a shadow needs to have a script on it.
    4) I had to make my own depth shader, I couldn't find a way to find the shader pass number for the depth pass. right now the standard shader requirement is because that's all I use for my game. I'll look at expanding it into other render pipelines later.
     
  27. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I got directional lights casting shadows today. I Need to improve the occlusion step as its too slow but its looking promising. if i can beat unity's performance ill move onto volumetric and cascade shadows.

    Already seeing a big improvement on shadow bias and quality of shadows when soft shadows are applied. seems to be a trend.

    Screen shot of custom directional light with hard shadows, I have 2x2 and 4x4 PCF working as well.

     
    chingwa, Rodolfo-Rubens, Fab4 and 2 others like this.
  28. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    This can make the workflow slow if we must attach a script on each object we drop in a scene we are making.
     
  29. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Not really, its just to register the object with the lighting system.
     
  30. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Sorry I read this wrong, I have a menu option that can go through the whole scene and adds the script automatically, or can go through your project folders and add it to the prefab, also their is an option to find all shadow casters on level load and add the script there. This is generally an issue with more advanced plugin from what I've seen. i wish unity allowed you to extend built in classes, that way we wouldn't have to resort to these methods.
     
    zenGarden and hopeful like this.
  31. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,342
    Does it work in VR?
     
  32. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    I know it's not something you're planning / needing for your own game but is there any chance you would be willing to support transparency for this asset? Is it possible with the approach you've taken? I would be very interested in it if so.
     
  33. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Lighting transparency and casting transparent shadows is a lot of work. honestly i think its outside the scope of this plugin. I don't have access to unity's source so its hard working on stuff like this in the sandbox that unity gives us.

    I haven't tested, but it works with multiple cameras. shouldn't be to hard to get it working with VR.
     
  34. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Beta version of my lighting replacement for Unity3D.

    Download link (windows only)

    Press space to switch between Unity's lighting and mine. Press V to toggle volumetric lighing.

    • Out performs unity lighting
    • Works with Gbuffer (PBS)
    • Real time Volumetric lighting
    • Real time shadows.
    • Way less light bleeding, shadow acne, peter panning
    • Smooth Shadow distance fall off (No more pop in shadows)
    • Better smooth shadows
    • Light cookies (works but not supported till unity fixes a bug)
    • Multi Camera friendly(reuses shadow maps from other cameras)

    Hold Left mouse + arrow keys to fly around.
    All the settings can be changed in game.

    Works best on high settings and below. (buffer size not set for very high)

    Currently supports point and spot lights, still working on a directional light.

    I've been working on this for a few months now, our game is procedurally generated so we need fast dynamic shadow casting lights.
    Let me know what you think, and let me know if you get any errors.

    Edit: Im still working on directional lights.
     
    Last edited: Jan 20, 2016
    code-blep, elias_t, Simie and 2 others like this.
  35. iamthwee

    iamthwee

    Joined:
    Nov 27, 2015
    Posts:
    2,155
    This is ridiculously impressive.
     
    Lexie likes this.
  36. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    I'm loving the demo. Great job. Even without transparency I'm sure I'd find a use for it. Great job.
     
  37. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    For comparison

    Unity




    Custom without Volumetric



    Custom with Volumetric



    And my specs...

    Geforce GTX 980
    i7-5820K

    So great performance - also the quality when in close is excellent.

    EDIT: Updated with new screenshots on a newer build
     
    Last edited: Jan 20, 2016
  38. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I Linked the wrong build, its updated now, should run even faster on very high settings. and has volumetric enabled by default.
     
  39. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    I'll download and update my post =)
     
  40. CWolf

    CWolf

    Joined:
    Oct 24, 2011
    Posts:
    106
    Even better performance (and I love the volumetric too!)
     
  41. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,186
    It's preferable for VR to use forward rendering rather than deferred as temporal aliasing is much more apparent in VR and MSAA doesn't work in Unity's deferred renderer. Post process anti-aliasing like FXAA don't handle temporal aliasing, only pixel aliasing. Until Unity's temporal SMAA is fully fleshed out to near the quality of UE4's TAA (possibly never) people shouldn't be using Unity's deferred rendering path for VR. Sadly that means these shadows don't really work for VR.

    That said people will certainly try, I just wouldn't spend too much time on it. Though if you ever got shadow casting on transparency working that's the most of the same work needed to get it working in forward.
     
  42. SteveB

    SteveB

    Joined:
    Jan 17, 2009
    Posts:
    1,448
    Yea this looks pretty great.

    Feature for feature, Unity vs. Custom I get a difference of about 12 - 15% speed increase with Custom. Yay!

    -Steven
     
  43. gurayg

    gurayg

    Joined:
    Nov 28, 2013
    Posts:
    232
    Wow! What kind of sorcery this is?!?
    Looking really gorgeous.
    I want to ask a few questions :

    -In most of the demo, I'm getting better fps with custom shadows but at the starting place (interior) Unity's shadows has higher FPS. I was wondering what does effect FPS?

    -"Doesn't support transparency" but "alpha cut" is OK
    Does that mean it doesn't working with "Fade" and "Transparency" Rendering Modes? Any possibility of a workaround?

    -Is it going to be easy to make custom shaders work with your system?

    -When do you think you'll be submitting the asset?

    -Can you explain "Hybrid" and "Dynamic Only" modes in detail? Is there a specific place in the demo to see its effect?

    -Did you had any chance to test your system with Post Effects?
     
  44. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    1) This is not the case for me, can you send me screen shots so i can see this in action? If you turn off volumetric does it help? The volumetric is extra on top of the lighting. there can be times that the speed increase of my shadows doesn't offset the extra cost of the volumetric pass. Also you might have disabled the hybrid mode (M)

    2.) correct it doesn't work with those modes. still working on a fix, but i think i'd need the source for that. I might do what most games do and only render every x pixel to
    imitate transparency. That's the only easy work around.

    3.) probably really hard and a lot of work to get custom shaders that dont render into the G buffer working.

    4.) hybrid mode will reuse shadow data for stationary lights from the previous frame. Stationary objects are render and saved out, while dynamic objects are rendered every frame. think of it like runtime dual light maps. it has a Vram cost though, You set a buffer size to dedicated to this mode. dynamic mode is similar to unity's dynamic shadows (not saving data between frames).

    5.) should work with most if not all screen effects.
     
  45. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,726
    Screen door sounds good. Only worrying part of this system is no enlighten. Shame it can't support it.
     
  46. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,427
    surprisingly impressive the performance and quality are far better rather than built in solution.
    ooh and i stay in quarter resolution for the volumetric just fyi
     
  47. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Thanks, quarter resolution is best, I included the other res just in case someone has a good computer. There are some artifacts from using quarter resolution but they aren't noticable unless you turn up the volumetric density by a lot.
     
  48. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,427
    Yep i notice the artifact actually, but honestly it's not too bad. Maybe you can some settings to use noise sampling and Blur the volumetric effect if user decide to use low resolution buffer that should help.
    Oh forgot to ask this but how many raymarch samples for the volumetric effect that you used?
     
  49. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    It does exactly that. its uses 16 samples per pixel and an 8x8 tile offset for noise, that results in 1024 samples per tile on full res. The sample count can be increased to what ever you want, although anything less then 16 looks bad and anything over 32 isn't that much more of a change in this scene. It then blurs the texture using a depth aware blur.
     
  50. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,427
    Hmm odd, that's should at least reduce the low resolution artifact or make the volumetric softer, but it's still noticeable when the volumetric intensity are high.
    nevermind it's still in development so i believe you gonna add some improvement :).
    Great job
     
unityunity