Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Dynamic Volumetric Lighting replacement

Discussion in 'Works In Progress' started by Lexie, Jan 11, 2016.

  1. Michal_

    Michal_

    Joined:
    Jan 14, 2015
    Posts:
    361
    You are correct but why would you have a shadow casting light if there is no object inside light's volume? The only problem I see here is that unity will turn the shadows off if the light is too far from the camera. But I can easily detect that and disable shadows for that particular volume. Anyway, here is what I do.
    1. Shadow casting lights - I use LightEvent.AfterShadowMap to get light's shadow map and render light's volume. It takes some reverse-engineering to replicate all the built-in lighting parameters but it can be done.
    2. Lights without shadows - I use CameraEvent.BeforeLighting to render volumes for all lights that don't cast shadows (including shadow casting lights too far from camera). I should cull them here based on visibility to save a few draw calls but I don't really care. I let the rasterizer clip them.
    It is a pretty standard rendering setup otherwise (with all the drawbacks). Render to smaller light buffer, ray marching with dithering, blur, bilateral upscale. Animated 3D texture for volumetric noise and mie scattering. Everything else respects built-in unity lights.

    I don't support directional lights. I did all of it just for fun. I have zero interest in using it in a game. And since I didn't need directional light for my demo I didn't bother. With that being said, I will need to trace cascade shadows for my next hobby project. I also love to hack and reverse-engineer stuff :) So I will look into it and tell you what I find.

    Also, great job with your new lighting. Unity can be pretty slow sometimes. We actually have zero built-in shader policy at work for performance reasons.
     
    hopeful likes this.
  2. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    You are correct, but you need to know when this happens. If you had a shadow casting light moving around and it was to go into the air then suddenly that LightEvent.AfterShadowMap would stop getting called. you wouldn't have anyway of knowing to render a none shadow casting volume there.

    calculating the values for point and spot lights are do-able, its the directional that i see issues with. actually i was able to get the directional light render position, I should be able to calculate everything else from that. will have to look into it more.
     
  3. Michal_

    Michal_

    Joined:
    Jan 14, 2015
    Posts:
    361
    That is a problem but I would say it is a pretty rare use case. You could add a tiny shadow casting 3d object to your light to hack your way around it. It is not very pretty I admit.

    I definitely hope Unity will improve command buffers though. They are pretty limited and not all of it works to make it worse.
     
    hippocoder likes this.
  4. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Yeah a custom material that doesn't write into the depth buffer but comes up in the shadow search to get around it would work. Kinda the same way "standard transparent shadows work" in unity.

    yeah I'm really liking command buffers, I hope they keep expanding on it.
     
  5. GoGoGadget

    GoGoGadget

    Joined:
    Sep 23, 2013
    Posts:
    716
    This is great stuff, on one hand it's cool to see how much you can extend Unity, on the other it's not nice to see all of the limitations and issues with the builtin Unity system (like using a cookie for spot lights, seriously?). Keep up the great work.

    Also, you say this system will only work with the standard shader, if I'm using something like Uber in deferred rendering, would it work then as well?
     
  6. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,727
    I just hope graphics plumbers like @Aras notice little topics like this tucked away, particularly as he asked for command buffer feedback at one point but I don't think he got it. Here is some good feedback from @Michal_ and @Lexie
     
  7. Michal_

    Michal_

    Joined:
    Jan 14, 2015
    Posts:
    361
    Ok, I have the directional light working. It turned out there is a simple solution. No reverse-engineering required. I'll explain tomorrow. I'm dead now...
     
  8. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Well done, would love to hear it, There is part of me that wants a custom directional light, and another part of me that just wants to use the inbuilt one. I know a few ways I could speed up the directional lighting and handle large worlds with massive shadow distance, Similarly to how farcray''s lighting works. But I don't think i need it for my game. soooo maybe the inbuilt lighting + volumetric will be enough.

    The biggest issue with a directional light is I wanted to increase the shadow rendering part to offset the cost of volumetric lighting, Similar to how my stationary lights work. I was thinking about a tiled system that updates one tile per frame for stationary objects. then the dynamic light would fall back to using the tiled system for anything past the dynamic cascade. this would mean i could have a really short dynamic shadow distance (reduce shadow draw call) but still have a large shadow distance.
     
    Last edited: Mar 12, 2016
  9. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,727
    Pretty much the holy grail. Unity needs this built in or frankly, open world kind of sucks with any sort of time of day.
     
  10. Michal_

    Michal_

    Joined:
    Jan 14, 2015
    Posts:
    361
    Sorry it took me so long. Crazy weekend.

    There is one key difference between directional light and point (spot) light. As I said I use LightEvent.AfterShadowMap to retrieve light's shadow map and render light's volume. The problem is that all the shader parameters related to lighting and shadows aren't set at that moment by Unity. I'm talking about parameters like _World2Shadow matrices, _LightMatrix0, _LightPos etc. So you have to figure out what exactly they mean and how to compute them in the exact same way as Unity would. And that is the tricky part since there is no documentation and some of the parameters differ on different platforms to make it more interesting. I mean it is easy to guess that _LightPos.w is 1 / (range * range) but getting the matrices 1:1 is more difficult. Doing all that for more complex cascaded shadow maps would be even more difficult. Thankfully there is an easy way out.

    For directional light I still use LightEvent.AfterShadowMap to store the shadow map to a global shader parameter but I don't render anything at this point. I wait for LightEvent.BeforeScreenspaceMask instead. It turned out that by the time BeforeScreenspaceMask event is triggered, all shader parameters related to cascaded shadows are already updated by Unity. All you have to do is look into builtin shader source code and copy the part that reads CSM. You don't even have to know what CSM is to use it. Here's the code snippet for reference (it doesn't handle shadow fading)
    Code (CSharp):
    1. UNITY_DECLARE_SHADOWMAP(_CascadeShadowMapTexture);
    2.  
    3.         inline fixed4 GetCascadeWeights_SplitSpheres(float3 wpos)
    4.         {
    5.             float3 fromCenter0 = wpos.xyz - unity_ShadowSplitSpheres[0].xyz;
    6.             float3 fromCenter1 = wpos.xyz - unity_ShadowSplitSpheres[1].xyz;
    7.             float3 fromCenter2 = wpos.xyz - unity_ShadowSplitSpheres[2].xyz;
    8.             float3 fromCenter3 = wpos.xyz - unity_ShadowSplitSpheres[3].xyz;
    9.             float4 distances2 = float4(dot(fromCenter0, fromCenter0), dot(fromCenter1, fromCenter1), dot(fromCenter2, fromCenter2), dot(fromCenter3, fromCenter3));
    10.  
    11.             fixed4 weights = float4(distances2 < unity_ShadowSplitSqRadii);
    12.             weights.yzw = saturate(weights.yzw - weights.xyz);
    13.  
    14.             return weights;
    15.         }
    16.  
    17.         inline float4 GetCascadeShadowCoord(float4 wpos, fixed4 cascadeWeights)
    18.         {
    19.             float3 sc0 = mul(unity_World2Shadow[0], wpos).xyz;
    20.             float3 sc1 = mul(unity_World2Shadow[1], wpos).xyz;
    21.             float3 sc2 = mul(unity_World2Shadow[2], wpos).xyz;
    22.             float3 sc3 = mul(unity_World2Shadow[3], wpos).xyz;
    23.             return float4(sc0 * cascadeWeights[0] + sc1 * cascadeWeights[1] + sc2 * cascadeWeights[2] + sc3 * cascadeWeights[3], 1);
    24.         }
    25.  
    26.         inline float SampleCascadeShadowMap(float3 wpos)
    27.         {
    28.             float4 cascadeWeights = GetCascadeWeights_SplitSpheres(wpos);
    29.             float4 samplePos = GetCascadeShadowCoord(float4(wpos, 1), cascadeWeights);
    30.             return UNITY_SAMPLE_SHADOW(_CascadeShadowMapTexture, samplePos.xyz);
    31.         }
     
  11. Michal_

    Michal_

    Joined:
    Jan 14, 2015
    Posts:
    361
    That would be great. Personally, I'm putting all my hopes into hw instancing at the moment. We're rendering entire forests in Ylands and the draw call count can get pretty ridiculous even without shadows...
     
    frosted and hopeful like this.
  12. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    For some reason i never tested BeforeScreenspaceMask map, I only tested AfterScreenspaceMask shadow map hoping that these values would still be set, turns out the scene view overrides them in AfterScreenspaceMask.
     
    Last edited: Mar 14, 2016
  13. Dreamaster

    Dreamaster

    Joined:
    Aug 4, 2014
    Posts:
    148
    I want this SO BAD.
     
  14. Beinscheibe

    Beinscheibe

    Joined:
    Sep 24, 2013
    Posts:
    78
    Can't wait either. This is exactly what we need right now ^^
    If only Unity would come up with something like this.

    @Lexie: You got a private message from me, regarding a question ;-D
     
    jdraper3 likes this.
  15. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    So I've added volumetric lighting to unity's inbuilt lighting now. It uses the command buffer to capture the shadow maps and renders the volumes into a temporary texture. It works with direction/point/spot lights. To get it working you simply drop a script on your camera and a script on any light you want to be volumetric. The rest is automatic.

    The main reason for doing this is my custom lighting solution I've made has too many downside to really release. Its perfect for my game so I think it will stay as in house tech for now. By adding volumetric support to unity's inbuilt lighting I can be assured it will work for everyone with out them needing to majorly alter their rendering pipeline/artstyle.

    @Michal_ is also working on volumetric support for unity's inbuilt lighting. they've said they will be open sourcing it soon in this thread for people that are interested, chances are it will beat me to release.
     
    John-G, Beinscheibe and GoGoGadget like this.
  16. Beinscheibe

    Beinscheibe

    Joined:
    Sep 24, 2013
    Posts:
    78
    I will watch both threads from now on
    Just Hope one gets released soon.

    But its nice to see that there are now 2 solutions in the work.
    Also I understand the reason to keep your lighting Solution unreleased
     
  17. Dreamaster

    Dreamaster

    Joined:
    Aug 4, 2014
    Posts:
    148
    WHAT?
    /SLAP
    But your custom solution had better shadows and no light leakage!
     
  18. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    652
    I too was looking forward to the better shadows and faster static shadows. :\ Other than that everything sounds good.
     
  19. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    It's also faster... you can modify unity's shaders to fix their awful shadow bias if you want to remove the light bleeding and have better shadows. I'm not doing anything special there. Just calculating bias correctly. You can also change the way softshadows work so that could also be added to unity's inbuilt
     
  20. Olafson

    Olafson

    Joined:
    Aug 7, 2012
    Posts:
    252
    Yeah I think it would still be worth for you if you release your solution for point and directional lights. Even if it does not improve directional lights to much and has problems with fully dynamic lights, I am sure people will buy it.

    I know I would.
     
  21. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Ill be releasing a volumetric plugin for unity's inbuilt lighting with-in a day or two. Just cleaning up all the editor stuff now.
    I was also able to refine my upsampling to allow acceptable quality at eight resolution. I think I have finally come up with a solution to render a lot of volumetric lights at acceptable speeds to use in game. It supports dx9 and above, works in forward and deferred path.

    I'm thinking about a price point of 10-15$. I know there are other volumetric lighting solutions on the store that cost way more then that but I wanted to make sure the price was reasonable for hobbyist.

    If/when my custom replacement lighting is at an acceptable level, I'll release that as a beta branch to this asset, There are just too many downside to charge money for it so at the very least I can offer the volumetric support with inbuilt lighting.
     
    moure, jdraper3, Kenaz and 3 others like this.
  22. GoGoGadget

    GoGoGadget

    Joined:
    Sep 23, 2013
    Posts:
    716
    Sounds great, a plug-and-play solution that doesn't go too deep will be very handy for those of us who already have a custom pipeline. I will definitely be picking this up when it comes out!
     
  23. Beinscheibe

    Beinscheibe

    Joined:
    Sep 24, 2013
    Posts:
    78
    This really sounds great. Will definitly pick it up and check it out.
    And the idea with your lighting solution sounds great as well. Keep it coming ;D
     
  24. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    It looks very cool and i will be sure to buy this asset if it's priced fairly.

    Keep up the great work.
     
  25. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I ran into some issues with DX9 support so it set me back a day, all fixed up now. As long as my closed beta goes well I should have it ready by the end of the week.
     
    Beinscheibe and Kenaz like this.
  26. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    227
    Any idea if this will render properly in VR? We've tried a couple of the solutions from the asset store, and they looked great in the editor, but once in VR the effect really fell apart.
     
  27. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    I would also be very interested in the performance in VR.
     
  28. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I don't have a VR head set to test it out. Going to borrow one from a friend to see how hard it is to get it working with it.

    My beta is complete, just doing up the documentation and marketing stuff now... No idea how long this will take.
     
  29. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    That sounds fantastic. VR is of course still a very new thing, so you should be fine in releasing the asset as long as you initially state that VR is untested.

    When do you think we will see a release on the asset store? Dying to mess around with it :)
     
  30. Elecman

    Elecman

    Joined:
    May 5, 2011
    Posts:
    1,342
    I would like to see it work in VR also,
     
  31. MyGenericUsername

    MyGenericUsername

    Joined:
    Sep 6, 2014
    Posts:
    95
    Any expected date to have this available to all? I'm really excited for this asset.
     
  32. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I'm borrowing the new oculus over the weekend, shouldn't be too hard to get it working. It already supports multi cameras.

    I should have all the marketing and documentation ready by the end of tomorrow. so that + how ever long asset store approval takes would be my ETA.
     
    MS80 and Quique-Martinez like this.
  33. MyGenericUsername

    MyGenericUsername

    Joined:
    Sep 6, 2014
    Posts:
    95
    NICE!! Can't wait.
     
  34. Quique-Martinez

    Quique-Martinez

    Joined:
    Oct 1, 2013
    Posts:
    112
    So impressed by the demo!!
    Looking forward the asset.
     
  35. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I should really update that, So what I'm releasing isn't that demo, That demo is my custom lighting solution that is a lot faster then unity's inbuilt lighting. It has a lot of downside so its hard to release it.

    What I'll be releasing in a few days is volumetric lighting for unity's inbuilt lighting. I think Ill make another thread for this so its less confusing. The upside is it works out of the box. but it just volumetric lighting, it doesn't have any of the nice shadows like my custom lighting has, or the massive speed improvements.

    I ran into some bugs in the closed beta. I think it will set me back a couple of days unfortunately. Nothing major, I just need to do some more testing on every platform to make sure it all works.
     
  36. moure

    moure

    Joined:
    Aug 18, 2013
    Posts:
    184
    Thanks for the updates @Lexie , be sure to link the new thread also here so that we can follow the development ;)
     
    Seith likes this.
  37. Olafson

    Olafson

    Joined:
    Aug 7, 2012
    Posts:
    252
    I am still looking forward to your custom lightning in the future.
    Lighting is a major part of any game, so even if it has some drawbacks, I bet people can live with them.

    Right now unitys lighting just looks S*** and performs bad.
     
  38. ksam2

    ksam2

    Joined:
    Apr 28, 2012
    Posts:
    1,014
    Thanks man, it's really cool, can't wait to see this release. are you going to release this for free?
     
  39. HolyFireGames

    HolyFireGames

    Joined:
    Apr 23, 2014
    Posts:
    118
    Could you post a few of the downsides that your custom lighting solution has to see if those would be major issues for other developers? I'm guessing some of us wouldn't mind the downsides if the upsides outweighed them. Thanks!
     
  40. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    Deferred only, doesn't interface with light probes or enlighten, no lod support, Lights will not light transparency (might be able to fix this). Every shadow caster needs a script on it to cast shadows. 20-80 mb of VRAM is recommended to put aside for the system to work at full capacity. You can assign zero VRAM, but then its only slightly faster then unity's inbuilt lighting.
     
    Last edited: Apr 16, 2016
  41. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    I got a little delayed on the volumetric plug-in for unity's inbuilt lighting. Doing one last beta test this weekend to make sure its running smoothly.

    Note: This is volumetric lighting for unity's built in lighting. This is not the same as my custom lighting engine.
    Here is a demo scene with it running. Its a pretty un-optimized scene, I should just buy a stock scene...

    All the lights in the tree area are shadow casting lights, These are the most expensive to render.
    If you look behind you there are 54 none shadow casting lights. They are really cheap to render in my system.
    This was one of the main reasons I was pushing for quarter or half resolution with really good upsampling. I think I've achieved the best I can with out resorting to down sampling the camera's normal buffer as well.

    There are a few settings for upsampling, its generally a fight between signal loss with sharp rays or blurred rays with less signal loss. The settings in this build are blurred rays as I prefer that over signal loss.

    This build is using DX9. I still need to optimize my DX11 shaders (it all works, its just a little slower).

    Right now all the lights simulate Mia scattering and light extinction, Everything is adjustable per camera or per light. I really need to implement Rayleigh scattering to get correct lighting of really distant objects and simulate a skybox.

    Let me know if anyone has any issues running this demo.
     
    Last edited: Apr 16, 2016
    ksam2 likes this.
  42. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,132
    Maybe you could use Unity's courtyard scene, but strip it down a bit so it isn't such a huge download for everyone.
     
    one_one and Elecman like this.
  43. benoneal

    benoneal

    Joined:
    Apr 15, 2016
    Posts:
    18
    Hi Lexie,

    TBH, after playing around with it, I feel that Michal_'s volumetric lighting solution produces phenomenal results, is very performant, and is open source. I probably wouldn't personally be prepared to pay for your volumetric lighting solution, after having played with the downloadable demo. I'm not at all implying that what you've made isn't fantastic and valuable, just that in comparison to other options, it's not viable for me personally.

    On the other hand, the shadowing you showed off in the beginning of the thread is unmatched by anything else I've found, and would be incredibly valuable for my game (it uses loads of dynamic lights, in a fully procedural world filled with destructible props, ruling out unity's light probes and enlighten entirely). If you were to release that asset, I wouldn't hesitate to purchase it, even in consideration of the downsides you've mentioned.
     
  44. Razmot

    Razmot

    Joined:
    Apr 27, 2013
    Posts:
    292
    If I was working at unity I would seriously evaluate granting some source code access to @Michal_ and/or @Lexie !

    Especially nowadays with epic giving source code access to ... everyone !

    Oh btw, there is a market for a "procedural games lighting solution" asset. On demand baked GI, optimised dynamic but non-movable lights ...
     
  45. ksam2

    ksam2

    Joined:
    Apr 28, 2012
    Posts:
    1,014
    Hi Lexie, any new on this project? really like what you're doing.
     
    MyGenericUsername likes this.
  46. MyGenericUsername

    MyGenericUsername

    Joined:
    Sep 6, 2014
    Posts:
    95
    Yeah... Any news on how things are coming?
     
  47. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    You must have a pretty good GPU or only rendering 1-2 lights cause I've tried Michal_ solution and its unusable on my 970m GPU when rendering more then a few lights. The half resolutions helped but the artifacts it produced were unacceptable for me. Just remember your users wont all have good GPU's. 10-15$ for 2x+ the performance isn't asking much.


    Sorry I had family in town and then proceeded to get really sick. My last beta is finished, I'm sending it to end night games (the forest) to test it in a larger world setting. if everything comes back ok then ill release shortly after. Sorry for the wait.
     
    Last edited: Apr 27, 2016
  48. IronDuke

    IronDuke

    Joined:
    May 13, 2014
    Posts:
    132
  49. Lexie

    Lexie

    Joined:
    Dec 7, 2012
    Posts:
    646
    My guess on how its implemented it wouldn't be too hard to add support for it. It would actually mean you could do most of the volumetric lighting in one raymarch pass by taking advantage of the voxelized light data that the GI uses.

    So no, not out of the box, but support could be added pretty easily if its the method of GI i think it is.
     
    IronDuke likes this.
  50. IronDuke

    IronDuke

    Joined:
    May 13, 2014
    Posts:
    132
    Nice! You'll have to give that a shot at some point.:D I probably won't be needing it for a year or so, but I'm just researching what I could put in the game I'm planning.
    Looking forward to seeing how that turns out. In the meantime, back to developing me lousy pinball game.

    --IronDuke
     
unityunity