Search Unity

[WIP] SE Screen Space Shadows

Discussion in 'Works In Progress' started by sonicether, Nov 19, 2016.

  1. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hello, everyone. I'd like to share my work-in-progress implementation of screen-space raytraced shadows in Unity.

    The name is pretty self-explanatory. The screen-space depth buffer is traced against in order to provide sharp and detailed contact shadows and shadows from objects not rendered to the shadow map. In addition to providing better contact and self-shadows, this can be very effective for fields of grass or small detail objects like rocks where rendering these objects to the shadow map would be very costly.

    Here are some comparison images of the effect (I recommend opening them in a new tab and switching between them to see the difference more easily).

















    This implementation works by using command buffers to render screen-space shadows to an intermediate buffer and then blend it with the screen-space shadow mask for the directional light. As a result, both diffuse light and specular highlights are occluded properly by the screen-space shadows. Since point and spot lights don't have an entry point for blending with a mask like this in Unity, unless I find another way, this will only work for directional lights.

    And, of course, any information not directly visible will not contribute to screen-space shadows.

    I still have a few things to finish, I will provide performance details when this implementation nears completion.
     
  2. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,508
    Looks very nice and makes a better more meaningful contribution than SSAO, so if it's cheaper than Unity's SSAO I don't see a reason why I wouldn't be interested!
     
  3. Hikiko66

    Hikiko66

    Joined:
    May 5, 2013
    Posts:
    1,021
    That grass looks delicious enough to eat.
     
  4. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,508
    It's another one of those "why ain't this baseline in Unity FFS" threads. Wondering how it's done - does it result in drawing all those meshes again or just cleverly sampling depth buffer and calculating from light position?
     
  5. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Thanks, guys!

    Performance as of now, in that grassy scene, is about the same as Unity's "Screen Space Ambient Occlusion" built-in image effect at its default settings. I think I can squeeze a bit more out of it with the things I have planned.

    Yep, it simply uses the depth buffer, there's no additional rendering of objects. Basically, in view-space, a ray is traced towards the light source and tested for collisions with the depth buffer--a similar principle to screen-space reflections. Because of this, the basic technique is independent of scene complexity, since all pixels are performing the same calculations.
     
  6. moure

    moure

    Joined:
    Aug 18, 2013
    Posts:
    184
    Looks like the UE contact shadows that were just released, i was wondering if there was something like it in unity :D
     
    RB_lashman likes this.
  7. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,508
    I'd love to be using @tomaszek Uber and this I think. Even if only directional it'll be lush.

    The link to UE4's implementation is very impressive, particularly the point light shadowing. I figure Unity needs scriptable render loops?
     
    RB_lashman likes this.
  8. Vagabond_

    Vagabond_

    Joined:
    Aug 26, 2014
    Posts:
    943
    Really nice looking shadows :)
     
    RB_lashman likes this.
  9. claudiubarsan

    claudiubarsan

    Joined:
    Jan 5, 2016
    Posts:
    7
    Very nice indeed! Will this work fine with SEGI ?
     
    RB_lashman likes this.
  10. LennartJohansen

    LennartJohansen

    Joined:
    Dec 1, 2014
    Posts:
    2,366
    looks good
     
    RB_lashman likes this.
  11. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Yep. I would need some sort of way to inject screen-space shadows into the point light shading before it's blended with the rest of the lighting buffer.

    Yep! I'm actually using SEGI in those Viking Village screenshots.
     
    TooManySugar and RB_lashman like this.
  12. claudiubarsan

    claudiubarsan

    Joined:
    Jan 5, 2016
    Posts:
    7
    That is awesome! I'm really impressed with your work. Looking forward to seeing it released on the Assets Store.
     
    RB_lashman and sonicether like this.
  13. eskovas

    eskovas

    Joined:
    Dec 2, 2009
    Posts:
    1,344
    :eek: That looks absolutely fantastic @sonicether
    And once again, sonic comes with an awesome new asset. That grass looks really good!
    Looking forward to more info and getting this.
     
    RB_lashman likes this.
  14. GoGoGadget

    GoGoGadget

    Joined:
    Sep 23, 2013
    Posts:
    690
    Screenshots look nice (although the grasses shadowed side is maybe a bit too dark without translucency) but screenshots do generally look nice for a screen-space effect :)

    How are you dealing with the issue of pop-in of shadows from items not currently on the screen?
     
    RB_lashman likes this.
  15. buttmatrix

    buttmatrix

    Joined:
    Mar 23, 2015
    Posts:
    609
    The screenshot of the grassy hills looks truly impressive. I'm interested in this asset, but I'm going to have to sincerely request that SESSAO owners be offered a discount for this release. Moreover, it would be nice to see similar screenshots without any additional post-processing added to compare against, e.g. only SE screen space shadows vs. no image effects.
     
    Arkade and RB_lashman like this.
  16. TooManySugar

    TooManySugar

    Joined:
    Aug 2, 2015
    Posts:
    864
    Will this require unity 5.x?
     
    RB_lashman likes this.
  17. HenriqueGomez

    HenriqueGomez

    Joined:
    Dec 13, 2012
    Posts:
    63
    Cody, how're doing? Sorry for your family member that's battling against cancer. I'll be praying for him/her.
    Just wanted to ask if its possible to add in Unity a version of Nvidia's HFTS? I'd love to see that feature. ^^
     
    RB_lashman likes this.
  18. mk1978

    mk1978

    Joined:
    Dec 27, 2009
    Posts:
    276
    Looks extremely nice! Finally there are solutions coming that will make Unity grass look comparable to UE.
     
    TooManySugar and RB_lashman like this.
  19. jdraper3

    jdraper3

    Joined:
    May 28, 2015
    Posts:
    117
    Literally have no words for how great this looks :)
     
    RB_lashman likes this.
  20. paxilon

    paxilon

    Joined:
    Aug 9, 2016
    Posts:
    25
    Absolutly stunning images! :eek: I will definitively buy it!
     
    RB_lashman likes this.
  21. IronDuke

    IronDuke

    Joined:
    May 13, 2014
    Posts:
    132
    HULK WANT

    --IronDuke
     
    TooManySugar and RB_lashman like this.
  22. zenGarden

    zenGarden

    Joined:
    Mar 30, 2013
    Posts:
    4,538
    But it's out of the box in UE4 :D
     
    nipoco likes this.
  23. Knightmore

    Knightmore

    Joined:
    May 11, 2012
    Posts:
    226
    Well UE4 has a lot of stuff we would want to have out of the box... Epic knows what their developers want. ("We've also introduced a new automatic LOD generation feature for static meshes that does not require a third-party library.")

    So I am really happy, that we have at least some amazing developers like sonicether in our community when the Unity Team can't get stuff done.
     
    zenGarden, RB_lashman and sonicether like this.
  24. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Unfortunately with screen-space effects there's really no way to get around this, though things can be tweaked to minimize the visual impact (like tracing z-thickness and a parameter that adjusts how smoothly or harshly samples fade out as they get to the screen edges). I've found that generally these issues aren't very noticeable unless you're looking for them. If you've ever played Skyrim with ENB Series, you'll have an idea of what screen-space shadows can do for a game (particularly for the grass) and how generally unnoticeable the pop-in artifacts are.

    I'm looking into how to do this through the Asset Store. I think that's a great idea.

    Yep.

    I just took a closer look at HTFS, it looks like it's pretty hefty performance-wise (4-6 ms). I personally think that simple traced screen-space shadows are a better option since those milliseconds could be put to better use, like real-time GI ;)


    Oh, also, today I managed to speed up the tracing by about 2x! At high settings, it's about 70% the cost of Unity's Screen Space Ambient Occlusion at default settings, and at medium settings, it's about the cost of FXAA. It is noteworthy for me to point out that I haven't added a bilateral filter yet for smoothing out noise. Once that's in and enabled, it'll add to the rendering cost a little bit.

    I do highly recommend using temporal anti-aliasing with this effect, since it can really clean up edges and flickering from movement. I've also added an option to leverage TAA by randomizing the jittering per-frame which smooths out noise and artifacts without having to add blur, which keeps the shadows extra sharp. Don't worry though, I will add bilateral blur for those who dislike TAA.

    Here's without TAA


    And here's with TAA and jitter frame randomization
     
    Last edited: Nov 22, 2016
    Seneral, jcarpay, Knightmore and 3 others like this.
  25. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Just figured I'd drop a few more screenshots here. It works surprisingly well with distant objects!







    Also, here's a comparison with no image effects (no TAA either)

     
  26. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,508
    Faster the better.
     
    RB_lashman likes this.
  27. pcg

    pcg

    Joined:
    Nov 7, 2010
    Posts:
    234
    As with all your work this looks stunning.
    Assuming this will work ok for mac?
     
    RB_lashman likes this.
  28. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    I can't personally test on Mac, but I can make sure it works for OpenGL.

    Also, I just tried it out with UBER's z-write POM functionality, and the results are quite awesome.





    So, any shader that writes to depth will have appropriate screen-space shadows!
     
    elias_t, Martin_H, Lex4art and 12 others like this.
  29. Dhaiku

    Dhaiku

    Joined:
    Apr 27, 2012
    Posts:
    131
    what about transparent materials ? they have issues with depth and AO
     
    hopeful likes this.
  30. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    26,508
    I has suspected and hoped this would be the case and the results look very sweet.
     
    RB_lashman likes this.
  31. NOT_Lonely

    NOT_Lonely

    Joined:
    Feb 2, 2013
    Posts:
    424
    Wow! Nice looking shadows. Say goodbye to "Peter Pan" effect :) I'm waiting for this asset!
     
    RB_lashman likes this.
  32. AurimasBlazulionis

    AurimasBlazulionis

    Joined:
    Aug 13, 2013
    Posts:
    209
    I know this is not the most important thing at this stage, but could this effect be used to fake the way real life shadows work (the further from the shadow caster the shadow is, the smoother the shadow is) by sampling the distance or something similar? Even though, this fading is not really necessary since it does not quite work in off screen-space.
     
    Seneral and RB_lashman like this.
  33. gurayg

    gurayg

    Joined:
    Nov 28, 2013
    Posts:
    230
    Add me to the "Wow!" list...looking wonderful, can't wait.
    I know it is still WIP but wanted to ask about something.
    I noticed translucency/transparency issue in one of your screenshots.
    Looks like SS shadows effect translucency or transparency. It might be a problem with the grass shader but just wanted to point out.

    sessshadows5on_2.jpg


    Hope everything goes as you want!
     
    RB_lashman likes this.
  34. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Well, the screen-space shadows do a lot to harden contact shadows, so in that regard, they help with this effect. Combining screen-space shadows with PCSS would probably give some great results.

    That issue actually has nothing to do with the screen-space shadows, that's actually an issue with SEGI (the global illumination I was using in these images). It's just more apparent when the grass has self-shadows on it. I'll have to look into that!
     
    IronDuke, gurayg and RB_lashman like this.
  35. 8Infinite8

    8Infinite8

    Joined:
    Jul 27, 2013
    Posts:
    112
    Really excited for this. Any idea on release date? Will you be selling this on the Asset Store?
     
    RB_lashman and jdraper3 like this.
  36. IronDuke

    IronDuke

    Joined:
    May 13, 2014
    Posts:
    132
    This will be so perfect for tiny shadows in a spacecraft's cockpit, since the normal Unity shadows would be more used to let the ship cast a shadow on a moon 50,000km distant. When you have that big a distance, keeping good detail in the cockpit is a pain, even with shadow cascades and multiple cameras. Keep it up man! This is incredible! \o/

    --IronDuke
     
    FreakForFreedom and RB_lashman like this.
  37. Gekigengar

    Gekigengar

    Joined:
    Jan 20, 2013
    Posts:
    444
    Amazingly beautiful #_#
     
    RB_lashman likes this.
  38. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Yep!
     
    RB_lashman likes this.
  39. Luckymouse

    Luckymouse

    Joined:
    Jan 31, 2010
    Posts:
    481
    Hey sonic ether, does your screen space shadow support WebGL? Currently unity shadow in WebGL looks pretty bad.
     
    RB_lashman likes this.
  40. FPires

    FPires

    Joined:
    Jan 5, 2012
    Posts:
    151
    Looks fantastic. Great job as usual.
     
    RB_lashman likes this.
  41. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Hey everyone, I'm just wrapping things up and figured I'd give some comparison videos regarding noise, Temporal AA, and bilateral blur.

    As I've said before, I highly recommend Temporal Anti-Aliasing to improve the look of the screen-space shadows. Anyway, here are the comparison videos.

    No TAA or blur: http://www.sonicether.com/wp-content/uploads/2016/11/sesssnoblur.mp4
    TAA: http://www.sonicether.com/wp-content/uploads/2016/11/sessstemporalaa.mp4
    Bilateral Blur: http://www.sonicether.com/wp-content/uploads/2016/11/sesssbilateralblur.mp4

    TAA really helps with shimmering from aliasing and helps to filter out noise. Of course, the bilateral blur is still an option for those who would rather use that. However, since the bilateral blur itself takes some render time, I'd personally rather put that into tracing more samples for a cleaner result, since blur would arguably defeat the purpose of sharp contact shadows in some situations. Ultimately you'll have the choice as the user!

    Since Unity 5.5 was just released and in 5.5 the depth buffer is inverted, I'll have to make sure everything is working for both 5.5 and 5.4 and under, so that'll take some extra time, but I'm almost done. I'll keep you guys updated.
     
    ftejada, pcg, TerraUnity and 7 others like this.
  42. semaphore

    semaphore

    Joined:
    Mar 21, 2011
    Posts:
    49
    Hello Sonicether,

    What Temporal Anti-Aliasing solution are you using?
     
    RB_lashman likes this.
  43. tomaszek

    tomaszek

    Joined:
    Jun 18, 2009
    Posts:
    3,777
    How would we distinct between realtime shadows and contact ones - can they be mixed properly ? On your bottom most screenshot it's hard to say wheter we see contact screenspace shadows or self-shadows (as an option in POM when we _don't_ write into depth or realtime shadows generated by writing into depth on shadow caster). I'm wondering, because with UBER's POM z-write I've got shadow caster passes. To save GPU processing time (POM is expensive), would be fine to write into depth only for main buffer, use contact shadows for slef-shadowing, still having main object correctly casting shadows. As it casts realtime shadows on itself - what's the result then ?

    Tom

    P.S. Suprisingly I found that with inverted z-buffer U5.5 brings POM with depth write works out of the box in UBER (phew... was preparing myself for hard fight).
     
    RB_lashman likes this.
  44. Zomby138

    Zomby138

    Joined:
    Nov 3, 2009
    Posts:
    652
    Does this only work with directional lights, or point and spot lights also?
     
    RB_lashman likes this.
  45. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    I'm using a custom solution that I've been working on in my spare time, but Unity's new TAA should work fine.

    So, I'm not quite sure what you're asking, but those screenshots show the z-write POM material with modified shadow caster depth. The first image of each pair shows the result without screen-space shadows, and the second shows the result with screen-space shadows. With the way I had things set up, the directional light shadow resolution was such that it couldn't properly provide self-shadowing on that material alone, which is where screen-space shadows come in to provide more detail.

    If modifying the shadow caster depth was disabled, I think you would run into issues because the depth modification in the main pass would have some areas of the surface "behind" the shadow map depth which could cast incorrect self-shadows.

    It only works with directional lights, since there is no entry point for me to combine a screen-space shadow result before adding a particular point or spot light to the light buffer.
     
    RB_lashman likes this.
  46. semaphore

    semaphore

    Joined:
    Mar 21, 2011
    Posts:
    49
    Any ETA when screenspace shadows will be submitted to asset store?

    Also, would you consider selling your custom TAA on the asset store?
     
    buttmatrix and RB_lashman like this.
  47. tomaszek

    tomaszek

    Joined:
    Jun 18, 2009
    Posts:
    3,777
    I was rather wondering about realisation of screen-space shadows mixed together with realtime shadows. For UBER objects that has POM z-write shadows produced by Unity realtime shadow casting/receiving has lo-freq. So - your screenshot shown screenspace hi-freq shadows. You see no real chance to skip expensive POM calulations on shadow caster still could you tell something more on mixing together realtime shadows with screenspace - do you deal with collected shadows texture to add your shadows ? And - most interesting for me - performance. We've got pretty GPU intensive vegetation in our game. Skipping shadow casting on it would be huge benefit as far as your solution will be faster and good looking. I'm a bit worried about necessity for filtering. There are still no perfect TAA solution available, but I've got hope on incoming Livenda's solution.

    Tom
     
    jdraper3, Seneral and RB_lashman like this.
  48. Vagabond_

    Vagabond_

    Joined:
    Aug 26, 2014
    Posts:
    943
    Not that much on the topic but isn't Livenda's CTAA solution available at half price as "Launch Event" for a long time now only trough their web site... I was hoping to see CTAA on the store as well but seems it won't get available there...

    P.S. Excited to see "SE Screen Space Shadows" in action !
     
    RB_lashman likes this.
  49. sonicether

    sonicether

    Joined:
    Jan 12, 2013
    Posts:
    265
    Should be within the next few days.

    Sure, I'd consider selling my TAA. It does use a novel technique for handling recently unoccluded pixels (no more streaks behind objects on detailed surfaces). There are a few areas that could improve still, though.

    So, after unity's realtime shadows are gathered into a screen-space buffer, my screen-space shadows are simply multiplied with unity's shadow buffer. My screen space shadows have an adjustable "fade out" to ensure that they blend well with the traditional shadows.
     
  50. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,070
    @sonicether - Any idea if this could possibly be a solution for casting shadows onto alpha-blended water? I imagine the answer is no, but I figure I should ask. :)
     
    RB_lashman likes this.
unityunity