[Next-Gen Soft-Shadows 2] Sophisticated dynamic penumbra Shadows for Unity

@arnoob,
Normally, you can access the shadow map itself but in the final (rendering) stage after everything have been set to be rendered by the rendering loop which isn't that much of useful except for rendering shadows. You can intercept the shadowmap for individual lights to do something with your light depth map based effects but still you don't know which lights will render things in the final render stage as you don't have that information. Not just light indexes but also no control of when things get rendered. Also the current system does fragment video memory a lot and there's plenty of render texture swapping which isn't good at all. Hence the need for SRP.
[EDIT] HD loop is just an example of how complex a renderer can be constructed with SRP, you can go beyond that or the other way around, go ultra simple with a set of lights and ultra simplistic rendering it will run glazing faster than any available renderer due to it's specific nature.

 

I see, it seems indeed very tedious.

Let me just tell you what I was thinking aobut, maybe it would interest you or give you some ideas.

I am currently trying to figure out a way to have a global lighting that wouldn't be to bland for a procedurally generated game that I am making. The main thing I want is to have light from the sky when outside, and interiors that stay dark. (Kind of actual ambient occlusion, not GI).

In order to do that, I was thinking about having a directional light, that would sample the static geometry (that I would isolate by putting it in a specific layer) from a different angle each frame. This way I would be able to have a somewhat diffuse ambient occlusion, without killing the drawcall count by sampling the scene by all the angles at once every frame.

I don't know, if my idea is properly put in word...

Basically I was wondering if it would be possible to create a script that would work like reflections probes where we could ask it to either update shadows every frame, or only at one moment by script.

It would also be very useful point and spotlights. Imagine an interior scene like a mall, with a lot of neons on the roof. You don't actually need the moving geometry to cast shadows, as it would be barely noticeable, bu at the same time you don't want the outside of the mall being lit by the inner neons because of light bleeding.

Do you think that it is something that can be possible? and if so, would it be possible without the new rendering pipeline?

 

@arnoob
Besides the pre-baking static shadowmaps and updating the dynamic shadowmap part when objects moves (if any) there's the dynamic culling of lights/objects (completly occluded or out of the frustum view) that is already in place btw. With scriptable render pipeline you got access to the culling data and you can set it the way you want.
As for GI I'm not sure to understand your idea. You want a skylight that will get occluded with shadows? If so then yes it's possible.
All in all, both of those things you described are possible and a lot more with SRPs.

 

@arnoob you might find this interesting

 

I was pretty sure it would be a matter of time someone add a real-time raytracing API, Microsoft and nVIDIA just did that:
https://www.unrealengine.com/en-US/...microsoft-s-directx-raytracing-and-nvidia-rtx

I'm going to take a closer look as this goes exactly with my ideas of raytraced shadows that I'm working on with NGSS 2.0.
The future is bright!
[EDIT]
Before everyone get crazy about, this API won't replace entirely rasterizers, it will mostly be used for secondary rays (such as reflection, shadows, AO, GI, etc).

 

I've been waiting for someone here to talk about nVIDIA RTX. Hope it works out!

 

I am really skeptical about this whole DSX stuff... I mean, re-rendering the scene (even in an optimized variant including only major objects) and then use pathtracing for some effects? This'll slow down rendering a lot with its complexity, no matter how you look at it. So much performance and memory just utterly wasted, ouch.

 

@Kolysisan,
Hence why I said "Before everyone get crazy". Still I think there's good use for secondary rays (if the implementation is done the way I think). But before pronouncing myself into this "mini-hype" I'll take a close/deep look and will give my conclusions about it.

 

More on Microsoft DXR,
Looks like we'll getting new types of shaders to handle ray processing with DXR, such as ray-generation, closest-hit, any-hit, and miss shaders. But the scene data needs to be set in a way for the GPU to properly do the ray traversal. Microsoft call this process the acceleration structure.
In other words, it won't be done only with shaders. Scene data needs to be set (and/or send)to the GPU in a specific way for optimal real-time raytracing. So an in-engine initial integration must be done first (the layout of the scene data) and then use ray shaders to get information outside the screen that is layered out in a specific way for optimal GPU traversal.
So my conclusion is, it won't replace rasterization, it will be used for secondary rays such as shadows, reflections, AO and GI. But hey, it will look better than any of those rasterized effects approximations.

 

I'm interested in seeing how Unity will support DXR. I'm guessing it'll be in the HDRP, which may be part of the reason why we've been told it's at least a year out before it'll be considered final. That's at least my hope/justification haha.

 

i tried to open your example scene, but for some reason the shadow became pixel
why is this happen? thanks

 

@williamdarma,
Enable Cascaded Shadows in your project Graphics settings.

 

@chiapet1021
To the look of it, I believe it will be something similar to current Compute Shaders, it requires some in-engine integration, rendering and scene graph wise, How exactly it needs to be integrated, it remains to be seen. I'm looking for more info about that. If no engine modification is required, I will start experimenting with that asap.

 

I got an early version of DXR SDK, I'm setting up a machine to start doing some experiments with. It will possibly fail but at least getting my hands dirty in deep with the API would be actually good. I already got an idea how the API works and it's fairly simple. As I expected, it works similar to a Compute Shader (but without the trading groups). Also you have 5 types of specialized ray shaders eg, closest hit, miss hit, any hit, intersection and ray generation. You dont need to use them all, for example to do area-like soft-shadows you only need closest hit (the far from surface = the softer the shading will be). This is exactly how I do soft-shadows with Signed Distance Fields in NGSS 2.0, only that It will work with full scene geometry (not just with primitives forms).
Application requires what Microsoft calls "Accelerated Structure" for optimal ray traversal. So whatever comes out of my experiments with the SDK, it won't be running within Unity at least until Unity integrates DXR which Microsoft said has plans to Integrate.

 

So you found a way to create signed distance fields? Will it be possible to intercept it from NGSS 2.0 in order to use it for other custom effects?

 

Please report the results of you're experiments with the DXR SDK here, if you are so inclined (!)

 

@arnoob
Yeah, Primitive Shadows in NGSS2.0 are based on "Signed Distance Fields" and yes it can be used for other effects. I've demoed a hacky Ambient Occlusion with directionality.

@buttmatrix
Will do.

 

I just purchased the asset. I use only one directional light, added to my directional light, it is still old pixelated unity shadows stays. Do I need to do that local setup for directional lights as well ?

Regards.

 

@MoribiboMT
No, if you just need Directional shadows, only the NGSS_Directional component needs to be added to the light. No need to restart your project neither.
Make sure that Cascaded Shadows are enabled in your project Graphics Settings.

 

Thank you.

If I do not use your asset, my shadows are pixelated, blocky. If I use your asset shadows become, softer, much more better looking. However I want well defined hard shadows. I tested your scenes, I tweaked settings, shadows are really good looking the way I wanted.

Do you think my own meshes have problems ? which may cause blocky, not well defined hard shadows.

Regards.

 

I don't think it works that way. Your objects should always be fine, it's more a matter of how much resolution you're getting on your shadows, and that in turn is related to shadow distance settings. Or at least that's how it's been in Unity for a long time; not sure if that's changed any in recent versions.

Try this. In plain Unity, in any scene - it could be just a primitive sphere or cube on a plane - set up a directional light and bring up the shadow distance settings in the inspector. Play with the distance. You'll see that as you make the distance smaller you have sharper shadow resolution, and as you make the distance longer, the resolution drops and you start to get pixelation. In fact, the pixelation can start to dance or jitter, which is really annoying.

NGSS does a lot to fix that sort of thing.

 

@MoribitoMT
Shadowmapping visuals and quality are only scene dependent if you are scaling your objects too big or too small. NGSS is calibrated with default Unity units (1Unity unit = 1metter). In the case of NGSS PCSS algorithm, it produces similar soft-ness as you'll see in real-life with direct area-light. At least that's how I tried to calibrate it by default.
So make sure your scene scale is right, other than that, check out what hopeful suggested (shadow distance, resolution, etc).
I'm investing a lot of research with raytraced-like algorithms and newer shadowmapping optimizations, NGSS 2.0 will provide even better and faster soft-shadows.
If you still need help to tweak your shadows, try to provide some images about your scene objects size/lighting/shadows setup and I'll be glad to help.

 

Hi, after installing ngss in project I have all shadows flickering and jittering. Done everything as it's written in documentation. What might be the problem?
Ngss itself is working, it does smooth the shadows, but they jittering in play mode.
Unity 5.6.5
Directional light
Alex

 

@Morseliot
Turn on Cascaded Shadows in your Graphics Settings?
And make sure your resolution is high and your shadow distance is small. Oh and try StableFit projection in your Project Quality panel.

 

Problem solved, it was Camera near plane clip reset to 0.01 it makes all shadows jitter in scene. Sorry for bothering you.
Awesome asset!

 

If what you are looking for are just Directional Shadows you don't need to install the cginc file, neither delete de ShaderCached folder. Only add the NGSS_Directional component to your dir light.
Can you post images of your project Graphics, Player (renderers), Quality and Light settings? Also what's your project target platform?

 

Target platform is only pc for current project, previous project is targeting on xbox one soon, but it's not using directional light. I've realized that it's also working with script, but as I'm using same version of unity for my first project I need to keep shadow library in there. Here is the project http://store.steampowered.com/app/389860/Far_Out/

 

NGSS 1.8.5p1 just got submitted to the Store for review. It's a quick patch release and adds compatibility with Unity 2017.2.2.
If you are using a different version than 2017.2.2 you don't need to upgrade to 1.8.5p1 as 1.8.5 already works with all those versions (including 2017.4 & 2018+).

 

NGSS 1.8.5p1 is live on the store. Down load it only if you are using Unity 2017.2.2. Skip it if not.

Future of NGSS:
I'm researching image denoising algorithms for a potential rise of quality in shadowmapping without any additional cost. In fact, if it works the way I'm thinking, I will be able to reduce the amount of samplings (shadow rays) on penumbra regions while rising penumbra size for bigger area soft-shadows like the ones you saw in the Unreal Engine Raytraced demo. Of course with shadowmaps you still have the classic artifacts that needs to be hidden with bias, soft-ness and/or addition of ContactShadows.
NGSS PCSS already do a great job at providing area-like realistic soft-shadows but I want to bring it beyond what it currently provides (bigger and wider penumbra size, noise-free, faster). First there's the SM5 intrinsic functions with SAT optimizations that will brings 2x-4x speeds ups for free, then screen-space accumulation and denoising filtering for all shadowmaps.
If my ray-traced like experiments turns out to be too good to be true (or to put it out "too slow for current hardware") at least bring current shadowmaping quality to a whole new level.
Secondly, there's Primitive Shadows that are primary target for LW pipeline with further improvements to the HD pipeline with Compute Shaders parallel acceleration and a static mesh Distance Fields volume baking tool. Primitive Shadows are planned for initial version of the upcoming NGSS system and on LW pipeline.
The Distance Field shadows for HD won't come on initial NGSS 2.0 but it's being worked on.
Also planned for initial release are LW hemispherical omni directional shadows, it will provide 180 degrees of shadows with the same cost of one spot shadows. You can combine two of those to produce a point shadows but don't require the scene to be rendered 6 times, only once or twice (180/360).
Lastly, LW and HD NGSS shadows will become separate products as both provides very different needs, techniques and targets different platforms/pipelines. The price will be around 20$ for each package. I will also offer a life-time discount for current NGSS users of 25% on both packages for every major release.
If I'm not mistaken Unity plans to launch LW first then HD pipeline later, I will also follow that plan for NGSS HD/LW development features.
PS: NGSS 1.x will remain supported with patch updates till current pipeline gets deprecated.
Questions?

 

Just wanted to say I love what you're doing and am glad to have something that helps with such a chronic sore spot in Unity.

 

Preliminary tests on image denoise algorithms using Matlab:


So you probably see where I'm going wit this?

 

Bought this looks great. For some reason NGSS_ContactShadows turns itself off at run time and wont let me put it back on. And NGSS_Directional on my sun doesnt turn itself off, but I dont actually see any shadows at all, any ideas? Im on Unity 2018.1.0b13 maybe not supported yet.

 

@radiantboy
Unity 2018 is already supported yes.
What are your project player settings, graphics, quality, target API and platform?

 

hey, im targetting high end pc. settings here https://www.dropbox.com/s/56oep2vlpci7cn3/Snap 2018-04-01 at 13.58.13.png?dl=0

Just noticed there are shadows, just EXTREMELY soft (can barely see) and they seem to come from different angles unity ones did.

 

@radiantboy
In your Quality settings I see "Hard-Shadows". Maybe you should change it to soft-shadows?
I would need more information about how your project is setup. Lights, Graphics settings, player settings with target rendering APIs. But to the look of it, sounds to be an incorrect project setup
You can always recreate the same issue in a small empty project and send the package to the support email provided in the documentation.

 

thanks I tried soft, doesnt help its as if the shadows arent casting as much, I have a moving sun with unity shadows I normally see them casted all over and moving as the sun does with this I have only seen shadows once. Using realtime GI, nothing really special. I just saw shadows for the second time after waiting for my sun to come back around, but theyre definitely not casting at the same angles unity was though they definitely appear better quality, do they have problems casting on reflective surfaces? What other settings could be wrong?

 

@radiantboy
Can you provide a small project reproducing that issue?

 

I have an odd issue that might be my own fault, but maybe you can help. In a brand new project everything looks fine with point lights. In my existing project the shadows only render when they are within about a metre of the camera (this includes the scene view), they stop rendering (fade out) as the camera moves away from them. Do you think this is related to NGSS, or something I have done in my project (the project existed before I got the plugin, but I followed the instructions)?

In the point light sample scene, for example, I have to get the camera right beside the sphere before the shadow renders.

 

@Zillus
It's unrelated to NGSS yes. Check your Project Quality settings, make sure you have more than one light (at least 10) or Unity will disable it at distance. Also check your light mode (Light settings) and your light render mode (Light component).

 

NGSS - Denoiser algorithm for shadows. Using only 8 samplers per pixel for shadowmap (extreme fast) and denoiser filter running at screen space (denoising only shadows):

Preserving shadow casters edges:

As you can see using few samplers on noisy shadows can be denoised aggressively and provide pleasant soft and larget kernel shadows. This thing is fast!

PS: The algorithm can be used to denoise RGB images too, not just shadows!

[EDIT] Youtube video quality is garbage, here's a better version of the video (don't mind the music and please delete the video after watching): Download.

 

New test of NGSS Denoising algorithm using only 4 samplers per pixel (default Unity soft-shadows does 49 samplers per pixel). Destroying Unity default shadows kernel size and rendering speed with as little as 4spp!

HD video cuss: Download.
Close-up Image:

PS: Can be used on other noisy effects such as raytraced thingies, GI, AO, rgb images, etc.

 

hello again @tatoforever
I have a question about NGSS.. I used some time ago and now I want to reuse it into my project:
I read the manual but is not completely clear:
I wish basically 2 things:

1. a soft shadow faster than unity soft shadow;
2. if possible, faster and better

I have the sun directional light (other spotlights with no shadows)

• I added the NGSS_Directional script to my light
• I set hard shadow for the light

is this correct? I ask you 'cause I can't find big improvements..

thanks in advance

 

Looks great!

But the video kind of highlighted something in the first few seconds I've been wondering about for awhile but keep forgetting to ask:



There's always a very harsh transition between shadow cascades, and as the camera moves it's very noticeable and jarring. Do you think there will ever be a way to smooth out this transition? (or is there one already I don't know about?)

 

@ghiboz
Hi, first make sure you have the latest version of NGSS (1.8.5). The steps you are doing are fine. What are the improvements you are expecting and what are the ones you are having?

 

@ceebeee
This is unavoidable with current renderer as Unity forces a shadows culling in StableFit projection that I can't turn off and f*cks objects at cascade boundaries so I had to came up with an intermediary solution. This is not a problem with SRP, only with default black boxed renderer.
Btw, the image you posted is Unity cascaded shadows, not NGSS cascaded shadows (which transitions are less visible).

 

but you can play with the shadow cascade settings to help relieve this artifact. In many cases, NGSS with two cascades is perfectly sufficient; sometimes even none!

 

Wow, the new Denoiser's incredible.

 

I hope more framerate.... but at the moment the fps are similar: ngss with hard shadows and unity soft shadows..
what parameter did you suggest to tweak to increase the framerate?
and also a quality config setting

 

@ghiboz
It's hard to have better looking shadows with more performance. Current version can guarantee better looking shadows with similar performances (and that depends on your target platform). In mobile better shadows means less performance due to hardware restriction.
That will change in upcoming NGSS version with raytraced distance field shadows.
PS: To increase framerate, lower the sampling amount and disable PCSS.

 

Already getting ~2X perf improvement similar visuals but I can bring it a bit further!
The denoiser algorithm does a bilateral kernel salt and pepper removal pass with 15 taps + 8 (4 blocker search + 4 for PCF). Even with 32 taps, the results are blazing fast because it gets parallelized better in the GPU (as zero dynamic branching is applied on the denoiser algorithm) even with the aggressive early bailing out steps.
So, instead of bruteforcing the PCSS filter with tons of taps and dynamic branching, only do a tiny 4 taps and let the denoiser do the rest of the job.
I'm investing lot of time researching raytraced and raymatched shadows but shadowmapping is not dead (not yet).