[Next-Gen Soft-Shadows 2] Sophisticated dynamic penumbra Shadows for Unity

pcss shadows come from direct light. you would need capsule shadows for indirect lit areas where there is no direct light. at least from my understanding and how they are used in ue

 

@nbac,
Capsule Shadows is planned somewhere in the NGSS 2.X life cycle, a couple of things to keep in mind:
- Requires artists to insert capsules inside characters body. There's others way to mathematically represent shadowed shapes (sphere is less costly btw). Can use some ragdoll-like tool to automatically create primitives shadows (but I'm not too fan of this tbh)
- Having too many primitive shadows can be costly (can hit shader instructions if too many primitives)
- Direct capsule shadows can be computed in the light shadow pass. This is also what PCSS does with a wide shadows penumbra but capsule shadows are cheaper to compute
- Indirect however is more tricky, on top of initial capsules/spheres addition to objects, it also requires some sort of post-processing (similar to AO) as light is supposed to come from all directions

What else the future holds for NGSS?
- Raytraced shadows. Already started to experiment with directional and if your GPU can handle it, it's probably the best option for precise shadows covering the entire view
- Distance buffer. Unlike unreal Engine where has to be enabled in a per-mesh import way, it will be entirely generated dynamically. This will make PCSS, CHS, even AO/HBAO and many other shadowing/AO techniques cheaply free to compute as no Blocker-Search will have to be performed at light space and it's a screen space buffer computed using the screen depth/normal buffer, pretty much like a motion vector buffer but distance is again computed using raymatching techniques. Once done, you can use it for many things that requires distance.

All in all, I'm always experimenting with new shadowing techniques and what not.

 

NGSS - Demonstration of Primitive (Spherical) Shadows:

- Directional and Local Shadows are disabled
- No pre-rendering of casters or anything else
- No depth map or cookie or projection generated
- Primitive Shadows can be casted into any geometry
- Can be blended with other shadowmaps
- Can be computed light or screen-space
- No shadowmapping artifacts (z fight, acne, peter panning, etc) as there's no really any shadowmap
- Can render shadows to any distance as no depth/shadow map is needed
- No screen-space fetching thingies
- Only the mathematical representation of the primitive, in this case a sphere
- Other primitives shapes can be represented
- No multi-sampling to obtain soft-shadows, Only one fetch per frag
- Insanely fast for low end hardware such as Mobile and the likes
- Can work with Unity built-in lights
- Coming in the future to NGSS family

 

Wow, those look great!

 

Light range can be of any size, doesn't matter as there's no actual depth pixel to texel mapping (point light range is 100m):

 

Is there any limitations, except using primitives shapes, when can we expect to get our hands on it, because it looks amazing !!!

 

@danix,
Other than "add the primitive shapes yourself" to the scene there's no limitation as it's probably the fastest shadowing technique on the planet. Faster than hard shadowmapping it self (stress test):

Test with 100 point soft shadows. Notice how the range of the light don't change the quality of shadows and the FPS still over 2000! I can even go like 1000 meters point range shadows and it will remain accurate on distance because no actual depthmaps is used.
I'm pretty sure I can do 1024 point light shadows and it will remain almost identical. The slowdown is probably due to the 100 point light selected on Inspector. :V
As shown in the first video, primitive shapes are not rendered, they just contain geometrical data (positions, sizes) for the primitive shadows representation.
[EDIT]
Test with 360 point light shadows:



PS: I'm also researching shadows on raycasted non-primitive shapes.

 

This looks great, does this work with Unity Cloud Build?

Looks like you need to copy CG includes into the Unity directory?
I don't have that luxury when building with cloud though

Maybe someway to copy over in a post process build script?

 

@CDF
You can do copy/replace/etc files in a post-process build script. I'm pretty sure there should be a way as lot of people does post-processing stuff with their build (and plenty of them uses Unity Cloud services) but I'm not a Unity cloud user so It's hard to tell you the procedure with Unity cloud though. :/

 

Thanks, never accessed the Unity installation from cloud before, but I'm sure it's possible

 

Primitive Raycasted Spot - Point shadows:

No visual difference between spot or point shadows. They are also both equality fast even when you change the spot angle as again, shadows are raytraced.

[EDIT] Sorry, is not raytraced, it's raycasted!

 

How many primitives can be used in this way? Sphere, cube, ...?

 

@hopeful
Any primitive that has simple shapes: spheres, cubes, capsules, rectangles, cones, etc anything that can be recreated quickly and fast in the GPU it's a good candidate. Right now I've just done Spherical because it's the fastest one but I'm experimenting with other primitive shapes.
The good thing is that is not limited to screen space so anything outside the screen (even miles away) cast shadows into the screen without even being pre-rendered! The power of raycasting.
Also forgot to mention that primitive shadows:
- Can be casted into any geometry
- Can be blended with other shadows (depth map based, AO, you named)
- Can be computed in any space, is not limited to screen-space

I'll show some other videos with other shapes very soon.

PS: NGSS v1.8 submitted to the store.

 

@clagrens
If your shaders shadow files points to default CGInclude folder, it should work out of the box. I've never used Octane but I can do any custom shader modification to make it work with NGSS.
You have any errors, images, video or project that I can take a look at?

 

Folks,
NGSS v1.8.0 have been submitted to the Store. Features and changes:

- Now support Unity 5.6 (again) all way up to Unity 2018 (Default pipeline)
- Backward compatibility with SM3.0. Still requires at least DX11 API on Windows
- Full shadows support on Mobile (No more need to install AutoLight.cginc works as in Desktop)
- Cascade Blending for all shadows projections (Stable and CloseFit)
- Improved Directional noise filtering, static noise filtering now public
- Contact Shadows opacity improved (both contact shadows and directional shadows match opacity perfectly)
- Improvements to point light shadow bias
- Added compatibility to SM3 APIs on supported platforms. If your GPU can't support inline sampling, it will chose the SM3 path automatically. This means only PCF filtering as inline sampling is required for PCSS filtering
- Removed dependencies on AutoLight.cginc making NGSS compatible with every shader and framework on the Asset Store that points to default UnityShadowLibrary.cginc
- Refactored/Optimized all shadow code. Also added a couple of fallbacks for old APIs/Platforms where NGSS can't run properly
Important: Remove any old trace of NGSS before installing v1.8. The new version requires only the installation of NGSS UnityShadowLibrary.cginc file but you still need to remove the old ones!

PS: You can bomb support if you want 1.8.0 now.

 

very nice! so we are half way there with those raycasted primitives ha?

 

@nbac,
Yes sir! Glad Im done with NGSS 1 so i can focus 100% on NGSS 2 now! Well except for bugs (if someone finds any in 1.8).

On top of raycasted shadows, Im also working on raytraced shadows, so bye bye to contact shadows. The resulting raytraced shadows are spectacularly free of artifacts with the best possible soft shadows quality, similar to raycasted shadows i shown yesterday and works with a custom G buffer layout (thank you SRP). Is not as fast as raycasted shadows but the implementation im working on is around 50 % faster than cascaded shadow mapping while using half bandwidth!
No need to render anything in light space, no additional rendering pass, shadows are computed on the lighting pass. Also works with transparent geometry and operates exclusively on the screen.

 

Have you considered expanding your work to include support for particles receiving/casting shadows?

 

Do primitive shadows scale as well with hundreds or thousands of shadowcasting primitives as they do with hundreds of shadowcasting lights?

 

@brisingre,
The idea behind primitive shadows is raycasting, raymatching and distance fields.
The only limit will be shader instruction (SM level plays a big role in this). The most simple form is obviously spherical ones, capsules and spheres, those are the ones that will come first.
The number of primitives can be as high as your SM level instruction limit can handle (specially when computing the shadows directly on your shader).
Beyond that, occlusion culling system can cut off occluded primitives which will reduce the number. Also we can test if the primitive is within the range of any light and skip shadows entirely on the GPU.

 

If you have problems with baking the shadows with mixed mode and Shadowmask, replace these lines in UnityShadowLibrary.cginc file and delete your ShaderCache folder:

Line 802 with bellow:
fixed4 rawOcclusionMask = UNITY_SAMPLE_TEX2D_SAMPLER(unity_ShadowMask, unity_Lightmap, lightmapUV.xy);

Line 842 with bellow:
return UNITY_SAMPLE_TEX2D_SAMPLER(unity_ShadowMask, unity_Lightmap, lightmapUV.xy);

PS: I did not fixed this, just found the bug ^^

 

NGSS v1.8.1 is about to get submitted for review. Provides fixes for a couple of bugs. One of them spotted by @danix.

 

Thank you for the fast support as always!

 

Great work as always tato! Always great to see so much progress being made.

I noticed you mentioned AO utilizing other data you calculate, are you implying that you're working on including AO as a feature too ? Cause that would make a lot of sense to save more performance for people.

I found a neat paper that I thought you might be interested in, about a technique dubbed "Reverse Reprojection Caching" (whitepaper video), though reading your last few posts it sounds like you might already be working on a similar speed-up technique now with the distance buffer?

 

@Invertex
Thanks for that paper, there's always useful things and ideas I can get from research papers.

So, regarding the distance fields buffer and the possibility of AO. I'm interested more in high quality high frequency AO (not simple SSAO). And for this, a sort of distance Fields buffer might be necessary, maybe not, depending how things rolls out (explained down below).

What I'm working/researching right now for NGSS 2.x+:

//Techniques that improves rendering speed of current shadowmapping based shadows
- 4x speed up pass using specific hardware intrinsic functions (SM5 requirement). Done
- 4x speed up pass using SAT techniques. WIP
- Screen space shadowmap filtering for all lights in one shot. Researching/WIP

//Techniques that tremendously improves rendering speed and visual quality of shadows in real-time applications
- Primitive Shadows, using raycasting and analytical geometry for Direct and Indirect shadows. Researching/WIP
- Raytraced Shadows, using a custom G buffer Layout. Researching

And there's more. Ever heard of WSAO (world space ambient occlusion)? Me neither hehe. But the idea is that with primitive shadows I can compute true indirect shadows (WSAO) using the same analytical data (reconstructed primitive objects), almost for free. What's nice is that it can work with Default Renderer too but keep in mind that Primitive Shadows are a NGSS 2.x feature though.

I'm focusing right now more in raytraced and raycasted shadows because I believe it's the future of real-time shadows and the future it's now. The other options that I'm working are kinda appealing too as intermediary solutions to improve things up.

PS: I'll show some more videos of Raycasted shadows and new ones of Raytraced shadows soon.

 

Folks,
Version 1.8.2 submitted to the Store, fixes a rare compile bug with custom surface shaders.
If you are having this issue, email support we'll provide the package.
PS: Back to NGSS 2.0
Regards,

 

Have you ever applied to work at Unity? At this point, I don't see the downside... for everyone lol

 

@buttmatrix
I already got an offer to work at Unity in the Montreal area but I'm kinda busy with various projects. A game currently selling, one coming to consoles and of course NGSS.
But sure, it's an option that I'm keeping for the future.

 

That'

That's great to hear, and I certainly hope you consider it in the future if it's the right fit.

 

Or even if Unity could do what they've done with other great plugin authors, buy out their asset store plugin and then contract them to work on integrating the features. So Tato would still work on these features in his freetime but have the power to get changes made internally that are needed to support his wild ideas

 

I don't think this has worked out great for the previous devs that have done it. IIRC, the NGUI guy ended up leaving Unity after just a year or two and the GUI system he worked on with them turned out a whole lot worse than NGUI. Ended up not really being great for that developer or in the end for the users either. Better to remain independent, I think.

 

Anima2D is another example. It's officially in maintenance mode. The dev for it (also working for Unity) hinted that something to replace it was forthcoming, but it has been quite some time since he made that remark.

 

Folks,
If you'r getting this error with v1.8.1:

I got it fixed in v1.8.2 and it's already being reviewed for release. Ask support for it.

 

Hi,tatoforever,
UnityShadowLibrary got error with Toony colors pro2 (Custom toon shader :https://www.assetstore.unity3d.com/jp/#!/content/8105)
Shader will be pure white, and I got syntax error like this.

Spoiler: error

Shader error in 'Toony Colors Pro 2/Examples/Default/Subsurface Scattering': syntax error, unexpected ';' at token ";" type name expected at token ";" at /UnityShadowLibrary.cginc(63) (on d3d9)

Please teach me how to fix this.

 

@Piahouka,
That one got fixed in v1.8.2, email support in order to get v1.8.2.

 

I sent you Email with invoice number.
Thaanks！

 

I just wanted to say I'm really happy with the progress on this asset, it's come a long way since I started using it around 1.2ish. I was initially worried it was going the wrong way with 1.7 as I was getting atrocious 'peter panning' with spot lights and characters:



But as of 1.8.1 it's looking amazing:



So awesome work, keep it up!

 

Folks,
More of my findings and advancements with raytraced/raycasted shadows.

First step (data reconstruction):
- Can be computed in a screen space buffer to massively accelerate the process (requires access to G buffer data)
- Can be culled by the GPU in this step (primitives outside lights = skip)
- This buffer can be then be globally access by the entire pipeline

Second step (data profits):
- Raytraced shadows! Not just from Primitives but also with data on the G buffer
- Raytraced AO? (Maybe, depends of the quality of the data generated in the first step)
- Raytraced global illumination? (If raytraced AO is possible then GI too)
- Raytraced reflections? (Maybe, it can reflect data already on the screen correctly but will also reflect data outside the screen which is just a bunch of occluding primitives)

Third step (optional fun with that data):
- Can be combined with other shadows (depthmaps shadows/lightmaps/shadowmaks/etc)

First two steps can directly produce the shadows as an screen-space mask. Or it can be be decoded at light pass if you need it for volumetric rendering of lights and other effects.

The first step is basically a distance field buffer reconstruction with analytical geometry. I'm looking into other ways of efficiently generating this data. The ideal world will be static dinstant field data (3D volumes maybe) that is only generated once and then analytical primitives data for dynamic objects.

Transparency is not really an issue cause G buffer already take cares of material IDs

Primitive shadows can also be recreated In a forward-like renderer on the fly (if there's no prepass). Numbers of primitives will be small compared to the prepass solution because it has to compute culling and analytical geometry reconstruction on top of shading and if your lighting is complex, then the number of shadow casters will be limited. This is exactly how i did in the shown demos.

NGSS 1.8.5:
I added the possibility to use ContactShadows without directional shadows in 1.8.5:




Obviously you don't want to do that, but some people will find it just fine for their project (specially when combined with shadowmaks) and it's ultra cheap as no directional shadows is rendered.
PS: Also, v1.8.5 will have cumulative fixes (if you or me) finds something to fix.

[EDIT] NGSS v1.8.2 is already live on the Store. It fixes a compile bug with surface shaders.

 

@ceebeee
Looking good, do you have higher ress images of those pictures? They look tiny.

 

Super interested in how these might turn out. I mean, I'm interested in all of it, obviously, but these more so.

 

Yeah sorry I don't, they were just something I was showing a friend the other day.

 

I see you are researching both raycasted and raytraced shadows. Maybe it's too early, but I am confused about how this implementation works in the context of SRP.

 

@buttmatrix
Well, the techniques I'm researching takes advantage SRP flexibility as it allows me to layout and construct that data the way I want. Let's say I want to encode some world data into some free channel in one of the available MRTs. I can do it (not possible with current default renderer). But yeah the idea is to stay within the limits of provided renderers so it will work with whatever shaders/materials people create for those pipelines.
Obviously for raytracing to work properly I need to access data outside the screen and G buffer data resides inside the screen. That's where analytic equations and distant fields comes into play, to help reconstruct stuff outside the screen. Then, magic happens afterwards.

 

Folks,
I just want to confirm that on top of the new shadowmapping optimizations I'm moving forward with primitive shadows for NGSS 2.x. Some of the benefits and limitations of Primitive Shadows:

- Primitive shadows will be limited to... primitives ^^
- Bandwidth free, shadowmap artifacts free, area soft-shadows free, ambient occlusion free (in world space), raytraced (no resolution problems)
- Can work alongside with shadowmaps lightmaps and shadowmasks
- First implementation will probably be spheres and planes (hopefully more than that for initial release of primitive shadows)
- Limit will always be shader instructions count*1
- You will have to place primitives manually*2

1) There's gazillions ways to speed up and increase primitive shadows (g buffer, vertex shader, compute shader, etc).

2) I'm also looking for ways to generate distance fields for the entire scene so it will work with any geometrical form (not just signed distant functions) and you won't have to place occludes manually. Probably with a compute shader. Once this step is done it can be used to generate interesting raymatched effects with little cost and extreme quality such as area shadows, true ambient occlusion (with directionality), even reflections and GI. All working in world space, not just limited to the screen (except for data outside the screen with reflections and GI).
I need to research and experiment a bit more but it will either be runtime generated data or static pre-baked data. In the case of the latter, primitive distant field data will remain a good complement for dynamic geometry.

 

YES PLEASE

also, runtime-generated would be awesome because my game world is generated on the fly.

 

@N00MKRAD
I'm going to upload more videos showcasing true raytraced AO and Shadows with different primitives very soon.

 

Introducing Signed Distance Fields Shadows:

- Nothing rendered outside the screen
- No shadowmap/depthmap (whatever map) generated
- No shadowmapping artifacts (cause is not a shadowmap)
- No extra geometry rendered
- No Render Textures used
- No filtering (pure raytracing goodness)
- Raytraces only what's on the screen
- Running in forward renderer
- Light range can be of any distance (in the video point light = 100metters), doesn't matter as shadows are raytraced
- Nothing is pre-baked, everything is analytically computed on the GPU
- Many more ways to optimize it
- Works with any primitive (in the future with any mesh)
- Runs at 5000fps in the editor with plenty of lights

Tested with 500 lights @1500 fps. At this point is the forward multi pass renderer the bottleneck (but heck it's really fast):



And that's just the beginning, many many more things to come.

 

Insane stuff - can't wait!

 

You said raytracing again, did you mean raymarching or raycasting?

 

@buttmatrix
Raycasting will possibly be used in the future to query other types of occluders (non primitives). It's basically a combination of raymatching through distance fields and some tiny bits of raytracing.