[Next-Gen Soft-Shadows 2] Sophisticated dynamic penumbra Shadows for Unity

Just noticed that the Gov of Canada (Canadian Space Agency) is using NGSS for their spacial simulations.
This have been a long and fun journey. Thanks to all our big supporters (yes you, the one reading this message right now)!

 

I'm seeing periodic spikes with Contact Shadow in v2. (I have Dir. Light and Contact Shadows set up, no local shadows.) Using Unity 2018.3.3, Windows, 1060 GPU. Any idea how to reduce those? (We are loading/unloading additive scenes with terrain tiles, but these spikes do not coincide with those moments.)

 

@gecko,
Can you test with earlier versions of Unity 2018/2017? I've seen degraded performance with 2018.3.
Do you have the spikes with the NGSS package scenes?

 

I can't test with older versions very easily (and can't roll back for sure). I'll import the demo scene and test.

 

Folks
ContactShadows no longer gets displayed on top of other cameras, you can now use as many cameras as you want:

Credits to Andre. K for providing a good test case project!
PS: I have a new patch version coming this week (or the next) with tons of fixes, ARKIT, ContactShadows, etc. I'm working in parallel with patch fixes and v2.1

 

And @alternativevisual is making good progress with the new user friendly inspectors. Preview (still in WIP, some of these properties will probably change, get removed, etc):

 

Saw this today, thought you might like it.
https://www.eidosmontreal.com/en/ne...x-mankind-divided-and-rise-of-the-tomb-raider

If you go to page 27 of the Tomb raider presentation, you can start reading about their Implementation of Sample Distribution Shadow Maps.

 

Yeah this is something I wanted to do with the standard renderer but I need to create my own cascaded rendering system and requires manual integration in user shaders (is not that hard but can put people off a bit). This technique distributes the cascades in a better way, saving shadowmap resolution (less VRAM bandwith usage) and speeding up the process or using the extra saved juice to rank up shadowmap quality a bit more.
Maybe for the SRP version of NGSS.

 

@ceebeee
Brain fart! I'll take that back sorry, it doesn't take people any shading modification. I can globally set a custom directional depth texture rendered with my own cascaded system (which can be cached statically and dynamically too) and being called directly on internal NGSS libraries. But yeah, this require time and might not worth in the Standard pipeline anymore. I had this planned for SRP anyway but I'll re-evaluate the possibility for the builtin pipeline in the future.

 

The Raytracing shadow in 2.1 is true raytrace? Or is it Frustum Trace？

 

@CG_Bull
Version 2.1 will provide both, frustum traced shadows (local and directional) and distance fields raytraced shadows. What I call primitive shadows which are primitive shapes that has analytical functions. True Raytraced shadows are planned to NGSS when Unity fully integrate the new DXR API (sometime this year).

 

OK. This cool.

 

@tatoforever
Difference between editor and standalone build disappeared after I checked "keep ondisable" option. I had also difference in cascade blending, so I checked again if I was not doing something wrong.... Now everithing is OK. Sorry

 

Better workflow with user friendly interfaces is coming

NGSS Explorer: Control all NGSS components in one tab, setup scene faster;
NGSS Presets: Setup your shadows per Quality Level;

Other improvements:
- Driving settings for multiple lights with one NGSS component;
- Exposed Denoiser and added shadows bias options;

All in v2.1 with Overhaul Inspectors Update [WIP]


music used in video is made by our artist, Chuli4n - forest: https://soundcloud.com/search?q=Chuli4ฏ๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎๎

 

Curiously eerie sort of zombie dancing. Cool!

Oh, and the shadow thing was nice too.

 

https://github.com/keijiro/PuppetTest

 

This is actually genius!

 

As you already know, I added a new developer @alternativevisual to move things faster!
He will be in charge of the new inspectors and Editor tools and new more stuff we are working out.
Version 2.1 comes fully loaded with new features, improvements and artist friendly tools.
When I asked for 15$ upgrade , this is what it means, on-going tons of updates, features and stellar user support (email or forums).
PS: NGSS 2.0.5 patch is almost done.

 

Is there going to be support added for the new area lights of hdrp?

 

@Ruchir
HDRP & LWRP support is planned for this year yes.

 

@tatoforever
I started using NGSS and I love the shadows quality. Though when I build I receive 3 errors from 2 3rd party shaders that I'm using:

Code (CSharp):

1. Shader error in 'CTS/CTS Terrain Shader Advanced': maximum ps_4_0 sampler register index (16) exceeded at line 646 (on d3d11)
2. Compiling Fragment program with UNITY_PASS_FORWARDADD POINT_COOKIE SHADOWS_CUBE SHADOWS_SHADOWMASK _USE_AO_ON
3. Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR

Code (CSharp):

1. Shader error in 'NatureManufacture Shaders/Standard Shaders/Standard Metalic Snow': maximum ps_4_0 sampler register index (16) exceeded at line 110 (on d3d11)
2. Compiling Fragment program with UNITY_PASS_FORWARDADD SPOT SHADOWS_DEPTH LIGHTMAP_SHADOW_MIXING SHADOWS_SHADOWMASK GPU_FRUSTUM_ON _USESNOW_ON
3. Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR

Code (CSharp):

1. Shader error in 'NatureManufacture Shaders/Standard Shaders/Standard Metalic Snow': maximum ps_5_0 sampler register index (16) exceeded at line 110 (on d3d11)
2. Compiling Fragment program with UNITY_PASS_FORWARDADD SPOT SHADOWS_DEPTH LIGHTMAP_SHADOW_MIXING SHADOWS_SHADOWMASK PROCEDURAL_INSTANCING_ON GPU_FRUSTUM_ON _USESNOW_ON
3. Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR

When I reimport those shaders the errors disappear, but reappear on build. Strangely enough the project finishes building and I don't see any problems when running it. This didn't occur without NGSS as far as I remember. I'm unsure if this is something influenced by NGSS, the shader authors or my project... I'm on 2018.3.5f1.


Could you help me out?

 

@bbn777
Your shaders looks to be quite complex plus NGSS adds to that complexity (depending on the number of samplers). If that the case, make sure you force your shaders to compile to SM5.0. In the editor they compile fine under SM5.0 but your target platforms maybe you have some APIs with SM3.0 enabled that's the reason why it runs fine on Editor but produces compile errors at build.
One tip to try (in case you can't reduce your shader complexity). You can try Deferred rendering, forward rendering is very limited and you can hit the SM instructions/interpolators faster as everything is computed in the lighting pass, while Deferred deffers lighting in a secondary pass (this pass is used for shadows) thus splitting the number of instructions.
I have other tips for you to try but first try looking into your shaders and make sure they don't target anything lower than SM5.

 

@tatoforever

Thanks for a quick reply. I'm on deferred rendering path. I've changed #pragma target XXX in those 2 shaders to #pragma target 5.0 and one of the errors disappeared. I'm not sure if that was what you mant by forcing shaders to compile to SM5.0.

Code (CSharp):

1. Shader error in 'CTS/CTS Terrain Shader Advanced': maximum ps_5_0 sampler register index (16) exceeded at line 646 (on d3d11)
2. Compiling Fragment program with UNITY_PASS_FORWARDADD SPOT SHADOWS_DEPTH SHADOWS_SHADOWMASK
3. Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR
4.  

Code (CSharp):

1. Shader error in 'NatureManufacture Shaders/Standard Shaders/Standard Metalic Snow': maximum ps_5_0 sampler register index (16) exceeded at line 110 (on d3d11)
2. Compiling Fragment program with UNITY_PASS_FORWARDADD SPOT SHADOWS_DEPTH SHADOWS_SHADOWMASK GPU_FRUSTUM_ON _USESNOW_ON
3. Platform defines: UNITY_ENABLE_REFLECTION_BUFFERS UNITY_USE_DITHER_MASK_FOR_ALPHABLENDED_SHADOWS UNITY_PBS_USE_BRDF1 UNITY_SPECCUBE_BOX_PROJECTION UNITY_SPECCUBE_BLENDING UNITY_ENABLE_DETAIL_NORMALMAP SHADER_API_DESKTOP UNITY_LIGHT_PROBE_PROXY_VOLUME UNITY_LIGHTMAP_FULL_HDR
4.  

Edit:additional information: those are not my shaders and I'm novice in shader code writing...

 

@bbn777
So yeah, it looks like its your shader complexity. They seems to remain quite complex even after forcing them to SM5.0.
Based on these errors you posed, you don't seems to be limited by the instruction number but by the number of interpolators (16 for SM5) and NGSS don't add any interpolator, is only more instructions per frag that NGSS adds (and memory fetches too obviously). Interpolators are data being passed from the Vertex program to the Fragment program. And NGSS uses the same interpolators as default Unity shadows (nothing is being added). However your custom shaders adds more interpolators for their specific effects (snow requires world space data, terrain requires lot of interpolators for all the layers, etc).
What are your target APIs under Player Settings (uncheck Auto Graphis and send me a picture of it).

 

Oh, you are fast @tatoforever !

Here it is:

 

@bbn777
Take a look a this thread and see if it can help:
https://forum.unity.com/threads/passing-more-than-16-textures-to-shader.500406/

Lastly, what happen if you build with the same settings and no NGSS?

 

Maybe I should have started with this... I'll try uninstalling NGSS and check what happens during the build...

 

That took some time. tl:dr: after removing NGSS the project compiles without errors.

Steps I did to remove NGSS:
- removing using NGSS menu (I've checked - it correctly moved .bak files over original ones)
- removing NGSS scripts from lights in the scene (directional light + one point light)
- switched default screen space shadow shader do build-in
- restarted unity
- removed NGSS folder
- also I've check 'auto' for Graphics API in Player settings.

 

One thing I'll try additional - revert shader code to defaults, provided by authors (so 3.0 for Nature Manufacture and 3.5 for CTS)

 

@bbn777
I still believe that you have some shader somehow not set to SM5.0. All your custom shaders needs to be set to SM5, including default Standard Shaders. Make sure this is the case. If is not the case then Unity is doing something funny whe compiling all shader variants at build time cause in Editor the shaders are compiled fine. Disable maybe other platforms (Mac and Linux) and see if it builds fine with NGSS.

 

@bbn777
If you still cant find a way to compile without the shader limit error. Can you reproduce the same compile error in a small test project with only NGSS package and your shaders?
And send it to support@psychozinteractive.com
I'll take a quick look at.

 

@tatoforever

This might be the case that I missed something. This was the first time for me to edit shader code

I'll reinstall NGSS and try to disable other platforms. If I read the docs (https://docs.unity3d.com/Manual/SL-ShaderCompileTargets.html ) correctly Mac is not supporting SM5.0.

If that will not help I'll try to reproduce the error in some mini-project and sent it to you.

Thanks for superb support!

 

You can also select your shaders int he Project View and remove all the variants you don't need. This is an extra step you can take to make your builds faster and make sure your shader variants is compiling to a platform that support your projects needs.

 

@tatoforever I did manage to reproduce the issue in small project. I've sent you an e-mail. Thanks again for your help!

 

Just a small heads up in case someone else runs into it, but if you have your project set to use .NET 4.x and .NET Standard 2.0, you will get errors with the NGSS install script.

Assets/Psychose Interactive/NGSS/Editor/NGSS_LibrariesSetup.cs(32,81): error CS0246: The type or namespace name `FileSystemRights' could not be found. Are you missing an assembly reference?

Assets/Psychose Interactive/NGSS/Editor/NGSS_LibrariesSetup.cs(32,106): error CS1070: The type `System.Security.AccessControl.AccessControlType' has been forwarded to an assembly that is not referenced. Consider adding a reference to assembly `System.Security.AccessControl, Version=4.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

Changing compatibility level to .Net 4.x will fix the issue. But you might want to consider making it degrade more gracefully as i'm sure not everyone wants to use .NET 4.x

Cheers

 

Folks,
I just fixed the bug with ARKit shadows projection if Cascaded Shadows was enabled. This fix will be included in this week patch.

 

@ceebeee
That was my mistake, I forgot to remove that code before submitting final version. Please remove this from the
NGSS_LibrariesSetup.cs

Code (CSharp):

1. using System.Security.AccessControl;

and his method:

Code (CSharp):

1.  
2.  
3.     // Removes an ACL entry on the specified directory for the specified account.
4.     public static void RemoveDirectorySecurity(string FileName, string Account, FileSystemRights Rights, AccessControlType ControlType)
5.     {
6.         // Create a new DirectoryInfo object.
7.         DirectoryInfo dInfo = new DirectoryInfo(FileName);
8.  
9.         // Get a DirectorySecurity object that represents the
10.         // current security settings.
11.         DirectorySecurity dSecurity = dInfo.GetAccessControl();
12.  
13.         // Add the FileSystemAccessRule to the security settings.
14.         dSecurity.RemoveAccessRule(new FileSystemAccessRule(Account, Rights, ControlType));
15.  
16.         // Set the new access settings.
17.         dInfo.SetAccessControl(dSecurity);
18.     }

I included this fix also in this week patch.

 

Upcoming patch (should be ready this week). This is what it contains (do let me know if you have an issue to be fixed):
Being fixed:
- Unity LTS 2017.4.19 Mac Desktop (Spot light shadows does not work if Metal API)
- Make default values to directional/local libraries to avoid shadowed screen on first install
- Improve NGSS library installer. Freezes sometimes with UnityHub, works fine on MAC
- Add more tutorials?
Already fixed:
- Arkit shadowprojection bug.
- Remove using System.Security.AccessControl; and its methode block code, no needed
- Hide Graphics Cascaded Shadows on NGSS_Directional till v2.1
- Contact shadows on top of the camera (check UFPS test example)
- Make sure Directional shadows works if no library (cloud builds for example) is installed or it will create compile errors

 

Forget about this, needed to update my Visual Studio

 

@TimeTours
Just in case, you need to add the NGSS_Local component to all your local lights (or at least to your main one). And make sure one of your NGSS_Local component has the NGSS_MANAGE_GLOBAL_SETTINGS enabled (making this light manage global settings for all local lights in your scene). LOCAL properties of your Spot/Point lights aren't affected by GLOBAL_SETTINGS.

 

Is it normal that I can't bake mixed directional lights with shadow mask?

All I get is a black scene no bounced light.

 

@sacb0y
NGSS doesn't affect your ability to bake. Baking and NGSS are different rendering system. Though, NGSS support all lighting modes once baked, computed, calculated, etc.

 

Yeah my bad, I was baking with the wrong active scene lol. I just thought t was the problem cause it happened right after buying the new plugin.

 

Contgrats @tatoforever #1 whoo!

 

@ceebeee
Yup,
I'm happy and it wouldn't never be possible without this beautiful community of supportive developers.

 

is the 2.1 update with raytraced local shadows out yet? I see the asset store version is listed as 2.0.2

 

@mehrankan
Version 2.1 is not out yet, we are still working on it.

 

cool, well I already bought ngss2 so really looking forward to those raytraced shadows.

 

So far so good (bullets points with stroke lines == fixed):

- Spot light shadows does not show (when Metal API is selected on Mac OS)
- Update documentation & tutorial
- Make default values to directional/local libraries to avoid shadowed screen
- Remove OrderedDitheringDir array out of function(causes compilation error in PS4)
- Arkit shadowprojection fix
- Remove using System.Security.AccessControl; and its methode block code, no need
- Hide Graphics Cascaded Shadows on NGSS_Directional till v2.1
- Contact shadows on top of the camera (check UFPS test example)
- Make sure Directional shadows works if no library (cloud builds for example) is installed or it will create compile errors

 

Folks,
I would need some hand with Mac. Just discovered that my mac doesn't support Metal (silly me) and I need to test if my Metal spot shadow fix works. I think i fixed it but need someone with a Mac and Metal support to confirm. Reach me out through support email. Once tested I can submit the patch for review.
PS: Blindly debugging GPU code for the win! And yeah I need a new Mac!
Cheers,