Question Shader graph takes unbearable long time to compile

Hi, after upgrading to Unity 2019.2, Shader Graph could take up to 5 minutes to compile each shader during the Compiling shader variants process, which eventually adds up to more than an hour. Is there any way to speed it up?

Update:
I guess it's related to this post https://forum.unity.com/threads/compiling-shader-variants-taking-ages.527724/
So maybe Shader Graph has some new features in 2019.2 but could significantly slow down the compile speed?

 

SG should be fast if not faster to compile as Shader forge.

Why not make it as Blender Eevee. UI interface doesn't slow down if there's large node three.
Compilation of SG nodes could also be done before Play Mode.
User shouln't wait when creating new node or redirecting node.

 

In my case that's not a UI problem, saving a graph is totally fine. It happened at compile-time when Unity starts to automatically build the game. In my opinion, the compile process takes a way longer time than needed to strip/optimize passes and other stuff for Shader Graph shaders.

 

I noticed an empty HDRP/Lit shader graph included 982 variants, really? Compile such a shader could take up to more than 10 mins on my 8 cores CPU with 100% load and stuck for a long time at 978th variant. So most likely it's a bug, trying to repro and submit it.

 

After dozens of builts, it turned out that Shader Graph Lit node spent a lot of time to stripped programs, like "GBuffer/Forward: Stripping Fragment programs" and this process happened in every built, despite the fact that Unity should have cached the shader already

Update：
Here are some test results on my end. Still trying to make a repro

-------------------------------------------------------
Environment: Unity 2019.2 + HDRP
Packages: Latest
Shader Graph folder: 18 Lit Graphs, 40 Unlit Graphs
-------------------------------------------------------

Scenario #1: Clean git workspace, delete Library/Project/Temp folder then build
Build time: ~5200 seconds
-------------------------------------------------------

Scenario #2: Based on #1, build it again
Build time: ~1400 seconds
-------------------------------------------------------

Scenario #3: Based on #2, delete Shader Graph Folder
Build time: ~200 seconds
-------------------------------------------------------

Scenario #4: Clean git workspace, delete 18 Lit Graphs
Build time: ~300 seconds
-------------------------------------------------------

Scenario #5: Based on #4, build it again
Build time: ~307 seconds
-------------------------------------------------------

Scenario #6: Copy a lit graph compiled for a long time to an empty project, assign it to a prefab then build.
Build time: +50 seconds the first time, +30 seconds the next time
-------------------------------------------------------

 

Copied a lit shader to a new project and build. My current game says there're 20k variants has been stripped while the new project only has 6k, what kind of project setting could triple this number?

 

Before testing, I set ShaderVariantsLogLevel to "All Shaders" in my HDRPAsset, it recorded how many variants had been stripped. Then I put a timestamp before
BuildPipeline.BuildPlayer(), it recorded elapsed time for each build.

After hours of testing, it's time to make some conclusion. In short, there're 2 main factors could impact the build time


1. "Optimize Mesh Data"(ProjectSettings>Player>Optimization) could increase the build time. I have a test HDRP project which contains about 50 Shader Graphs(Lit and Unlit), this option increased the build time from 60 seconds to 85 seconds.

2. "Shader Strpping"(ProjectSettings>Graphics>SHaderStripping>LightmapModes==Custom).
I've no idea whether if it's a bug of HDRP or Shader Graph, but enabling basic things like Baked Directional and Baked ShadowMask could triple shader variants count as you can see in my previous post. Please be aware that each shader in my case needs to run the stripping process 10 times for different passes and programs, which add insanely "stripping time" for each build. Disabling them all literally halves the build time in my real project(1500s->750s).

I recommend you always disable #1 for dev build.
For #2, I hope Unity would fix/optimize it.

----
Test tricks:

1. Importing Shader Graph is not going to do anything, you've to use it on prefab materials, I guess that's the way Unity count it in.

2. Start from a clean test project and record your time/ variant count in an excel, it saves lots of time.

3. Record shader compiling and stripping time. Unity caches compiled shader, so if you keep building the game, you're recording its stripping time.

 

Any update on this? Yesterday I moved our project to 2019.2.6f1 and build times are taking ages.
I have tried everything found on the forum (setting Instancing Variants to Strip All, saving preloaded shader in an asset, optimize mesh data unchecked, even deleted library...) nothing works.
We are using addressables I we have to go through this hell two times everytime we need to test anything, one while building adressables and another one building the client (and I think it recompile one time for each assets bundle, I have seen the same shader name more than one time while building addressables), it doesnt matter if I didnt do any change, it compile the variants again.

 

No, nothing changed, but I managed to understand why it happens.

If we're using simple or hand-coded shaders, everything is fine since they provide us exactly what we need, nothing more. If we're using ShaderGraph, things could be changed. SG by design should be multi-functional and compatible with different platforms, so unavoidably, it would use "#pragma multi_compile KEYWORDS", this pragma will generate codes for each keyword combination.

Let's say we have "#pragma multi_complile _ A" and "#pragma multi_complile _ B", where a single "_" means "without it", then it will generate shader variants like "A", "B", "AB" for us. This feature could be convenient unless we've too many keywords. IIRC, HDRP/lit master node could have over 900 shader variants for each shader, that's a huge number. So the engine has to strip out unused combinations for us according to our graph/code, "#pragma shader_feature" is the alternative way to do this, it compiles keywords only when they're needed.

Then, why it takes so long? First, I guess many keywords could be redundant or not very commonly used, they're here because SG has to be fully functional. I think a hand-coded shader with 100 shader variants should be sufficient in most cases, leave alone 900, so maybe Unity would optimize it someday. Second, I guess the engine can not cache the shader correctly for now, so it has to run the stripping process for each build.

 

5 minutes? Heh. I'm waiting up to an hour or more for Unity to process a single connection change in my ShaderGraph Ubershader.