Search Unity

Help Wanted Shader graph takes unbearable long time to compile

Discussion in 'Shader Graph' started by zhuchun, Aug 8, 2019.

  1. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    379
    Hi, after upgrading to Unity 2019.2, Shader Graph could take up to 5 minutes to compile each shader during the Compiling shader variants process, which eventually adds up to more than an hour. Is there any way to speed it up?

    Update:
    I guess it's related to this post https://forum.unity.com/threads/compiling-shader-variants-taking-ages.527724/
    So maybe Shader Graph has some new features in 2019.2 but could significantly slow down the compile speed?
     
    Last edited: Aug 8, 2019
  2. konsic

    konsic

    Joined:
    Oct 19, 2015
    Posts:
    638
    SG should be fast if not faster to compile as Shader forge.

    Why not make it as Blender Eevee. UI interface doesn't slow down if there's large node three.
    Compilation of SG nodes could also be done before Play Mode.
    User shouln't wait when creating new node or redirecting node.
     
  3. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    379
    In my case that's not a UI problem, saving a graph is totally fine. It happened at compile-time when Unity starts to automatically build the game. In my opinion, the compile process takes a way longer time than needed to strip/optimize passes and other stuff for Shader Graph shaders.
     
  4. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    379
    I noticed an empty HDRP/Lit shader graph included 982 variants, really? Compile such a shader could take up to more than 10 mins on my 8 cores CPU with 100% load and stuck for a long time at 978th variant. So most likely it's a bug, trying to repro and submit it.
     
  5. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    379
    After dozens of builts, it turned out that Shader Graph Lit node spent a lot of time to stripped programs, like "GBuffer/Forward: Stripping Fragment programs" and this process happened in every built, despite the fact that Unity should have cached the shader already:confused:

    Update:
    Here are some test results on my end. Still trying to make a repro

    -------------------------------------------------------
    Environment: Unity 2019.2 + HDRP
    Packages: Latest
    Shader Graph folder: 18 Lit Graphs, 40 Unlit Graphs
    -------------------------------------------------------

    Scenario #1: Clean git workspace, delete Library/Project/Temp folder then build
    Build time: ~5200 seconds
    -------------------------------------------------------

    Scenario #2: Based on #1, build it again
    Build time: ~1400 seconds
    -------------------------------------------------------

    Scenario #3: Based on #2, delete Shader Graph Folder
    Build time: ~200 seconds
    -------------------------------------------------------

    Scenario #4: Clean git workspace, delete 18 Lit Graphs
    Build time: ~300 seconds
    -------------------------------------------------------

    Scenario #5: Based on #4, build it again
    Build time: ~307 seconds
    -------------------------------------------------------

    Scenario #6: Copy a lit graph compiled for a long time to an empty project, assign it to a prefab then build.
    Build time: +50 seconds the first time, +30 seconds the next time
    -------------------------------------------------------
     
    Last edited: Aug 10, 2019
  6. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    379
    Copied a lit shader to a new project and build. My current game says there're 20k variants has been stripped while the new project only has 6k, what kind of project setting could triple this number?o_O

    20k.png

    6k.png
     
  7. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    379
    Before testing, I set ShaderVariantsLogLevel to "All Shaders" in my HDRPAsset, it recorded how many variants had been stripped. Then I put a timestamp before
    BuildPipeline.BuildPlayer(), it recorded elapsed time for each build.

    After hours of testing, it's time to make some conclusion. In short, there're 2 main factors could impact the build time


    1. "Optimize Mesh Data"(ProjectSettings>Player>Optimization) could increase the build time. I have a test HDRP project which contains about 50 Shader Graphs(Lit and Unlit), this option increased the build time from 60 seconds to 85 seconds.

    2. "Shader Strpping"(ProjectSettings>Graphics>SHaderStripping>LightmapModes==Custom).
    I've no idea whether if it's a bug of HDRP or Shader Graph, but enabling basic things like Baked Directional and Baked ShadowMask could triple shader variants count as you can see in my previous post. Please be aware that each shader in my case needs to run the stripping process 10 times for different passes and programs, which add insanely "stripping time" for each build. Disabling them all literally halves the build time in my real project(1500s->750s).

    I recommend you always disable #1 for dev build.
    For #2, I hope Unity would fix/optimize it.

    ----
    Test tricks:

    1. Importing Shader Graph is not going to do anything, you've to use it on prefab materials, I guess that's the way Unity count it in.

    2. Start from a clean test project and record your time/ variant count in an excel, it saves lots of time.

    3. Record shader compiling and stripping time. Unity caches compiled shader, so if you keep building the game, you're recording its stripping time.
     
    Last edited: Aug 12, 2019