Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice

Compiling Shader Variants Taking Ages

Discussion in 'Shaders' started by Nigey, Apr 20, 2018.

  1. Pyxis-Belgique

    Pyxis-Belgique

    Joined:
    Jul 1, 2016
    Posts:
    14
    Just here to report the same unoptimized problem under 2019.3.4f1 with the default scene over URP and building for Android.

    I don't know if can call it a bug, but it's very regressive!

    All the time, energy and material we could save if this wasn't a thing.
    it would be worth finding a solution.
     
  2. Lightning_A

    Lightning_A

    Joined:
    Nov 30, 2018
    Posts:
    7
    For me, checking "Optimize Meshes" did the trick

    Edit: For those who checking optimize meshes didn't work for them, I suspect that the very first time you build your game it'll take a while (or when you make changes to a shader it'll recompile that shader) but try rebuilding it a second time (overwriting your first build) and see if that makes it much faster.
     
    Last edited: Mar 29, 2020
  3. Ville

    Ville

    Joined:
    Oct 5, 2012
    Posts:
    19
    I can report that deleting library helped me. The project had 120 possible shader variants with only a few shaders, but the build still got stuck compiling them over and over. Must be a Unity bug. Using 2018.4.19f1.
     
  4. CGTA

    CGTA

    Joined:
    Mar 17, 2020
    Posts:
    9
    I'm currently using Unity 2018.4.19f1 and trying to build fr UWP to deploy on the HoloLens and only using about 8 Shaders (Just added a partical shader btw too standard XR settings) and all the steps mentioned in here didnt fix. Tried deleting Library, Mesh Optimization is unchecked. Unity is building forever with the message: Particles/Standard Surface: x of about 61440.
    Any ideas what I could try out? Upgrade to 2019 maybe?

    Edit: I deleted the particel shader from the "always included" list and this brought me back to 1 min. build times and just tried it out on my hololens and I can see the gameobject with the material and the shader on it without any problems. Is there any problem not adding my used shader in this list?
     
    Last edited: Apr 2, 2020
    andreiagmu likes this.
  5. Whiz-Bang

    Whiz-Bang

    Joined:
    Jun 25, 2011
    Posts:
    20
    I am building my little project for the first time. A single scene that is using about 10 materials (URP/Lit/PBR shader).

    But what's going on??

    upload_2020-4-25_12-42-3.png
     
    Serhii-Horun and id0 like this.
  6. User340

    User340

    Joined:
    Feb 28, 2007
    Posts:
    3,001
    andreiagmu and Whiz-Bang like this.
  7. mancuso

    mancuso

    Joined:
    Dec 5, 2011
    Posts:
    63
    I tried checking "Optimize Meshes" and also saving the variant file, still no go. Had to delete Library folder and let Unity rebuild it. First build took about 8 minutes, now subsequent builds are much faster at about 2.5 minutes. I'm on 2019.3.13f1
     
  8. picolino0

    picolino0

    Joined:
    Aug 26, 2015
    Posts:
    9
    For me using the using the JDK that comes installed with unity fixed the issue.
     
    andreiagmu likes this.
  9. lucca_unity

    lucca_unity

    Joined:
    Mar 24, 2020
    Posts:
    3
    In my case, do this:
    ProjectSetting -> Graphics -> Built-in shader settings -> Always include shaders -> Remove Standard Shader

    If you want to use AssetBundle with standard shader, download official standard shader, and package into AssetBundle.
    https://unity3d.com/get-unity/download/archive
     
    andreiagmu likes this.
  10. joepeijkemans

    joepeijkemans

    Joined:
    Apr 24, 2018
    Posts:
    6
    Hi, how did you find out which shaders were causing this problem?
     
  11. lucca_unity

    lucca_unity

    Joined:
    Mar 24, 2020
    Posts:
    3
    When build
    When building shader, it's name will display under the progressbar.
     
  12. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,835
    It takes me like 5-10 minutes to make a very simple shader modification, I dont know if its because the project is too big or what, I was running into problems where I was running of kewords (256).
     
  13. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    256
    it's funny how people complain about 10 min... i left my computer run overnight and it never ended. In the morning, the "cacheshaderlibrary" folder had millions of file. It's completely insade. i had to force close unity. I'm mostly using standard shader, plus I have beautify, NGSS, AFS and some naturemanufacture. I have no idea how to get a build now :(

    should i delete unused shader ? they should not be included anyway as far as i know ?
    should i delete "exemple scene" that use shader i dont need ? again, my understanding is if the scene is not included in the build, then the shader used in them shall not be included.

    how can i build ?
     
    sivabalan12 likes this.
  14. bhk_unity

    bhk_unity

    Joined:
    Apr 11, 2018
    Posts:
    3

    Hiya - was in your same exact boat until about 30 minutes ago (although not quite as bad, I was seeing 2 hour builds). Whats worse, the game would basically immediately consume tons of memory when I ran it (I think trying to load an ungodly amount of shader variants into memory) and crash immediately.

    Following advice from some folks earlier in this thread got me back to < 5 minute build times and more importantly a working build.

    What ended up working for me was a combination of following Nigey's advice:

    1. Removing all entries from Project Settings > Graphics > Always Included Shaders list and only adding back ones I really needed (like UI/Default and UI/Default Font)
    2. Creating a ShaderVariant asset via Project Settings > Graphics "save to asset" at the bottom and adding that to my "Preloaded Shaders" list.
    3. Setting Project Settings > Graphics > Instancing Variants to "Strip All"

    And then I use the Oculus Integration for Desktop package so I also followed fxtentacle's advice about the OVRLint
    script and clearing out that stuff:

    > "FYI, the "OVR Performance Lint Tool" from the "Oculus Integration" asset keeps a list of FixRecords which contain pointers to game objects, materials and shaders. Apparently, that causes problems, because my build times increased by an hour after I imported the Oculus scripts." -fxtentacle

    I'm not sure which of these fixed it but just want to let the thread know these recommended solutions worked for me. I was tearing my hair out. Thank you Nigey and fxtentacle
     
  15. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    256
    have already done step 2 and 3, ill try number 1 and post back. tried again, left overnight and my shader golder now show 2.2 millions files. rock on unity :( (my shadervariant file talk about 232 variant)

    EDIT : didn't change anything. Final count is 3.3 millions files.... what the h is that

    EDIT 2 : despite the shared asset being extremely high in file count once it is created i can produce build in 20 minutes. I still need to understand how to get that file count much much lower.
     
    Last edited: Aug 24, 2020
  16. Tony-Lovell

    Tony-Lovell

    Joined:
    Jul 14, 2014
    Posts:
    127
    I don't see anyone asking: WHAT PROVOKES THE NEED FOR THIS?

    Is there something we can NOT change that will avoid the need to compile these things?

    it seems to me that something that has not changed should not require recompilation

    tone
     
    futurlab_xbox, andreiagmu and Unplug like this.
  17. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    256
    good question, my limited understanding it that it compile and cache EVERY single shader that are in the project instead of just caching all the variant for the one you actually use. Now why do i get more than 3 millions variant is out of my understanding.... it's pretty dumb for the editor to process everything when not require. It's slow, it crash, it use too much space and having so many file also prevent backup tool to work (sync toy cannot process the folder, the structure is too big, it doesn't need to be backup but microsoft sync toy dont offer to exclude subfolder...).
     
    Fibonaccov and andreiagmu like this.
  18. andreiagmu

    andreiagmu

    Joined:
    Feb 20, 2014
    Posts:
    175
    yoonitee likes this.
  19. yoonitee

    yoonitee

    Joined:
    Jun 27, 2013
    Posts:
    2,363
    Hello, I am trying to build the Oculus VR example for Unity 2019 LTS and am getting this problem.
    I have Unity XR package installed
    I have disabled Optimise Mesh which helped a bit.
    But it is still trying to build 1500 variants of each shader!

    Edit: Following another suggestion I did a "replace in files" with
    #pragma multi_compile replaced with: #pragma shader_feature
    Now only 150 variants.

    Seems a little better for the custom shaders. However still building 1200 versions of the standard shaders: Standard - FORWARD, Standard - FORWARD_DELTA, Standard -DEFFERED. Multiple times each.
     
    Last edited: Jan 15, 2021
    andreiagmu likes this.
  20. BeRtRoLlT

    BeRtRoLlT

    Joined:
    Nov 8, 2017
    Posts:
    3
    I had similar issues where the compilation of shaders just took way too long. After i moved to URP it took more than 12 hours to compile the shaders when building.

    What solved the issue for me was deleting the "ShaderCache" folder inside the [ProjectName]/Library folder. After deletion Unity recreated the shader cache when i ran the build next time, which took around 60 minutes. Once that was done each build completed between 2-4 minutes.

    I guess when there have been too many changes in shader code or shaders added/removed from the project the shader cache just gets too large, or it was just corrupted in my case.

    As far as i know it is not risky to delete the "ShaderCache" folder inside the "Library" folder, but there might be risks that i am not aware of.
     
    andreiagmu and yoonitee like this.
  21. yoonitee

    yoonitee

    Joined:
    Jun 27, 2013
    Posts:
    2,363
    Well I built it at last. BTW anyone know the previous version of Unity that doesn't spend so long on shaders? I haven't got the patience to wait that long every time!
     
  22. See_Sharp

    See_Sharp

    Joined:
    Mar 3, 2016
    Posts:
    74
    I'm on 2020.1 and I worked extensively with WebGL for the past year. I must say the compiling shader issue is gone. It might still be there in an initial build but uncompressed builds now take +- 4min for me to make.

    I'm talking about a relative - small file sized - project with only code. All assets are loaded externally. Might have something to do with it too.
     
    yoonitee likes this.
  23. yoonitee

    yoonitee

    Joined:
    Jun 27, 2013
    Posts:
    2,363
    I have a feeling it might be related to the Unity.UI package. It would make sort of sense if it was compiling a shader variant for each individual letter of the alphabet!

    Unfortunately the Oculus XR Plugin has a dependency for Unity UI (for some unknown reason) so I can't delete it. :(
     
  24. See_Sharp

    See_Sharp

    Joined:
    Mar 3, 2016
    Posts:
    74
    If that is true, I am grateful not to be Chinese. Can you image compiling shader variants for an alphabet containing 6500 characters... I imagine there's somewhere a Chinese cloud build server with a project with an average build time of 45 years.
     
    yoonitee likes this.
  25. yoonitee

    yoonitee

    Joined:
    Jun 27, 2013
    Posts:
    2,363
    Well this just sucks. The reason I liked Unity over Unreal is that Unity didn't compile loads of shaders.

    Mind you, just tried one of my own PC games that I updated from Unity 2018 to 2019 LTS and I don't get this problem with millions of standard shader variants. Maybe it's an Android thing.

    Probable Solution: I removed Vulkan from the list of graphics APIs under the graphics tab. So now it only has OpenGL ES. I would say that has reduced the number of shader variants to about a quarter of what it was. It can now compile in a few minutes.

    Still hundreds of variants though (I don't even know what a variant is for!!!!!).
     
    Last edited: Jan 17, 2021
    EvOne likes this.
  26. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    That preloaded shader list is not always a great thing to do. I had done that earlier, but my project changed along the way (making the list irrelevant I guess?).
    Removing it cut my build time 10 times!
     
  27. RoyBarina

    RoyBarina

    Joined:
    Jul 3, 2017
    Posts:
    98
  28. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    Unity 2020 LTS
    Build Time : 3 minutes 53.69 seconds.
    I Disable "OptimiZe Mesh Data" from Player > OtherSettings > Optimisation
    Build Time : 15.31 seconds.
     
  29. RoyBarina

    RoyBarina

    Joined:
    Jul 3, 2017
    Posts:
    98
    Did you cleared the shader cache in library before build?
     
    AlanMattano likes this.
  30. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,501
    No, Why?

    I delete it now.
    I compile and it took 3 minuts
    The second time 11 seconds
     
    Last edited: Jun 9, 2021
  31. RoyBarina

    RoyBarina

    Joined:
    Jul 3, 2017
    Posts:
    98
    Well, after a build it caches the variants so in the next build it won't compile them again (if they're the same) thus reducing building time :)
     
    AlanMattano likes this.
  32. Ecto-1

    Ecto-1

    Joined:
    Sep 28, 2013
    Posts:
    2
    Yeah, this is not going to work. My game is small, I'm using probably 4 or 5 shaders and only building 2 levels.
     

    Attached Files:

    • Ok.jpg
      Ok.jpg
      File size:
      18.7 KB
      Views:
      412
    craftsmanbeck likes this.
  33. astracat111

    astracat111

    Joined:
    Sep 21, 2016
    Posts:
    725
    Coming from a full time developer in Unity of 5 years, for what little I know I DEFINITELY know this:
    1. Unity cache's your shaders, so you build it once, it's not going to take that long to build again.
    2. Use as few of shaders as possible in your project, ESPECIALLY if you're targeting mobile in the future, you're not gonna want all those shaders in your project. Please listen to me when I say this, you do not want a lot of shaders you're not using in your project folder. Also, keep what shaders you do have in your project insanely organized...well...keep everything insanely organized, but especially shaders. You absolutely need to know what shaders your project is using and where at all times if you're planning on porting. Shaders are among one of the #1 causes of a badly optimized port. In Unity's project browser window, under the search bar you can type "t:shader" and you'll see all the shaders in your project. Watch out for downloading any Unity asset store assets as these can add more shaders, and many many times useless ones that do exactly the same thing that you're already doing with other shaders.
    3. Shaders aren't materials, so you can have many materials, but make sure to have fewer shaders!
    4. Stop using the Resources folder(s)! Use UnityWebRequest and StreamingAssets folder if you're on Windows, and if you're targeting Android create an asset bundle, and download it to your persistentDataPath and use UnityWebRequest to get the asset bundle, then use AssetBundle.Load.
    When I first started Unity, I was using Resources.Load a ton, but after I switched to a class I wrote called ExternalResources.Load (that just uses the UnityWebRequest class), after just deleting ALL resources folders and stopping using Resources all together, I can now build a 2GB project in about 60 seconds time, where as before it was taking 10+ minutes to build after shader cacheing, and like 1 hour 10 minutes+ to build before it. Library re-importing was also taking forever, caused by switching graphics APIs in player settings or caused by switching build platforms.

    Here's my class for 'ExternalResources' if anyone's interested. With Resources.Load I got used to the format for loading files like 'Resources.Load("System/Clock") without the '.png' and all that, so I kept that format with it. You can change the data path to whatever you'd like.

    The class is tested and will work on Android devices, so long as your assets are first in the persistentDataPath folder, this doesn't use AssetBundles. You use the class like this:

    Code (CSharp):
    1.  
    2. using System.Threading.Tasks;
    3. using UnityEngine;
    4.  
    5. public AudioSource audioSource;
    6.  
    7. IEnumerator GetAudioClipExample() {
    8.  
    9.    Task<AudioClip> taskAudioClip = ExternalResources.LoadAudioClipAsync("Audio/MyExampleFile"); //relative path without file extension, but this has to be .ogg here.
    10.    while (taskAudioClip.Finished == false) {
    11.  
    12.        Debug.Log("Loading audio clip....");
    13.  
    14.        yield return new WaitForSeconds(0.05f);
    15.     }
    16.  
    17.      AudioClip myNewAudioClip = taskAudioClip.Result;
    18.  
    19.      audioSource.PlayOneShot(myNewAudioClip,1.0f);
    20.  
    21. yield break;
    22. }
    Above is untested code, below is tested and works on Android.

    Code (CSharp):
    1.  
    2. public class ExternalResources
    3. {
    4.  
    5.     public static string UnityDataPath = Application.persistentDataPath;
    6.  
    7.     #region PUBLICMETHODS
    8.  
    9.     public static async Task<AudioClip> LoadAudioClipAsync(string relativePath)
    10.     {
    11.         string filePath = GetFileFullPath(relativePath, ".ogg");
    12.         if (filePath == null)
    13.             return null;
    14.  
    15.         UriBuilder uriBuilder = new UriBuilder(filePath);
    16.  
    17.         using (UnityWebRequest unityWebRequest = UnityWebRequestMultimedia.GetAudioClip(uriBuilder.Uri, AudioType.OGGVORBIS))
    18.         {
    19.             unityWebRequest.SendWebRequest();
    20.  
    21.             while (unityWebRequest.isDone == false)
    22.                 await Task.Delay(5);
    23.  
    24.             if (unityWebRequest.result == UnityWebRequest.Result.ConnectionError || unityWebRequest.result == UnityWebRequest.Result.DataProcessingError)
    25.             {
    26.                 UnityEngine.Debug.LogError("Data Processing or Connection Error for filePath: " + filePath);
    27.                 return null;
    28.             }
    29.  
    30.             return DownloadHandlerAudioClip.GetContent(unityWebRequest);
    31.  
    32.         }
    33.  
    34.     }
    35.  
    36.     public static async Task<Texture2D> LoadTexture2DAsync(string relativePath)
    37.     {
    38.         string filePath = GetFileFullPath(relativePath, ".png");
    39.         if (filePath == null)
    40.             return null;
    41.  
    42.         UriBuilder uriBuilder = new UriBuilder(filePath);
    43.  
    44.         using (UnityWebRequest unityWebRequest = UnityWebRequestTexture.GetTexture(uriBuilder.Uri, false))
    45.         {
    46.             unityWebRequest.SendWebRequest();
    47.  
    48.             while (unityWebRequest.isDone == false)
    49.                 await Task.Delay(5);
    50.  
    51.             if (unityWebRequest.result == UnityWebRequest.Result.ConnectionError || unityWebRequest.result == UnityWebRequest.Result.DataProcessingError)
    52.             {
    53.                 UnityEngine.Debug.LogError("Data Processing or Connection Error for filePath: " + filePath);
    54.                 return null;
    55.             }
    56.  
    57.             return DownloadHandlerTexture.GetContent(unityWebRequest);
    58.  
    59.         }
    60.  
    61.     }
    62.  
    63.     public static Texture2D LoadTexture2D(string relativePath)
    64.     {
    65.         string filePath = GetFileFullPath(relativePath, ".png");
    66.         if (filePath == null)
    67.             return null;
    68.  
    69.         byte[] bytes = File.ReadAllBytes(filePath);
    70.  
    71.         Texture2D texture2D = new Texture2D(2, 2);
    72.         texture2D.LoadImage(bytes);
    73.  
    74.         SetTexture2DDefaultSettings(ref texture2D, relativePath);
    75.  
    76.         return texture2D;
    77.  
    78.     }
    79.  
    80.     public static TextAsset LoadTextAsset(string relativePath)
    81.     {
    82.         string filePath = GetFileFullPath(relativePath, ".txt");
    83.         if (filePath == null)
    84.             return null;
    85.  
    86.         TextReader textReader   = new StreamReader(filePath, Encoding.UTF8);
    87.         string textPulled       = textReader.ReadToEnd();
    88.         textReader.Close();
    89.  
    90.         TextAsset textAsset = new TextAsset(textPulled);
    91.  
    92.         return textAsset;
    93.     }
    94.  
    95.     #endregion
    96.  
    97.  
    98.     #region PRIVATEMETHODS
    99.  
    100.     private static void SetTexture2DDefaultSettings(ref Texture2D texture2D, string relativePath)
    101.     {
    102.         if (relativePath.Contains("AniSets")) {
    103.             texture2D.filterMode = FilterMode.Point;
    104.             UnityEngine.Debug.Log("aniset type!");
    105.         }
    106.  
    107.     }
    108.  
    109.  
    110.     private static string GetFileFullPath(string relativePath, string extension)
    111.     {
    112.         string fullPath = UnityDataPath + "/" + relativePath + extension;
    113.         if (File.Exists(fullPath) == false)
    114.         {
    115.             UnityEngine.Debug.LogError("No asset found with path of: " + fullPath);
    116.             return null;
    117.         } else
    118.         {
    119.             return fullPath;
    120.         }
    121.  
    122.     }
    123.  
    124.     #endregion
    125.  
    126. }
    If you're on Windows, this is way easier, just use StreamingAssets and use a methodology like above, but whatever you do never create a 'Resources' folder for your assets, and never ever use Resources.Load. Your build times will EXPONENTIALLY grow.

    What most people would recommend is building an asset bundle, on Windows it's really easy you can just put this bundle in your StreamingAssets folder, but this won't work on other devices you port to, so if you're thinking of going multi-platform, just keep this in mind, you want to use persistentDataPath for your external data.

    The documentation on asset bundles can be annoying at first, but once you get the hang of it it isn't so bad.

    You can't load Scenes and Prefabs externally unless you use asset bundles.
     
    Last edited: Jul 25, 2021
    RoyBarina likes this.
  34. R1PFake

    R1PFake

    Joined:
    Aug 7, 2015
    Posts:
    540
    @astracat111 Did you try the new Addressable system yet? Apparently it's the new/easier way to deal with asset bundles (but it will still create asset bundles behind the scene if I remember correctly)
     
    astracat111 likes this.
  35. astracat111

    astracat111

    Joined:
    Sep 21, 2016
    Posts:
    725
    I did and I like it, but hadn't tested it with Android yet. I really like how they only have asynchronous loading, I think it's important right off the bat to start learning asynchronous code both using IEnumerators and Tasks.
     
  36. MassimoFrancesco

    MassimoFrancesco

    Joined:
    Jul 6, 2019
    Posts:
    44
    Encountering the same issues described here with 2020.3.2f1 currently. ~1 GB sized project is currently building for 40 minutes and still not finished.

    Sad to see this has been an ongoing issue for so long. Really hope Unity Team could give a statement on this.
     
  37. id0

    id0

    Joined:
    Nov 23, 2012
    Posts:
    455
    I hardly doubt it. I'm on 2021 beta and I build my project about hour(and this is not too bad). When all shader are compiled, and if you don't change any shader after that, next bulds will be lot faster. 5-10 minutes.
     
  38. primaerfunktion

    primaerfunktion

    Joined:
    Jan 16, 2012
    Posts:
    98
    Also running into some crazy shader variant numbers it after upgrading my project. Already aborted the build yesterday evening after about three+ hours. Restarted this morning.
     

    Attached Files:

    TigerHix likes this.
  39. id0

    id0

    Joined:
    Nov 23, 2012
    Posts:
    455
    primaerfunktion, have you switch to "strip unused" or "strip all" in the Graphics settings?
    Untitled-1.jpg
     
  40. primaerfunktion

    primaerfunktion

    Joined:
    Jan 16, 2012
    Posts:
    98
    Yup it is. Followed all the tips people gave on here. Didn't do much. The second time the Build continued to for 5 more hours btw. The project is really not particularly big.

    Edit: Looks like some the Shadergraph Shaders I did are very happy to create around a million variants? Even a complete freshly created Lit one. Yolo. Unlit is 113.

    Just tested on a fresh Project. A just created shadergraph sits at about 170k variants.
     

    Attached Files:

    Last edited: Aug 15, 2021
  41. Kitsune_bcn

    Kitsune_bcn

    Joined:
    Jul 1, 2021
    Posts:
    16
    My experience with 2020 LTS is that, folowing the advices here, I managed to lower the "1st" build from 60 mins to 15. Following builds take only 2 or 3 mins (simple scene) so everything is ok buuuut, if you do this, then the editor starts to be very laggy (like 1 min to save a scene for example). If i delete the shadercache folder, then the editor goes snappy again, but then, of course the next build is like the first one, again 15 mins to build. So, I lose time on one side, or the other, but the 2 things together don't work :(
     
  42. driveshack_leo

    driveshack_leo

    Joined:
    Jul 2, 2021
    Posts:
    1
    Unity might as well remove the build progress bar at this point. It just sits at 99% for hours while it compiles shader variants.

    I've tried all the tips in this thread and nothing seems to get the number of shader variants down. I guess we just accept that builds take longer now until Unity devs actually address this.
     
  43. Shizola

    Shizola

    Joined:
    Jun 29, 2014
    Posts:
    473
    I saw on the Discord that improvements have been made but haven't been pulled into the main branch yet.
     
    andreiagmu likes this.
  44. andreiagmu

    andreiagmu

    Joined:
    Feb 20, 2014
    Posts:
    175
    Some years ago (after first encountering this thread :p), in Project Settings -> Graphics, I had changed my project's Shader Stripping settings (Lightmap Modes and Fog Modes) to custom values, and I left Instancing Variants as "Strip Unused".
    It did give a nice boost on the builds with already cached variants at the time, but it still seemed a bit slower than it should, as every build seems to make the shader stripping process all over again; sometimes it randomly loses some cached variants and has to compile them again.

    Now, I'm using Unity 2020.3.19f1 and URP 10.6.0.
    Today, I set both Lightmap Modes and Fog Modes back to Automatic, and I had a HUGE boost on my cached variants builds. The parts where it would always take some time "Compiling Shader Variants" (actually stripping the variants), now they basically complete instantly. :eek:
     
  45. iGoA

    iGoA

    Joined:
    Aug 23, 2020
    Posts:
    22
    Hi all!
    Yesterday, I've tried to build my game for Android the first time:

    I've implemented all suggested measures in this thread ("strip all", save shadervariants to asset, set asset to preloaded, no "optimize mesh data", no "gpu instancing", emptying shadercache) and migrated the project from Unity 2019.4 to Unity 2021.1, but I was't even able to get at least one build done the whole evening.
    Now my question: My project uses the standard render pipeline. Is there a realistic chance, that this problem can be solved by migrating to URP? Or is there any way to reduce the amount of variants?
     
  46. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    433
    Write shader code instead, use shader features wisely, the problem would be eliminated
     
  47. pantang

    pantang

    Joined:
    Sep 1, 2016
    Posts:
    219
    I had this previously, I added all my shaders to a empty disabled gameobject in my first scene, and then reset the Always inclduded shader settings to default. Problem solved for me went from 50+mins down to under 10.
     
  48. iGoA

    iGoA

    Joined:
    Aug 23, 2020
    Posts:
    22
    @zhuchun:
    Unity offers Shader Graph, but we rather code our own to avoid 3h build time? I'm not sure, if this answer satisfies me.
    Using shader features wisely might be a good tip in general regarding performance, but what is wisely here? Most materials in my game have the standard shader with a normal map and a single color (no texture) as base map. Is this already too much?
     
  49. zhuchun

    zhuchun

    Joined:
    Aug 11, 2012
    Posts:
    433
    I would argue that it's not a Unity thing, it's a Shader thing, the Material Editor of Unreal Engine has the same problem as well. The more keywords a shader used, the more variation is needed to be compiled, stripping itself is the process to help people get rid of variation they would never use. It's MUCH faster than compiling them.

    Using graph tools is a great way to help artists approach those fancy effects, however, engines do NOT know how are people going to use it, so the way they solve this problem is by Template. Each time you drag a master node into a graph, you're choosing a template. In order to meet various needs, templates are needed to be flexible, which means MUCH more keywords than you actually need. As long as stripping is working, things should be fine.

    So as you can see, it's not about how people could write shaders "wisely", I mean if you're writing a shader, no doubt you would only use keywords you actually need, right? The problem is that templates are using them "unwisely", Unity is optimizing them, that's cool, but I'm not going to be stuck here for any moment. In case you wanna know how "unwise" it is, IIRC, the early version of the PBR template would generate over 1k variations, for comparison, most well-tailored shaders only generate about 5~30.

    In most game studios(well, I guess), we use graph tools to make effects, then, we will redo it again in code for performance-critical shaders. We also customize our own "Standard" shader to make sure that the stripping process is minimal.

    I hope this helps.
     
    Last edited: Nov 13, 2021
    RoyBarina likes this.