Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice
  2. We've opened up a space to discuss, share feedback, and showcase everything related to the Unity Shader Graph! Come show us what you've made.
    Dismiss Notice

Compiling Shader Variants Taking Ages

Discussion in 'Scripting' started by Nigey, Apr 20, 2018.

  1. qq47845988

    qq47845988

    Joined:
    Dec 20, 2018
    Posts:
    1
    This is radiculas, we upgrade from 2018.1 to 2018.3.0f2,and now it takes me 400 seconds to compile a "Standard Shader".

    I am also waiting for the solution.
     
    Shorely likes this.
  2. CoolTomatoYT

    CoolTomatoYT

    Joined:
    Apr 23, 2018
    Posts:
    2
    I'VE FOUND A FIX! If your game uses the Unity Post Processing Stack, then if you remove it the time will go much quicker! My shader variants went from 15k to just 120.
     
  3. embeddedt

    embeddedt

    Joined:
    Mar 10, 2018
    Posts:
    22
    I think I may have found a way around this, but it only works for a new project. I am using 2018.3 so it's possible they did fix something internally.

    Create an empty project. Build it for your platform. It will compile some shader variants. Slowly add all your assets, building every so often. This seems to keep the compile time to a reasonable level. Further changes to the scene don't appear to cause all shader variants to be recompiled.

    This is a workaround and I hope that Unity comes up with a better way of handling this.
     
  4. embeddedt

    embeddedt

    Joined:
    Mar 10, 2018
    Posts:
    22
    Unfortunately my game doesn't use the Post-Processing stack. Even with 120 shader variants I have seen it repeatedly compile them many times.
     
  5. andreiagmu

    andreiagmu

    Joined:
    Feb 20, 2014
    Posts:
    10
    I was pulling my hair out because of extremely slow shader variants compilation. I'm currently using Unity 2018.3.0f2, this slowness began for me since some versions earlier (from last year).

    But when I unchecked "Optimize Mesh Data" (in Project Settings -> Player -> Other Settings -> Optimization), this made a MAJOR difference to the build, far less shader variants were being compiled after I unchecked that option. And in subsequent builds, the cached shader variants compile way faster (as long the shaders aren't modified).

    I guess that option was renamed? Is "Optimize Mesh Data" equivalent to "Optimize Meshes" that was mentioned?
     
    Last edited: Jan 13, 2019
    Nigey likes this.
  6. andreiagmu

    andreiagmu

    Joined:
    Feb 20, 2014
    Posts:
    10
    Another thing I did that seemed to make a positive difference:
    In the Player Settings for Android, in "Other Settings -> Rendering -> Graphics APIs" option, I removed Vulkan support (I kept only OpenGLES3), I guess this halves the work from the shader variants compilation.

    I had also bought the asset "Shader Control" (by Kronnect), this asset really helped me to identify the shaders that were generating more variant permutations by having lots of keywords.
    Then, you can easily identify and deactivate unused/unneeded keywords from the shader, thus lessening the number of generated variants (and finally, lessening the build time).
    https://assetstore.unity.com/packages/vfx/shaders/shader-control-74817
     
    Last edited: Jan 13, 2019
  7. androidfejleszto

    androidfejleszto

    Joined:
    Dec 27, 2018
    Posts:
    4
    I just imported this:

    After that I wanted a build for my android, but compiling stucked there: compiling shader variants... xx of about 15552

    I started the build 10 minutes ago. My computer is lagging, and loud as a vacuum cleaner :S

    Using the latest Unity 2018.3.0f2
     
    Shorely likes this.
  8. multimediamark

    multimediamark

    Joined:
    Mar 17, 2017
    Posts:
    25
    Is there anywhere a response from Unity what is going on here? We used make builds in 10-20 min. Now it takes around 2 hours. All the unused shaders being compiled. We see it as a clear bug.
     
    Shorely likes this.
  9. KMarc

    KMarc

    Joined:
    Feb 5, 2018
    Posts:
    20
    Unchecking "Optimize Mesh Data" made a HUGE difference for me. However I don't understand what the trade off to having this unchecked might be.....
     
    Shorely likes this.
  10. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,344
    If you're able to reproduce this issue, could you please submit a bug-report, including the project to reproduce the issue, as described in this document:
    https://unity3d.com/unity/qa/bug-reporting

    Submitting a bug-report allows Unity Technologies to take a look at this issue.

    After you submitted the bug-report, you receive a confirmation email with a bug-report Case number. You can post this Case number (the number only, not the link) in this forum thread for Unity staff to pick up, in case they see it.
     
  11. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    356
    @multimediamark If you provide a small repro project instead of your full project, chances are way higher someone is actually looking at it.
     
  12. x-architect

    x-architect

    Joined:
    Apr 10, 2018
    Posts:
    2
    Unity, please give some kind of progress to the user during Compiling shader variants (HDRP/Lit)!
     
    MNNoxMortem likes this.
  13. Munchy2007

    Munchy2007

    Joined:
    Jun 16, 2013
    Posts:
    1,097
    Aren't the LWRP an HDRP still in preview and as such likely to be prone to problems like this still?
     
  14. MikeHergaarden

    MikeHergaarden

    Joined:
    Mar 9, 2008
    Posts:
    974
    Did anyone here already submit a repro case?
    I do hope Unity has one as they have let this problem become waaay too annoying.
    Our project is over 80gb and not the best/easiest repro case.
     
    id0 and MNNoxMortem like this.
  15. id0

    id0

    Joined:
    Nov 23, 2012
    Posts:
    218
    MikeHergaarden, they already know about it, they just can not do anything, because this is how HDRP works. That's just terrible. I'm really not understang WHAT THE HELL did they do to make the shaders compiles SO LONG?! I have not seen such things happen in any other engine. Last time I built my project 24 hours. After that, I'm not even trying, just hope that something will change for the better.
     
    Last edited: Jan 28, 2019
  16. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    316
    A couple of notes on our project using 2018.1.9
    * Caching does appear to work for us as subsequent builds were about 40% faster than a build with a cleared cache.
    * Unchecking the following reduced our build times considerably ( a further 33% from the above) as we were not needing these particular variants.

    upload_2019-1-30_17-36-57.png

    *Disabling "Optimise Mesh Data" in the player setting reduced times by a further 25% from the above.

    however I don't believe its actually a shader "compilation" issue. Rather I think its reading the shader variants to determine what mesh structures can be removed. I think the progress bar description is just red herring and Unity is doing what it should. I'm guessing its doing some sort of Material to Mesh to Shader cross check resulting in a lengthy process.

    My advice would be to enable "Optimise Mesh Data" for release builds and turn it off for Dev purposes.
     
    andreiagmu, MNNoxMortem and id0 like this.
  17. phygitalmadhouse

    phygitalmadhouse

    Joined:
    Jan 19, 2017
    Posts:
    7
    By this time , I read the entire thread , and still not finish this nightmare.(about 40 minutes now).

    I will try all stuff I read , and sure will comment results for research.
     
  18. phygitalmadhouse

    phygitalmadhouse

    Joined:
    Jan 19, 2017
    Posts:
    7

    1:05 hours and nothing , stuck in HDRP/Lit 1038 of 1040
     
    id0 likes this.
  19. minibotan

    minibotan

    Joined:
    Oct 4, 2016
    Posts:
    3

    That moment when he wrote this half a month ago and still no comment from him..

    p.s. i have the same problems..
    p.p.s Unity 2018.3.4f1
     
    fherbst, MNNoxMortem and Peter77 like this.
  20. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,344
     
    Deadcow_ and id0 like this.
  21. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    84
    I seem to have been able to get my builds down to a reasonable time. But it's still adding thousands of shader variants that I don't seem to need and it's roughly doubling the size of all my scene bundles. It seems to me that the Standard Shader is just grossly ineffective at whittling down which shader variants it actually needs.
     
    Last edited: Feb 14, 2019
  22. elettrozero

    elettrozero

    Joined:
    Jun 19, 2016
    Posts:
    216
    I't would be nice to have a word from Unity staff, at least like: guys it is the way it is.
     
    Deadcow_, BigToe, ihgyug and 2 others like this.
  23. Excelerator

    Excelerator

    Joined:
    Nov 14, 2013
    Posts:
    28
    I have had this terrible problem up until now. Reduced from what seemed like forever down to 1 minute build time. Here are my specs:

    2018.3.4f1
    Build Platform: PC
    LWRP
    "Graphics Jobs (Experimental)" Disabled
    "Optimise Mesh Data" Disabled
    "Instancing Variants" Strip All

    I also went to each scene and Saved out a shader variant asset after hitting clear button.

    It's a fairly big project taking up 9GB for the root folder. Lot's of art assets and even asset store purchases.

    So not all hope is lost! Unity team really needs to get on this and fix whatever is broken. They can't see this many complaints and act like there's no problem. This is a big deal breaking so many projects and wasting countless hours of frustration! We need video tutorials showing what Unity expects of us coming from earlier Unity versions which used to build fast! We need a user interface where you can click the "help" button and actually see something other than a 404 webpage.
     
    andreiagmu, BradZoob and id0 like this.
  24. hwaet

    hwaet

    Joined:
    Mar 21, 2015
    Posts:
    22
    I was encountering the same issues as above, and tried all the suggestions mentioned here with no luck. It was still taking a long time on around 20k shader variants.

    So I ripped everything out of the project and added each asset folder back in, one at a time. In my case, I had a folder of obj files that had "use external materials(legacy)" selected as their import settings under the materials tab. Changing that setting to the other option, "use embedded materials" with on demand remap, solved my issue entirely.

    My bet is that specific option reeeeeeally doesn't play well with the new unity version builds. Maybe this change, in combo with the suggestions above, will help?
     
    andreiagmu likes this.
  25. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,344
    If you're able to reproduce this issue, any chance you could you submit a bug-report, including a (mini) project to reproduce the issue, as described in this document?
    https://unity3d.com/unity/qa/bug-reporting

    Submitting a bug-report allows Unity Technologies to take a look at this issue. Unfortunately, the forum isn't really the place where UT is looking for bug reports (more).

    After you submitted the bug-report, you receive a confirmation email with a bug-report Case number. You can post this Case number (the number only, not the link) in this forum thread for Unity staff to pick up, in case they see it.
     
    Last edited: Feb 12, 2019
  26. fxtentacle

    fxtentacle

    Joined:
    Jan 23, 2014
    Posts:
    1
    FYI, the "OVR Performance Lint Tool" from the "Oculus Integration" asset keeps a list of FixRecords which contain pointers to game objects, materials and shaders. Apparently, that causes problems, because my build times increased by an hour after I imported the Oculus scripts.

    I modified OVRLint.cs around line 142 to add another button:

    Code (CSharp):
    1.         if (GUILayout.Button("Clear", EditorStyles.toolbarButton, GUILayout.ExpandWidth(false)))
    2.         {
    3.             mRecords.Clear();
    4.             mRuntimeEditModeRequiredRecords.Clear();
    5.         }
    6.  
    clicking that button and then running the "Build and Run" makes my project build within a few minutes again :)
     
  27. Excelerator

    Excelerator

    Joined:
    Nov 14, 2013
    Posts:
    28
    I could try, but problem is more on this being blamed on the users. Lack of documentation isn't a bug. Setting up your projects wrong isn't a bug.

    I'll see if I can make a mini project anyway.
     
  28. Nigey

    Nigey

    Joined:
    Sep 29, 2013
    Posts:
    1,109
    Oh just to add in. This happens when creating asset bundles too. I will get a bug report asap for it out of work hours.
     
  29. Excelerator

    Excelerator

    Joined:
    Nov 14, 2013
    Posts:
    28
    I think this is just a performance problem with old assets. I can't seem to reproduce this issue in a smaller project. In my big project, the loading bar goes backwards. In my reproduction, it progresses as one would expect.
     
  30. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    317
    I've just opened a new project with the Lightweight-VR Template and hit build for Android Daydream. Now I'm waiting for an hour and it's still compiling 17k shader variants. There's no user content in this project, only the default demo scene with a couple of objects apperently some post-process effect. I'm working on a high-end machine and usually I have build times of only a couple of minutes, even in complex projects. Usually it compiles a few hundreds or maybe two thousand, but not that many.

    I haven't tried the variant collection asset, but I already know that even if it works, it's ugly to work with, because it becomes invalid the moment I change any shader or material and then I have to compile them anyway, save the asset to disk manually and manually drag the asset into the graphics settings windows. Also I think that the empty template project should "just work out of the box", meaning I expect to simply hit build and run the demo in a reasonable time.

    Edit: Oh no, it's done compiling the 17k shader variants after roughly an hour, but now the loading bar jumps to the start and shows that there are more shaders and 10k more variants to compile. I would report a bug, if it wasn't the project provided by Unity without modifications.
     
    VR_Unity_Account, id0 and Nigey like this.
  31. VR_Unity_Account

    VR_Unity_Account

    Joined:
    Aug 17, 2017
    Posts:
    14
    I do wonder: What fundamentally changed between 2017 and these current versions that makes this happen? We're trying to work on an Oculus Go project and these build times are nightmarish.
    Tinfoil hat time: Is Unity trying to force users to use their new render pipeline by artificially making it so the original pipeline is terrible to work with?
     
  32. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,344
    Can you post the link from where I can download this project? I would submit the bug-report then.
     
  33. Excelerator

    Excelerator

    Joined:
    Nov 14, 2013
    Posts:
    28
    The new pipeline has the same problem.
     
  34. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    317
    From the Unity Hub create a new project with Unity 2018.3.5 and select the Lightweight VR Template. I then switched to Android and set Daydream in VR Settings. It’s been this taking really long to build because of the shader variants ever since last year.
     
    Peter77 likes this.
  35. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,344
    Nigey and Xarbrough like this.
  36. Excelerator

    Excelerator

    Joined:
    Nov 14, 2013
    Posts:
    28
    Peter77 likes this.
  37. hottabych

    hottabych

    Joined:
    Apr 18, 2015
    Posts:
    85
    The same problem
    Unity 2018.3.5f1
    Android / Vuforia
    Gradle, IL2CPP, .NET 4x, standard renderer
     
  38. See_Sharp

    See_Sharp

    Joined:
    Mar 3, 2016
    Posts:
    44
    I have a project with the post-processing asset from unity imported. Development build times take up more than 30min for each platform. There is not much imported aside from a few models and scripts.

    I compiled over 80+ unity projects in the past years and I can assure that none ever felt this slow. I'm gonna try all the alternatives posted above and will notify you guys.

    Also sad that Unity didn't respond yet :(
     
  39. dndn1011MR

    dndn1011MR

    Joined:
    Jan 23, 2019
    Posts:
    1
    This issue has been dragging around for 6 months, with no helpful response from Unity. Why is it taking so long for a response? I have encountered this problem twice now on two different projects, this is why I am here again. I have a very small project and the slowest part of making an Android build is shader compilation, every time. Now that the project is getting bigger this seem to be growing exponentially. We had this problem on a previous Android project as well. Now, Unity, I would like you to imagine, for a minute, how much this bug is costing developers. Perhaps what you need, is to be building games with your own tools in order to test your tools properly? The current strike rate on projects with this issue here is 2 out of 2.
     
  40. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    91
    I have the exact symptom. I have a scene that is using Unity's ARKit tools for image tracking.

    This is were it gets strange...
    My project builds in 4 minutes. But in the build settings, when I position a scene above the image tracking scene under Scenes In Build, the build takes 20 minutes. This happens with any scene, even a completely empty scene (just a camera with light). And it happens 100% of the time. When I place the scene, or any scene, below the Image Tracking scene then the build takes 4 minutes. In short, if the Image Tracking scene is at the top of the list in the 0 position in Scenes In Build, the build time is 4 minutes. If the Image Tracking scene is in any other position other than 0 in that list, then the build takes 20 minutes.

    Any ideas about what is going on?
     
  41. b4th

    b4th

    Joined:
    Jul 4, 2016
    Posts:
    64
    For us, Unity appears to be compiling every single shader variant for every material. So if two materials reference the same shader, Unity attempts to compile all variants of that shader twice. Reducing the number of materials and shader references in each scene seems to speed up the compile process slightly (compiling a shader seems to take longer the first time around, and speeds up on subsequent passes), but if a shader has 10,000 variants, like the Standard shader, it still seems like a crazy amount of processing.
     
  42. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    91
  43. SpiderJones

    SpiderJones

    Joined:
    Mar 29, 2014
    Posts:
    91
    I tried another solution. I created a LightweightRenderPipelineAsset and added it to the Scriptable Render Pipeline Settings field. I then had to update materials to LightweightRP Materials. I then checked "Optimize Mesh Data" and then set Instancing Variants to "Strip All" and removed all Preloaded Shaders. My build time was quick, less than one minute. After testing the build I discovered that ARKit does not support LWRP, so I removed it from the field and did another build, that build took 3 minutes...

    So I now I have -
    - "Optimize Mesh Data" checked
    - Instancing Variants set to "Strip All"
    - 0 Preloaded Shaders

    And my build times are now quick... So it seems that updating my materials to LightweightRP Materials fixed my long build times?
     
    TJD269 likes this.
  44. TJD269

    TJD269

    Joined:
    Oct 17, 2017
    Posts:
    12
    This was my worry too. Mines much better after tweaks (pretty much all the ones you have mentioned), but not amazing and I imagine it's because my project has been updated to literally like 20 different Unity editor builds over the past 2 years. I bet if I reimported all materials or something I could get it to go faster.