Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

compiling shader variants takes forever when building for Android

Discussion in 'Android' started by strongbox3d, Aug 5, 2018.

  1. malrion

    malrion

    Joined:
    Dec 23, 2016
    Posts:
    10
    I just want to add my voice here -- I'm seeing incredibly long build times when building with
    • Unity 2018.3
    • VR And Android as build targets
    • Interestingly enough, it's the speed tree shaders that are taking a long time for me to build as well, similar to @ASIM-SENYUVA
    • It builds much faster when targeting desktop VR platforms.
     
    tigerleapgorge, ROBYER1 and JasonDaze like this.
  2. PanayotCankov

    PanayotCankov

    Joined:
    May 28, 2018
    Posts:
    16
    I am using Unity 2018.3.7f1, VR LWRP, just switched to Android platform and tried Build and Run with attached Oculus GO device. Compiling 16848 shaders. Takes forever.
     
    tigerleapgorge likes this.
  3. FloBeber

    FloBeber

    Joined:
    Jun 9, 2015
    Posts:
    166
    It takes very long only the first time, shader variants being cached in Library/ShaderCache.
    Then, if you don't change LWRP settings, it builds faster, but still takes much longer than when using the Legacy RP.

    After a few months experiencing with LWRP in VR (Oculus Go), I decided to switch back to Legacy. Performances are pretty much the same in the end. The big difference being: MANY more AssetStore options and much faster build time. Switch from one RP to another is not that complicated in the end. It pretty much only depends on how many Materials you need to convert (Legacy to LWRP being easier thanks to Edit > Render Pipeline > Upgrade Project Materials)

    In my opinion, you might want to stay away from LWRP in VR for now.
     
    tigerleapgorge likes this.
  4. PanayotCankov

    PanayotCankov

    Joined:
    May 28, 2018
    Posts:
    16
    @13Flo thank you!
    That's pretty much the state at my end. The LWRP with VR is the only project type with "VR" in the name but I guess it will take some more iterations of Unity on it before I jump on that train. I believe the "LWRP VR" + "Oculus Quest" will have a potential to drive a switch to LWRP but honestly I wouldn't wait half an hour to setup a new project.
     
    tigerleapgorge and FloBeber like this.
  5. Carocrazy132

    Carocrazy132

    Joined:
    Aug 16, 2015
    Posts:
    15
    I'm getting this issue in 2018.2.15f1. It's ridiculous that people in this thread are saying this isn't a bug. I'm not using LWRP or HDRP just the standard shader that I added a wireframe option to. It is absolutely ridiculous that I have to wait 20 minutes for the barely-edited standard unity shader to compile. This needs to be fixed this is absolutely unacceptable. This type of "We don't care" / "That's not a bug even though it's crippling to your company" attitude from Unity devs is going to push me right over to the unreal engine. I need decent compile times if I'm going to continue using your engine in the future.

    You call the network API obsolete and you don't give us a new one, making server authoritative unity games a huge pain in the ass to create.
    You call the graphics API obsolete and you give us one that half works, no one can rely on LWRP or HDRP and writing a custom shader for them not using shadergraph is a 100% undocumented hell.
    Compile times are skyrocketing.

    This is unacceptable. You guys need to get it together because I'm sick and tired of running into completely broken features, and then seeing snarky responses from Unity devs when I try to google the issue.

    Fix it.
     
  6. FloBeber

    FloBeber

    Joined:
    Jun 9, 2015
    Posts:
    166
    I agree this is very annoying and frustrating, but that's the way GPU works unfortunately.
    Unreal Engine has the exact same issue. Create an empty new project there and you'll also have to wait for all the basic shaders to compile.

    I don't remember Unity saying the Legacy RP was "deprecated". LWRP in 2018.2 was in preview. As far as I know, LWRP will be out of preview in 2019.1 which hasn't been released yet. So you can't really say they "gave it to us". It is just an early feature.

    This IS totally acceptable to release early versions of LWRP so we (game developer) can try it and give feedback. If you're sick and tired of Unity, cancel your paid subscription and switch to another engine (Godot?). But your anger will not solve anything.
     
  7. Homicide

    Homicide

    Joined:
    Oct 11, 2012
    Posts:
    657
    I've been wondering whats up with this too. It seems every time i build on android , it literally *compiles* AL the shaders again and again and again. Seems absurd to me.
     
    ROBYER1 likes this.
  8. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,450
    Having this issue too on 2019.1.5f1 and it's taking ageees to rebuild the app at any time for testing
     
  9. demonixis

    demonixis

    Joined:
    Aug 20, 2013
    Posts:
    185
    It seems to be fixed in the latest 2019.2 beta. It was a known issue (starting 2019.2) and it's no more present in the last changelog. I hope it's fixed..
     
    ROBYER1 likes this.
  10. munger

    munger

    Joined:
    Aug 13, 2013
    Posts:
    2
    Has anyone looked at their cpu and disk usage when compiling shader variants? My disk isn't hit much at all (< 1mb/sec), Unity.exe is taking a single core (~12%), and all 25 or so sub-processes like UnityShaderCompiler and conhost sit at 0% usage.

    v2019.1.7f1 and targeting the Quest. There are around 25k variants listed at the moment, but it seems to go through several lists, and it seems to back up and recompile a few hundred variants on occasion.

    Edit: Shader compilers for "Lightweight Render Pipeline/Lit" don't use the CPU and takes forever. Shader compilers for "Lightweight Render Pipeline/Simple Lit" loads all the shader compilers at ~8% CPU usage and it completes in seconds.

    /ben
     
    Last edited: Jun 20, 2019
    ROBYER1 likes this.
  11. ROBYER1

    ROBYER1

    Joined:
    Oct 9, 2015
    Posts:
    1,450
    I am interested to know if this is definitely improved in 2019.2?
     
  12. eduardo-pons

    eduardo-pons

    Joined:
    Mar 31, 2009
    Posts:
    176
    2019.2.4f1 and OSX build taking 3+ hours versus minutes on windows.
    Same project.
    Any fix on the horizon?
     
    ROBYER1 likes this.
  13. FIFTYTWO

    FIFTYTWO

    Joined:
    Oct 21, 2014
    Posts:
    49
    We have updated a project to Unity 2019.2.6f1. We did not upgrade PostProcessing and it still uses PostProcessing v1 stack. Android build has Vulcan and GLES 3 enabled. So, UberShader in PostProcessing has about 15000 variants. When I build on Windows it takes some minutes to complete build on clean project, the next builds are much faster. But it seems Android build on macOS is going to last forever. Now I'm waiting for completion and UberShader progress stuck on 8000 of 15000 variants, it is not changing during an hour already...
    PS: macOS completes the following targets in human reasonable time: OSXUniversal IL2CPP, Linux64 Mono, iOS, tvOS. So, our problem is about Android only.

    UPD:

    Android builds freeze forever on my mac when I use Unity 2019.2.6f1 or 2019.2.8f1. When I force quit Unity and start build again it moves next but in the end I receive unknown shader error and I see pink & black image on the device. I rolled back to Unity 2019.2.3f1 and it works great for me. All builds pass without freezing. PostProcessing works OK.
     
    Last edited: Oct 6, 2019
  14. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    @FIFTYTWO can you please submit a bug report?
     
  15. FIFTYTWO

    FIFTYTWO

    Joined:
    Oct 21, 2014
    Posts:
    49
    I can't reproduce the bug anymore, it seems it was fixed with new Unity versions
     
  16. AhJun

    AhJun

    Joined:
    Dec 30, 2019
    Posts:
    14
    looks like the fix didn't make it in for 2019.3.0f5
     
  17. demonixis

    demonixis

    Joined:
    Aug 20, 2013
    Posts:
    185
    It depends of the target platform for me. On Windows Standalone, it's almost fine. Compilation time is about 2h the first time, then 15mins depending of what changed in the project.
    On Android it takes 7 hours the first time. I'm still building the UWP version (2h for now).

    I've only one 3D Api selected per platform, and a reduced set of shaders.
     
  18. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    @demonixis compilation for DX11 (I assume that's your API for Windows Standalone) is always faster than GLES2/GLES3/Vulkan, just because it goes via D3D compiler for those APIs as well, but then does extra steps to produce GLSL or SPIR-V.
     
  19. demonixis

    demonixis

    Joined:
    Aug 20, 2013
    Posts:
    185
    Thanks for the clarification. On UWP it's DX11 too but it took almost 4 hours, any idea why? I check all my shaders, I only use one URP/TerrainLit and many URP/Lit. The shader compilation took a very long time for the terrain.

    About Android, is it a good idea to uncheck GLES 3.1/3.2 and just keep GLES 3.0? Is compilation faster without it?
     
  20. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    No, this will not affect compilation time. Only the API list matters.
    I don't know. I'll ask someone internally, and get back here if I find anything :)
     
    ROBYER1 likes this.
  21. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    @demonixis I'm not sure, but it sounds like it builds a double set of shaders there. Can be for desktop and XBox, for example. You can check this in the editor logs, it should say, for which platforms it built shaders.
     
  22. demonixis

    demonixis

    Joined:
    Aug 20, 2013
    Posts:
    185
    Ok thanks, I've reduced my shaders count. Is it a good idea to use Shadergraph instead of URP/Lit? Maybe less variants to compile?
     
  23. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    I think it's best to ask either in the shader graph or in the URP subforum as a separate question :)
     
  24. demonixis

    demonixis

    Joined:
    Aug 20, 2013
    Posts:
    185
    You're right ;)
     
  25. Unplug

    Unplug

    Joined:
    Aug 23, 2014
    Posts:
    256
    maybe anyone can help me. I'm on unity 2017 LTS and despite that Unity tell me i use about 235 variant, it compile millions of file in shader cache folder and the build time is insane (left overnight and still didn't stop, ran out of space on the SSD because of having millions and file created). I'm using standard shader and vegetation use manufature / afs. terrain use CTS. Nothing crazy or special
     
  26. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    @aleksandrk
    Could you guys please add a default shader settings? Or it has been implemented already?
    It has been asked since standard shader was introduced many years ago.
    Many of us have no use for the standard shader since it is too heavy for low end mobile, and even 1 simple test cube that you forgot to chance the material means the shader will be included in the build. Always have to do a lot of digging around to find the culprits.
     
  27. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    What do you mean by this, exactly? :)

    There's Build report, which logs, what went into the build.
    It's also quite simple to implement a script that scans every material in your project and checks/prints the assigned shader.
     
  28. 00christian00

    00christian00

    Joined:
    Jul 22, 2012
    Posts:
    1,035
    Every built in primitive(cube, plane,etc) and all imported models by default have a material with standard shader applied.
    It would be nice if instead of the standard shader there was a preference that let you set the default shader for all of these.

    It's a non-issue for the final product, but while testing it makes everything slower, because you have make sure to change the material for everything, or a single primitive/test model can bring down the fps to something unusable.
     
  29. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    I think you can submit this as a feature request :)
    Just note - it creates an object that has a default material assigned. This can be overridden for SRPs, so if you're on a custom SRP, you can provide your own material right off.
     
  30. yoonitee

    yoonitee

    Joined:
    Jun 27, 2013
    Posts:
    2,363
    I tried downloading the Oculus Integration and when I tried to build one simple scene (lip sync demo) for Oculus 2 (Android) it is taking ages with tens of thousands of shader variants. (Unity 2019 LTS as recommended)

    Is there a work around?

    I already disabled Mesh optimisation.
    And I replaced #pragma multi_compile with: #pragma shader_feature in all the shader files which helped somewhat. But still tens of thousands of shader variants. (Mostly Standard Shader)
    Probably will take an hour to compile.

    This doesn't seem very efficient to me.
     
    IS_Twyker likes this.
  31. zackblack

    zackblack

    Joined:
    May 17, 2015
    Posts:
    76
    Every year or so I come back to this issue, going from "Oh I'll just leave my PC building overnight" to "There HAS to be a way to make this more efficient." Went through multiple videos and guides yesterday to try to decrease build times. The thing that no-one has addressed though is: How did this used to work prior to 2018? What's the difference between the legacy render pipeline and URP that changes the build times from 5-10 minutes to 4-6 hours!?
     
    IS_Twyker likes this.
  32. LordCafe

    LordCafe

    Joined:
    Aug 26, 2015
    Posts:
    35
    When HDRP come up to Unity even in thoose early days it was not taking like 2h to build a scene with a 2x2 plane with a single cube... was only about 30s-1m to build... now it's impossible to work even URP is taking years to build i would never think to switch to HDRP since this bug is still around... better to go the the first HDRP/URP version of unity since in that time this did not happen...

    I hope they fix this... already did all 'fix' and recomendation by unity devs and unity 2022 still loosing for unity legacy even with all thoose asset store shaders to increase graph, no ideia why the default shaders take years to build while putting thousands of asset store shaders in legacy unity had literraly zero effect on building time.
     
    Daven88 and Immu like this.
  33. TheBlueLines

    TheBlueLines

    Joined:
    Mar 25, 2020
    Posts:
    1
    10 hours in... (I have the same bug)
    upload_2023-8-25_8-31-42.png
     
  34. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,983
    @TheBlueLines this won't finish compiling, you can safely hit cancel instead of waiting. 3.5 million variants is not feasible.
    Please update to the latest patch release if you're not on it already and follow advice from the post here to reduce the number of variants.
     
  35. snipshotmedia

    snipshotmedia

    Joined:
    Nov 13, 2023
    Posts:
    10

    This is still an issue