Bug Build time is sooo long

So, as the title says, we have a HDRP (hdrp 10.1) project stuck in out test pass because build time is a killer.

We use many 3rd party assets for shading (such as nature renderer, TVE, etc) they look good, everything works fine out of the box, editor runs smooth, we use some SS for our characters (in the house shaders), we got ready to deploy our test build and time for compilation is bad, just bad, 24 hours or more.

We tried reducing our shader variants,make some shader collections, prob our in house shaders were bad we stripped them out, every single step we could find in the web, nothing, just nothing. Unity Lit-forward (while is set to deferred ?) still on 99k variants.

 

Hi, I think this is not normal.
Yes the first build takes time but not too much time.

How did you optimize your shader variants?

 

Hi, soleron.

I remove -manually- keywords and create shader collections (clear asset, play whole game in editor, save the asset containg shaders is that the way?) . Many shaders have a debug display keyword, I remove that cuts in half the variants.

 

I typically use hundreds of materials with quite some variation not just lit shaders, in my project and it doesn't cause any issues. Builds fast and without issues. I have never seen more than 800-1000 shader variants. 99K variants is a ridiculous number.

Are you having redundant materials in your project?
Have you used 3rd party assets that could contain problematic materials/shaders?

This nice tool called "Resource Checker" could help you clean up your project. https://assetstore.unity.com/packages/tools/utilities/resource-checker-3224

I assume you have already tried these basic options.


The one above is a very simple project.

This one below is a lot more complicated. Hundreds of models and materials.

When I upgraded my project, there were more than 500 shaders tracked and no more than 800 variants
By clearing and opening all my scenes I got that:
31 shaders and 303 variants.


A different project. Also quite complex.
Before:


After:


So I can't begin to imagine how you managed to have 99K variants.

 

Just to add my two cents. In the past we have had 3rd party shaders that caused what I can only describe as a "variant recursion" that generated millions of variants due to badly placed ifdef, defines that seemee to stack with other definitions. So in this case I would start by eleminating the 3rd party assets one by one.

 

I also suspect the influence of 3rd parties and that is why I mentioned it in my previous comment.

I have only seen similar behavior with 3rd party assets and that is why I avoid importing their demos and their use in general unless it is from a reputed and dependable team.

 

Hi solereon, yes I already tried the graphics options in project settings.

I got: Currently tracked: 51 shaders 198 total variants.

What I do is clear the tracked shaders by clicking on clear, then I play the full game or well navigate through all my scenes an it goes up as high as 90 shaders, 320 total variants or little more. Then I save the asset.

Probably doing it wrong next. I save the asset on Preloaded Shaders, then on instancing variants I select strip all, when doing this the Scriptable render pipeline assets clearf off so I just select the main default of my project.

Variants still above 50k or more.

I have three main 3rd parties add ons, never imported their demo scenes, showcase shaders or similar. One of them pops a lot of warings and errors.

Mind me, I already tried one build without them. variants are not as high yes, still 10 to 20k. :/

 

The Vegetation Engine does tend to build large shader variants in build, you can join our server on discord maybe to find out more on that.

It's not always the case however, and this may be due to some users using L2CPP incremental building from the begining.

I honestly forget the name, and I know you mentioned you didn't want to rely you heavily on 3rd party solves for this. But as a last resort, she is a shader optimiser tool on the asset store that can often solve this. And also reduce issues like HDRP assets costing over 5GB of memory.

Edit https://assetstore.unity.com/packages/vfx/shaders/shader-control-74817
Boxophobic Discord

 

Mind you, the first build always takes more time than the subsequent ones.
i.e. The first build could be 45 minutes and the next ones about 5 or 10 minutes. It depends on the project of course.

AND maybe after optimizing the shaders, you should try a clean build if you haven't already.

 

Hi. Already got that asset, is nice, but is also heavy on the need to know what a shader is. Dont wanna just stripping keywords out of nothing some do need the multi_compile other will work with local feauture, question is how to know what is what.

With shader control I can see the keywords that adds variants, one of the keywords is Debug_display. As I not actually debugging graphics on my builds not really need that on, so I remove it with the tool, variants cut in half, I can tell this is a safe one.

My problem is with HDRP or the compiler just doing what it wants. Tried a build with three mats on scene, no transparecies, toggle off transparency on the HDRP asset (hovering on the toggle the tooltip mentions toggling this off should reduce build times!)). Build still logs 2k variants on Transparent backfaces :/

TVE is one of my 3rd assets. Will join you!

I try to go as vanilla as possible on the building settings, can't tell off if I doing one L2CPP any way to find out?

 

Hey, TVE has the built-in keywords for HDRP which are required + 2 keywords that are TVE specific for Leaf and Grass and 3 keywords for Bark and Prop. I'm not sure if the stripping is working correctly or not, URP usually generates millions of variants with the same configuration (URP required keywords + 2 TVE keywords)

Try to keep only one HDRP asset in the project and build with that. Not really sure, but I think unity is gathering all hdrp assets and generates all required variants based on that.

 

This actually could make sense since HDRP does build with all HDRP Assets whether active or not, usually, you'd have to remove non-platform target assets from your build.

I believe both @alexandre-fiset and @Rowlan picked up on this some time ago.

If all assets trigger all variants to build, then it may be a good A/B to build with one or two profiles in the project.

Nice thoughts on that!

 

We have a relatively big project built with HDRP, something like 350 gb of assets and 40 gb build size.

Our first build on a machine takes saveral hours, then all subsequent ones take less than an hour (with only script /minor changes it's less than 10 minutes). We found really important to have a fast SSD and a lot of ram (32gb+) on our machines to get the fastest build possible.

Plus we only keep one HDRP asset reference at build time. Under Quality > HDRP, you can see them listed on the right. The best is to keep only one reference when building, that means deleting all quality levels referencing assets you do not wish to use. This affects both build time and runtime ram usage. Two can work too, just as long as it doesn't differ greatly from your other asset. More details can be found in that thread.

I believe a lot of work has been done in that regard since I logged the issue, with most fixes present in HDRP 14. But I haven't tried Unity 2022 yet to test these out.

 

If you upgrade after URP, try delete URP Asset. It was very help for me.