Bug Memory and performance issues related to shaders

Hi,

we are facing performance issues with addressables that are linked to the shaders. We have custom hierarchical structure of models in our scene. The lowest HLODs are always present in the scene, so they are not in any addressable groups. Higher HLODs and their appropriate LODs are streamed based on proximity. All the models are using the same shaders with the same variants. However when the models are added to addressables then the shader is inserted in the group as well. Once the model from the addressable group is loaded there is massive stall due to the Shader.Parse() (even though the shader is already present due to scenes objects with the same shader). After that the shader takes up to multiple GB in the RAM. Without addressables all the shaders occupies around 50-60MB, but with addressables this ramps up to something like 4-5GB (this behavior is only present in standalone build or when play mode script is set to Use existing builds)

After reading multiple forums and posts about addressables I have come to the understanding that addressables don't know what is going to be available once loaded and therefor the shaders and other stuff has to be included as well. Based on that we have tried adding the shaders to the always included list, but this does nothing and the problem persists. After trying multiple workarounds the only solution that is currently working is to create standalone addressable group just for shaders and shader collection referencing them. During build we are injecting our custom shader processor that strips the shaders based on predefined variants. During the game initialization all the addressable shaders are loaded and then pre-warmed. This solution is working, there is no hiccup due to the Shader.Parsing and there is no memory penalization however it is extremely inpractical, because we have to go through all the shaders and manually determined all the variants in order this to work properly.

 

Hello, I am facing exactly the same problem. Having shaders take a ton of RAM memory when builded via addressables, and the more scenes/content I add the my addressable build, the more memory the same shaders takes on the RAM ! Even if some of these shaders are only used in one spot, they 10x their size in my complete build.

I will try this !

Would you mind being a bit more explicit about this, been looking to strip shader variants but I am unsure how to correctly identify unused keywords.

Thank you for your time !

 

We are doing inverse process. Identifying only the variants that will be used and during the strip process we discards the ones that are not listed under the used variants.

 

How do you identify the used ones?

 

Hey ! I am currently working on making my own solution. The lack of resources on this is disturbing since I believe this is a major issue of addressables.

There is a lot going over my head and this is not a completed solution but I'll still share it with you.

1. I make my build and with the Memory Profiler package, I can know which shader takes a ton of memory.
2. After running my scene, I go into Graphics Settings and see all used shader variants and save them to a Shader Variant Collection. Instancing Variants to Strip Unused.
3. I make an editor script that implements IPreprocessShaders interface and implement the OnProcessShader method which is called BEFORE unity compiles any shader. See this 2018 unity blogpost for more info
   
4. I made a scriptable object class (ShaderScriptingData) to store shaders for which I want to MANUALLY control which variants are compiled. I link Shaders to Passes which contains combinations of keywords I found in the Shader Variant Collection
   
5. In the OnProcessShader method, I check if the shader is in my list of manually controlled shader, if it is, I check if the pass that is about to be compiled is a pass I am interested in, if yes I check if the combinations of keywords of the about-to-be-compiled variants correspond to one I am interested in. In short, if the combinations is identical to one of mine, I compile the shader, else, I discard it.

With this workflow, Unity do whatever it wants with shaders that are not in my list and it let me manually controls variants of some specifically selected shaders. I must say I am still working on this at this time but I went from 1GB to 100mb in no time.

Drawback : If you change the fog, lighting or anything that shaders are interested in and you don't update the wanted shader variants list, you will be lacking needed variants in game. So this is only a time-consuming workaround but it was deeply needed here for our project.

One of many examples I have done in my project :



I haven't tested on many devices yet, so far so good but I am scared this workaround could break something on some devices/platforms. Anyone have any input on this ?

 

really appreciate the reply!
Will try to implement a method similar to this!

Will also test on a couple platforms and see if I see any issues if I manage to implement it.