ShaderLab memory doubled in size

Hi,

We recently made an update to our game built for iOS and afterwards we had to make a hotfix because of some bugs. At this point I was running the game in XCode and I noticed that our memory had increased by around 100mb.

We did not add any new assets in the update, so this could not be the problem. I immediately built a debug version and connected it to the profiler to see where the increase came from. There was no increase from our assets as suspected. However the "Other" category in the profiler was suspiciously big, when I opened it I see that "ShaderLab" takes 140mb.

We had switched from using the resource folder to using asset bundles in the update so I immediately blamed them and the multiple copies of shaders that is loaded because the shaders gets included in each asset bundle, this seemed logic, more shaders == bigger ShaderLab.

Unity has some pretty good documentation on the asset bundle system but not when it comes to troubleshooting or common bugs, and it seems that not that many Unity developers is using the system because there is not that many posts about them compared to other unity bugs and problems.

But, after some research I found an article from Unity that describes how to get rid of the loaded shader copies. The solution was to have one asset bundle with all the shaders used in the project and then having the other bundles depend on the "shaderAssetBundle".

Cool i think, lets do that...

...but how the hell do you add the Standard shader to the bundle? It is not included in the project, not so you can select it anyway.

I knew you could download all built in shaders from Unity, so I download the shaders for our Unity version (2017.2.0p4), add it to the project and to the "shaderAssetBundle".

Now comes question nr2, do I need to set all the materials in the project to use the imported standard shader? Probably... but I make a new build to see if the shader copies are still there, and they are. We have over 1800 materials in our game so I had to write an editor script that would change the shader on all materials.

Then I make another build now that all the materials are using the imported standard shader. I'm confident this will work, the game is starting and I look at the memory in xcode, GAAAAAAAHHH it's F***ing higher than before!

I connect to the profiler and check the shaders, the copies are gone but "ShaderLab" is now taking 181mb, wtf.

I'm still blaming the asset bundles at this point, it must be the asset bundles...So I revert back to the previous version of the game, the one where we still used the Resource folder, I want to confirm it's the asset bundles. I create yet another build and check "ShaderLab" in the profiler, it's 60mb, that's 80mb smaller. Now I'm positive it's the asset bundles, so we make a decision to revert back to using the Resource folder just to get the hotfix out to our players as fast as possible.

We revert our resource loading code and I create a new build to check that the memory is back to "normal"(I have no idea how big ShaderLab should be on iOS), WTF, it wasn't the asset bundles, ShaderLab is still 120mb, 60mb bigger so it has only decreased 20mb.

Me and my colleague's have lost many days now trying to figure out why ShaderLab is taking so much memory, why did it increase 60-80mb between game versions? We do not have any more ideas and we need help from someone from Unity that knows what is happening behind the scenes, is it possible to debug somehow what is in ShaderLab?

During one of all our tests we made I took screenshots from the memory profiler, I'm attaching them to this post, maybe someone can spot something that we have missed, at least you can see the huge difference on the “ShaderLab” memory. The screenshots are between version 1.0.12, this is the version that used the resource folder and version 1.0.14, this is the hotfix version using asset bundles and this specific build is from where I tried having all the shaders in one "shaderAssetBundle".


I hope someone from Unity or from the community can help us with this one!


Here is a summary of some things


1. Version 1.0.12 used the resource folder to load assets. Shaderlab is taking 60mb in this version.

2. Version 1.0.13 and 1.0.14(hotfix) is using asset bundles to load assets. Shaderlab is taking 140mb in these versions.

3. Version 1.0.14 reverted to Resources to load assets. Shaderlab is taking 120mb in this version.

4. We are using shadervariants in the game. But we have tested with and without them and it has a very small impact on the size of ShaderLab, around 2mb only.

5. We reference the shadervariant file in a script and use “WarmUp”.

6. We do not include any shaders in the “Always include shaders” list under Graphic settings

7. We have not added any new shaders between the versions

8. Whenever we load a scene we load a empty scene in between and call Resources.UnloadUnusedAssets()

 

A couple of questions

1. How do you include the standard shader in a bundle? Is there any other way then downloading and importing it?
2. When you import the standard shader and include it in a bundle it seems that it includes all/more variants?
3. When you DO NOT import the standard shader you get copies of the shader from each bundle. Is it possible to setup the structure of the bundles so you do not get the copies?
4. If you include a shadervarant file in a bundle, will the assets use the shaders in the shadervarant file?
5. Does the size of the shaders have any direct effect on the "ShaderLab" memory?

 

Attaching 4 screenshots from two of version of the game

1.0.12 - This version is using the resource folder to load assets and we "Warmup" a shadervariant file when the game starts.

1.0.14 - This version also use the resource folder to load assets and we "Warmup" a shader variant file when the game starts.

As you can see 1.0.14 uses less shaders and shader memory but ShaderLab is double the size compared to 1.0.12. Where does the memory come from? I want to know what is in ShaderLab!

Can memory used by ShaderLab be unloaded somehow?

 

After setting the Shader Stripping manually we got our memory back and we even decreased the memory consumtion in "ShaderLab". It is still a question why it increased in the first place between our versions of the game. We did not add any new shaders or materials with different settings that would have produced more variants.

 

I also got shaderlab take big memory, where can I find the solution and above setting UI?