Official Dynamic shader variant loading

@aleksandrk

Thanks sounds good. Waiting for the backport to 2021 LTS
Updated the project from 2019 lts to 2021 1.5 Weeks ago increasing the build time by xx% because of shader variants going into the millions.

Any idea when we can get a fix for 2021 LTS?

 

This looks to be a change to runtime shader loading. So it won't affect compile times at all.

 

Do we have to do anything to enable it or it just works straight away?

 

Just to make sure, it will be available in 2022.2.0+ as well? Aka all versions that will come from now on? Dynamic shaders will work on webgl as well? I am already waiting for Monday to try it out

 

Greetings. So I am working with Unity 2022.2.1 now ( tho same goes with all previous versions) and addressables package. So here I have a problems, this is workflow that I expect to work:
1) Go to Player Settings -> Graphics -> Save Shader Variant Collection.
2) Add that SVC file to addressables.
3) Build addressables with addressables scenes.
4) In Player Settings -> others I set shaders default chunk size to 16 and default chunk count to 2
So expected behavior is, then when scene starts and loads, shaders loaded to memory will be only the needed ones. However what I get is shaders using over 300 mb.

Spoiler: Memory profiler report

With custom shader striping code I am able to reduce it to ~100 mb.

So my question is it I am doing something incorrect? is it problem of addressables? Or there is something I am missing to use dynamic shader loading?

Extra info:
Platform: WebGL
43 shaders and total 91 variants used.
In editor URP Lit uses 0.8MB

 

First of all, thank you for your time,

I know that this is more problem with Addressables than dynamic shaders, but I am very thankful for your help.

After reducing chink size to 1mb the memory size of URP/Lit got reduced:
- Universal/Lit shader is in Packed separately addressables group together with Shader variants collection: to 67.6MB.


-Universal/Lit shader is NOT ADDED manually to addressables group, but Shader variants collection is added manually into addressables group: to 273.8 MB.


From these results I could guess that Addressables does load all possible shader variants by specific keywords. When everything is in same Addressables group it does load less. But regardless of that the size of loaded shaders in memory is still way too big, from what I can only guess that dynamic shaders loading does not work fully with it.

Example in Editor the URP/lit reports at about 0.4 MB of memory (as editor does not use addressables directly.).

 

On iOS, this literally slashed my app's total memory useage in half. Fantasic work, crew.

 

Firstly, I'd like to say that I've very limited information about how shaders are processed, so my question may make no sense (understandably): Could you explain why decompressed chunks stay in memory? If we're done with them once they are uploaded to the GPU, I couldn't understand why they remain in the memory. If we need them while the shader is in-use, then if we set the maximum number of chunks to 1 and multiple shader variants (chunks) are used by the scene, won't these chunks fight for the single available chunk slot, causing an infinite loop?

 

OK so it's like cache prefetching in a way, got it! Thanks for the explanation 0.02 ms difference per variant with default settings sounds like a very insignificant con so the pros heavily outweigh the cons IMO, thanks for the new feature!

 

This is great, we are fighting with shader variants and this helped to reduce our memory usage on scene load by 50%

Would you be able to give more detail on the maximum chunks value?
the maximum number of chunks that are kept decompressed simultaneously for each shader at runtime.

Kept decompressed where? In memory I assume?
What happens to chunks which are no longer needed or the number of chunks needed is > maximumChunks?
If unneed chunks stay in memory are they compressed again to save memory?

Finally, is there any way we can gauge a good chunk size and max number of chunks?
It sounds like once stuff is loaded into memory its not unloaded from your other comments. So if we know we have lots of variants, because we aren't stripping them yet, setting the chunk size to 1MB and 1 chunk max would theoretically load in the fewest variants we need and avoid the bloat from the unused variants?

 

Thank you for the information, that really helps!

 

Hi @aleksandrk

Will the "dynamic shader variant loading" help improve build times? (Not at runtime) .. sorry I'm new to all this.

Can we trouble someone on the Unity team to make tutorials on how to reduce the shader variants compilation time? Or at least best practices?

I've been trying to make a build the whole day .. my pc is still building.. it's probably at 9 hours now.

I'm clearly not doing something right..

Unity 2021.1.28f1


Thanks for the help

 

You're on 2021.1, not 2021.3. 2021.3 is the LTS (Long Term Supported) version that comes out after the end of the year and gets the long-term maintenance and sometimes backported features. So you'll have to upgrade to that version if possible. Just make a backup of your project first in case there's something in your project that ends up having obsolete API in 2021.3 and can easily revert if you need to.

 

I'm going over the material now. Seems I'll need to update to 2021.3 to access the new tools. Crossing fingers nothing breaks. Thanks for the links !

 

Hi, first I have to admit I have little knowlegde in shaders so it is possible that my question might not make much sense
I was wondering how "Dynamic shader variant loading" and "shader variant prewarming" can work together properly?

Our game lets users upload their own 3d models. I ran the game, loaded those models, created a shader variant collection, and added it to Preload shaders in Graphics Settings or call WarmUp() in Awake() to prewarm it.
However, variants in that collection are taking nearly 300mb memory. We've tried to strip the variants, but in our case it's really hard to strip more.
After seeing this new feature "Dynamic shader variant loading" I am testing it now. With my collection placed in Preload shaders in Graphic Settings, I set chunk size:16mb & chunk count:5. The build indeed uses less runtime memory(220mb) than before.
But what I can't understand is:

From what I read in How Unity loads and uses shaders doc:

1. When Unity loads a scene or a runtime resource, it loads all the compiled shader
   variants for the scene or resource into CPU memory.
2. By default, Unity decompresses all the shader variants into another area of CPU memory. You can control how much memory shaders use on different platforms.
3. The first time Unity needs to render geometry using a shader variant, Unity passes the shader variant and its data to the graphics API and the graphics driver.
4. The graphics driver creates a GPU-specific version of the shader variant and uploads it to the GPU.

and:

• To avoid visible stalls at performance-intensive times, Unity can ask the graphics driver to create GPU representations of shader variants before they’re first needed. This is called prewarming.

My understanding is that prewarm happens in 4(creating a GPU-specific variant), but not 1(loading variants into CPU memory).

Will "Dynamic shader variant loading" make some variant not loaded? (my 300mb reduced to 220mb) If so, is my prewarm still working effectively? (if the variants are not in cpu memory, how do I prewarm them?)

It would be awesome if you would help me figure it out. Thank you for reading!

 

Anyone care to share their suggested values for relatively simple games, like "Boat Attack", targeting iOS, Android and webGL?
for PlayerSettings.SetDefaultShaderChunkCount & PlayerSettings.SetDefaultShaderChunkSizeInMB

 

I was hoping to be able to discard my current (messy) solution for optimizing runtime shader memory, but after setting the two values to 1, it seems to have made a difference, but not enough of a difference. I would expect memory to be around 2.7 MB from my testing based on what actually should be loaded in, but I am getting around 7 MB per shader instead. Is it keeping variants around that aren't needed anymore? Or maybe somehow loading more than necessary? Is there anything I can do to improve memory usage further here?

@aleksandrk

 

Ok, still not sure how that adds up though. Are you saying the parameters as in if the shader has a uniform of 100 float4s, then it will allocate sizeof(float)*4*100*shaderVariantCount?

 

I see. Would love to see that get reduced in the future to only for variants loaded in, as that sounds quite possible. We have many variants that a player will never use due to never using those associated graphics settings, as I'm sure many other games do. Thanks.

 

Hi folks! From my understanding the dynamic shader variant loading should work out of the box with unity 2022.3 too, right?

If yes then, well, the size of my URP/Lit shader is 1.9 GB in memory

If not then please let me know what is the extra step required to make it work.

In my project I give the player the option to switch from forward to deferred, I have a few URP renderer assets in the URP asset and i set the camera to use different URP renderer assets. I'm saying this because indeed, I surely get the forward+deferred variants in my build, but dynamic loading should take care of that and load only the variant that is in use (forward or deferred ones based on what player chose in options) , right?

Thank you!


Edit: yup aight, the thing seems to work, i set the default chunk size to 1mb and the size of the lit shaders got much smaller indeed:


https://gyazo.com/39c6ebcae7e316201d1a83dfcf1f2aa6

However, could you give me any tips on making it even smaller than this?

 

Hi @aleksandrk

In my case: I have about 2000 variants in svc, the memory usage increased 300MB after pre warmup, to avoid stalls. Just like @qwert024 posted before.

Will "dynamic loading" unload what I have prewarmed before, and cause stalls when I use some variant(like creating a model of effect)?

In short words, can I use both "svc prewarm" and "dynamic shader variant loading" in my game ?

 

Yes, I said in my post that "i set the default chunk size to 1mb and the size of the lit shaders got much smaller indeed, However, could you give me any tips on making it even smaller than this?"

Because the size is still too big, as you can see in my screenshot

 

Wdym by stripping? I'm pretty sure the unused variants are stripped.

So you're saying 0.8gb for shaders is the best that can be done?

 

Well that sounds about right, i have around 10000 materials.

Can I have your best tips to reduce variant count please?

 

I'm having trouble figuring out what are good values to enable this feature. I don't feel like the docs or this thread provide enough info so I can decide what are good chunk count and size values for my project. Any pointers?

 

Appreciate the info, but this prompts another question: Could I have other goals when using this feature?

From the way it is presented and my understanding of it, it seems it could potentially greatly reduce ram usage, with a tradeoff of maybe making hitches worse (a bad configuration would introduce extra ones, a good would maybe make the "existing" ones last a bit longer). Are there configurations where a completely different goal (and different tradeoff) is achieved?

 

Can I use dynamic shader variant loading to bypass the entire shader stripping/building process that occurs when creating a player?

It would be tremendously useful for me if this is supported, as it would significantly reduce the iteration time when testing a player. When the shader variants are loaded and compiled asynchronously at runtime, it would not pose much of a visual problem for development builds for me.

 

Hi, thanks so much for this feature @aleksandrk !!
And thank you SO MUCH for backporting it.

I've carefully re-read the thread as well as documentation multiple times but there are a lot of assumptions that I do not have the prerequisite knowledge for, nor are there any resources linked to learn more, and I don't know what to search online for.

Could you please provide more information regarding these settings for those of us who aren't as knowledgeable?

The only thing I've really gleamed is:
Try 1MB - 1 Chunk. If it stutters, try increasing chunk size.

CHUNK SIZE
1. The default was 16MB, and the suggestion is to try 1MB.
• Can you please explain the large difference from the default--what were the benefits of the default 16 MB vs 1MB?
• What are the tradeoffs between a larger chunk size and smaller chunk size?
• Is there any reason for a user to choose 4MB? Or 8MB? Is there any reason to set a larger size like 32MB?
• My standard shader is taking 500 MB at runtime -- what would be a potential recommendation for this?
• Does this affect when shaders get Unloaded as well?

CHUNK COUNT
2. Your recommendation is Chunk Count 1, seems like a big change from Unlimited. Could you explain?
• Why is this recommended to try? And why is the recommendation to increase Chunk Size vs Chunk Count?
• Could you please explain in what scenario would a user set Chunk Count 2? Or 4? Or 8?
• Does Chunk Count change performance?
• Is there any reason to set unlimited chunks with this feature?

PERFORMANCE
3. The chunks are decompressed -- I assume CPU usage? Could you please give a little more info about this?
• Does increasing Chunk Size increase or decrease the decompression CPU usage? You mention it might be negligible? • Is this cost per-frame, or is it only upon loading a new shader?
• Does increasing Chunk Count increase or decrease the CPU usage?

I apologize for all the questions. The settings are a little opaque to me, perhaps because I don't have the prerequisite knowledge nor do I know what to search. I want to make an informed decision when setting these. I will do further testing for my project as well -- but my build takes hours, and changing these settings around takes a lot of time.

Thank you! Please take your time to answer and thank you for the hard work!!

 

Thank you so much for the answers. Super super helpful, cheers!

I've written a summary based on my understanding. Please chime in if I got anything incorrect.
I've marked the ones I'm unsure about with a (?)


By default, Unity will load all shader variants into memory.
This can cause big memory usage by shaders, as the memory is taken up by a lot of unused variants.

Dynamic Shader Variant Loading gives us the ability to not load all variants into memory!
+ This can drastically reduce memory usage at runtime.
+ This can also greatly reduce scene load times.
- This could introduce stuttering when a new shader is needed.

Setting Chunk Count to 1 means that Unity will only load the 1 Chunk that's needed. (?)
+ This reduces load time, and reduces runtime memory usage.

However, whenever a shader variant is needed, Unity will need to load that Chunk.
The default is 0, which means Unity will load all Chunks for all requested shaders.

If we set the Chunk Count to 1, that means Unity will only load 1 Chunk for each Shader variant at startup. (?)
+ Now Unity is not loading all variants into memory. This can greatly reduce memory usage!
+ This can decrease initial scene load time as well.
- This may add stuttering/hitches when a new variant is needed -- for example, when an object first appears.
- This hitch happens only once, when that variant is loaded.

At Chunk Count 2, that means Unity will load at minimum 2 Chunks for each Shader variant (?)
By increasing the Chunk Count:
+ This can increase the amount of Chunks loaded, which may reduce stuttering.
+ This can decrease the amount of decompression needed, as more "ready" chunks remain in memory.
- This can increase memory usage and initial scene load time.

We can further reduce the memory usage by reducing the Chunk Size to, for example, 1MB. The default was 16MB.
By reducing the chunk size:
+ We have lesser wasted memory in each chunk, reducing the amount of wasted chunk space.
+ We load smaller chunks into memory, which can reduce load times.

However, the smaller chunks come with a worse compression ratio:
- This can increase build size.
- This can increase the CPU usage by about 10% when loading chunks.
- Fewer variants are in memory, which means hitches may be more common.
- Decompressing a bigger chunk is easier on the CPU than decompressing multiple smaller chunks.


Summary

Try 1MB, 1Chunk, and see if you have increased hitches.

If you have hitches, to reduce it:
• Increase Chunk Size or increase Chunk Count:
Which one to increase can depend on the Hardware (?)
• Increasing Chunk Size is slightly more wasted space in each chunk.
• Increasing Chunk Count is slightly more CPU usage when decompression. (?)