Shader keyword limit

Definitely a great start for loads of projects!

 

Great! (Yes, this was what I was asking about)

Thanks for the update

 

Thank you very much! That will give us a little bit more breathing room until a proper fix.

 

Bout time.. JK. but kinda serious. GOOD WORK UNTIY TEAM

 

I dont understand why my project was working fine, then suddently when I reboot the editor, I get a dozen of these errors. Since then, can't seem to fix the issue. I did NOT add a single shader...

2019.4.14f1

 

easy solution

Increase keyword count to 4096, not to 1024. Unless the hit is exponential or something, please let us know why is so much worse to just have more. Is it that much slower for example ?

 

To those who get the same situation: Shader keywords are basically incremental in your editor, so as you use a new one in the editor, it adds to the count. Turns out, Just emptying the shader cache fixed my error, as many unused shaders aren't part of the cache anymore.

You can just delete the Shader Cache folder in the library folder, and relaunch unity.

 

Interesting, will try for sure.

 

Obviously, it worked for me considering I don't actually use enough shaders to cause the problem + I used shader control to uncheck everything I know I won't be using, so no problems in builds.

 

I appreciate the increase. Just noticed the increase with 2019.4.19.

https://unity3d.com/unity/whats-new/2019.4.19

Thank you.

 

Well done man

 

@aleksandrk
One thing that would be very nice is if Unity's internal keywords could be in their own pool. I'm working on a shader that uses a lot of local keywords, and using Better Shaders to automatically cross compile it to Standard, URP7, URP10, HDRP7, and HDRP10 versions of the shader. The problem is we don't really get 64 keywords, we get 64 keywords minus whatever Unity uses, and because that is different in each version of an SRP, there's really no easy way to know how many we really get. So in HDRP11, my shader might completely break and I might have to remove features from it, after people have built content on those features, simply because unity decided to use an extra local shader keyword.

 

Yes- I think I'm getting a bit lost in the complexities of HDRP's "Is this set via a property? Or a property which sets a keyword? Or a keyword? Or an internal define? Or a magic thing the material editor changes depending on a property?".

I create the HDRP abstraction for BetterShaders/MicroSplat via the shader graph output code, which has a ton of undocumented local keywords/properties/stencil for determining how something renders unlike the other 2 pipelines, which just use keywords for almost everything and are much easier to figure out.

To get things to behave properly and not loose my sanity I keep the structure as close to Unity's as possible, because I know I'll just have to rewrite it all on the next release if I change it. So from my perspective it's a bunch of local keywords, properties, and material code that come with the system- though technically you are correct, they are unique to that shader. I could, for instance, decode all the things the properties/editor are doing, and make a clean system where defines in the shader control everything, but in practice cleaning that all up would just mean more work on every release, and I'd likely break a thousand edge cases in the process, because it's often unclear how those things effect each other without digging through massive amounts of code. So yes, I guess my issue is really with HDRP's implementation, not keywords..

 

Well that didnt take us long! I'll go ahead and try to remove more shader keywords, but would it be possible to add more information to this log? For example knowing what shader and what keyword is being requested would help me out a ton.

I'd also kill for a reliable way to list all current keywords.

 

Considering the max is 384 i figured trying to get one at index 385 would be erroneous. But you're saying its a local one, so that shouldnt matter. Back to the drawing board.

It still seems to come from unity internal somewhere, hope i can get a repo.

 

@ALEKS

Sorry if it's a dumb question, but if 2021.1.0b2 has the changed keyword limit, does that mean the official release of 2021.1.0f1 has this too? Or is it solely on that beta?

 

@aleksandrk So whats the cost of raising the local keyword limit? For ages, I've had my own shader generators and keyword systems so haven't really had to deal with the limits much, but I'm writing a bunch of shaders these days using shader_feature_local. It seems to me that a lot of the push for larger global keyword counts is based on legacy shaders which aren't updated to use the local keyword system (particularly when you install 10 such assets from the store). multi_compile makes a lot of sense for global rendering switches, but shader_features being global at all doesn't seem to make a lot of sense anymore because, well, they can be toggled per material anyway. So for me, I see myself rarely using global keywords at all, but needing a lot more local ones.

For instance, my latest project is at 59 local keywords right now. I've optimized a bunch of keywords out of the HDRP adapter (Better Shaders) I use so that fits, as well as out of my own code by using branches when it makes sense - but I'd like to add a few more effects and options to the shader. While any fix to this is out of timeline for this project, I can see myself hitting this limit again in the future several times before I even add a single global keyword to a system.

Note that in Better Shaders format this level of complexity is pretty easy to reach without writing lots of code. The whole system is based on stackable shaders, and in this case I'm doing something like the HDRP layered lit shader. Each layer has options for Triplanar, Stochastic Texturing, Albedo, normal, mask map, emission map, noise, blending mode, filtering, and if the layer is on or off. So it works out to 11 keywords per layer, which if you stack it 4 times is at 44 keywords already, from 408 lines of code that's been stacked 4 times. This would be the same if I wrote out all that code, but still pretty cool that it's only 408 lines of code for the equivalent of HDRP's stacked lit shader. Anyway, I'm rambling now, but would love to know thoughts on that limit..

 

Better Shaders currently does render adapters for URP2019, URP2020, HDRP2019, HDRP2020 and Standard (7.x and 10.x series of the render pipelines). This, however, is a separate project that's authored using Better Shaders, since writing a raw vertex/fragment shader and maintaining it across 5 pipelines is insanity. Better Shaders exports the resulting 5 versions into a special file which has a scriptable asset importer, and unpacks the correct shader for the installed pipeline/version on import, so it looks like a shader that just works across all the pipelines to users, and doesn't need them to have Better Shaders installed. Works really well, but it means I can't generate code on the fly like I do with MicroSplat, so that's why I'm using stock #pragma shader_feature_local for everything instead of going the code generator route.

From the projects you have looked at, does it seem like most people run over the shader keyword limit due to the reasons I mentioned? If so, it would seem a shame to end up with a 384:64 keyword ratio because of something that will eventually get sorted as more shaders move to local keywords, though I certainly understand why it's where it is today.

 

Haha - Yeah, I guess I push the limits a lot..

 

I mean that just gets back to where we started, right? If 10 different shaders from ten different developers add another 30 global keywords because they run out of local, we're over the limit again.. Or am I missing something?

 

Damn, the madmen at Unity have finally done it. Congratulations on the fix and release.

 

Thanks for this Aleksandrk.

 

Any increases of the 256 limit on Unity 2020. LTS versions?

 

Hey just a question, Where do I increase/Decrease, edit these? been having trouble figuring out where to edit these. thanks

 

Hey. Have a question about reworked keywords system
In my project i'v used build preprocessor for shaders
The main goal of this preprocessor is to add few specific variants to the build

Pipeline for this case is to create new ShaderKeywordSet(), add keywords to it, and add result to existed data list
But in Unity 2021 shader keywords api was changed
And for correct ShaderKeywordSet work i need to know value from few IntPtr fields:

private IntPtr m_KeywordState;
private IntPtr m_Shader;
private UInt64 m_StateIndex;

And my questions is: What this fields means/how they are used? And, How i can get this values clearly?

Ofc i can copy this values from other instance in data list, but i wanna find more clear way