Search Unity

Feature Request Specify build order of shaders (IPreprocessShaders.OnProcessShader)

Discussion in 'Shaders' started by D1234567890, Oct 12, 2022.

  1. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    I need to strip variants from a shader based on the stripping result of another shader. IPreprocessShaders.OnProcessShader calls are grouped together by shader so callbackOrder does not help. What is needed is a method of setting the build order for a shader. This could be implemented as something in the shader's metadata.

    The use case is shared keywords. I have a primary shader which has shader_feature keywords and another shader which mirrors those keywords using multi_compile. Changing the latter to shader_feature and having users managing these shared keywords across multiple materials is problematic and tedious.

    A workaround would be a global settings file (similar to SRP settings files) where users set shader stripping but that is an inferior approach to using materials in my opinion.
     
  2. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    3,019
    I think you could gather all materials that go into the build and store the keywords somewhere, then use this to strip the variants.
     
  3. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    Thanks for the reply. That is what I currently do (by searching the scene) but it has issues and I am struggling to work out an approach that I believe is similar to Unity's content stripping/searching (I don't know how it works as I couldn't find a resource on it). This is for a UAS asset so it ideally should match Unity's content stripping/searching (scenes, addressables etc) for it to be correct.
     
  4. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    3,019
    AFAIK it just goes from the scenes in the build and collects all materials that are used.
    What issues does this approach have?
     
  5. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    There doesn't appear to be an API to match Unity's approach. And hard to verify the spec without it being documented.

    SCENE

    Resources.FindObjectsOfTypeAll FAIL
    Finds anything in memory which can include materials outside of the scene.

    AssetDatabase.GetDependencies FAIL
    Passing the scene to this method can get more materials than Unity would if recursive is true and less if recursive is false.

    I couldn't find any other suitable methods but maybe I missed one. GetDependencies implies it is the correct method but testing proved otherwise.

    RESOURCES

    Resources.LoadAll PASS
    This will get everything in the Resources folder. So that one is easy.

    ADDRESSABLES

    No idea how to handle these... and this was the query I received that started all of this. I was not handling addressables. And it highlights the problem of high maintenance due to new approaches to managing assets.

    SHADER VARIANT COLLECTION

    Do I have to handle these too?

    ETC

    And who else knows what other approaches I would have to handle. My assumption is that OnProcessShader will run over all shaders that will make it into a build thus being an obvious choice from my perspective to handle the problem.
     
  6. Invertex

    Invertex

    Joined:
    Nov 7, 2013
    Posts:
    1,550
    The issue with wanting a specific order control through the OnProcessShader callback would be that the shader compilation would have to be linear instead of multi-threaded, drastically slowing down compilation speed. It would have to process one shader after the other, instead of compiling shaders in parallel for each core you have.
     
  7. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65

    According to this reference it is not parallel until compilation. What I am discussing here is before then. Unless I am mistaken.
     
  8. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    3,019
    It doesn't mean it will stay this way.

    I think you just need to get all root game objects in the scene and walk down the hierarchy, checking whether each has a renderer component. This way you'll get all materials in one scene; iterate through all scenes in the build and you're there.
     
  9. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    Unity also picks up serialised materials fields. It even finds materials in prefabs where the prefab is set to a serialised field. It could even pick up more but I wouldn't know as I don't have a spec.

    Understood. I'll ask about gathering materials on a different subforum.
     
    aleksandrk likes this.
  10. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    Thinking about this more, I do not think this would be a problem as it could be done in stages. So if I had one shader dependent on another, all shaders could be stripped in parallel except that one shader which would be done in a second step. At scale stripping could be grouped into sequential batches. Finally it would be opt in.
     
  11. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    3,019
    You're right, but I still don't think we should implement this.

    It would introduce an implicit dependency between two otherwise unrelated assets. What if one of them is not part of the build?

    I think a better way would be to somehow simplify the workflow here. Are you trying to allow switching shaders at runtime?
     
  12. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    The first shader is a water shader and the second is an underwater shader. The underwater shader mirrors keywords and properties from the water shader so they match. It cannot work without the other so it being missing wouldn't be a problem. Details:
    • water shader has shader_feature keywords
    • underwater shader mirrors these keywords as multi_compile
    • underwater material created in script (not an asset)
    • Material.CopyPropertiesFromMaterial to sync the water material to the underwater material
    • underwater variants are stripped in OnProcessShader to reduce build size and time based on water shader variants
    Only options I can think of is a global settings file to managing stripping or removing forgoing keywords. Both options aren't great for UX or performance respectively. The water material managing keywords is the ideal.

    Yeah, users could have different water materials with different variants in a build.
     
  13. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    3,019
    Got it. Let me think about it a bit.
     
  14. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    3,019
    Perhaps you could programmatically make a copy of the material, change the shader on it, and make sure the new material is included in the build? This would guarantee that the keywords are in sync and that both shaders get the same variants. The shaders need to use the same keywords (with the same directives) in this case.
     
  15. D1234567890

    D1234567890

    Joined:
    Aug 11, 2019
    Posts:
    65
    That would have the same problem of gathering all materials. We use to do something similar but proved to be flawed because of not knowing what materials end up in a build.

    If there was an OnPostProcessAsset callback (or something similar) which is called for every asset that makes it into the build, that could potentially be a solution. But would require more justification than for this use case alone.