Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Feedback New Shader WarmUp API

Discussion in 'General Graphics' started by HG_SB, Jan 27, 2022.

  1. HG_SB

    HG_SB

    Joined:
    Dec 9, 2015
    Posts:
    38
    Lags due to shader compilation have always been an annoyance because they are often quite noticeable and they happen especially often on a players first session which leaves a bad first impression.

    So we as a developer want to hide those lags by prewarming our shaders in loading screens or in the background while streaming in new parts of a level. This never was Unitys strength. Up until recently we had https://docs.unity3d.com/2021.2/Documentation/ScriptReference/ShaderVariantCollection.WarmUp.html which had multiple design issues:
    • ShaderVariantCollections could only be created in the Editor
    • You only had the choice between "Click this button and save everything" or "Add them by code but then you have to find out what to add on your own"
    • WarmUp is a synchronous API. You cannot prewarm a large collection asynchronously over multiple frames. This means it is unusable for background prewarming and even in loading screens you get unpleasant lag spikes
    For modern graphics APIs https://docs.unity3d.com/2021.2/Doc...ntal.Rendering.ShaderWarmup.WarmupShader.html and https://docs.unity3d.com/2021.2/Doc....ShaderWarmup.WarmupShaderFromCollection.html was added. While this fixes the problem that the old system does not work for newer graphics APIs this does not solve any of the issues above. It actually adds another one:
    • Both functions require a vertex attribute setup to work but it seems like there is no easy way to get this information. Sure I can loop over all renderers in my scene and get that information. But that will only work with static or objects that are in the level at startup. I could scan periodically while playing the level. But how often should I check? How can I assure that I get every object even if it is short lived without adding a script to all of them? How am I supposed to get that information from VisualEffects? Their renderer type is internal. Do I have to use reflection for that? And after all of that: Did I get everything? I don´t know because Unity will not tell me.
    This needs a rework. My main requirements for a new API would be:
    • There should be no guesswork and manually collecting everything. Let us register a callback somewhere that gets called for each new shader compilation that provides all information we need to preload those shaders in the next build. Something like:
    Code (CSharp):
    1. ShaderWarmup.RegisterShaderCompilationCallback(MyCallback);
    2.  
    3. private static void MyCallback(Shader shader, Experimental.Rendering.ShaderWarmupSetup setup) {
    4.     //We can save this information however we want
    5. }
    • This API would be very flexible. We can collect everything in a big list. Or make one per level (This would require the possibility to somehow reset the system so the same shaders can trigger the callback multiple times so we can record them for multiple levels). We can add things later or remove them if we want to.
    • The callback should work at runtime. We want to create builds with this callback enabled so we can collect prewarm information while QA is testing these builds.
    • Prewarming should be an asynchronous API so we can avoid big lag spikes and use it in the background while playing.
     
    Last edited: Jan 27, 2022
  2. eggsamurai

    eggsamurai

    Joined:
    Oct 10, 2015
    Posts:
    105
    bump, shader warmup takes 10mins longer sometimes
     
  3. jamessnow

    jamessnow

    Joined:
    Jul 19, 2022
    Posts:
    3
    my game, warmup costs 60s... No Player could put up with a game with 60s lag. :(
     
  4. KimmoFactor

    KimmoFactor

    Joined:
    Dec 29, 2020
    Posts:
    39
    Just bumping up this as I was just looking into it and very much agree with the original poster. The experimental API is near pointless as I see no sane way to gather all VertexAttributeDescriptors and map them to shaders. You're already automatically gathering some of the info in editor, so you might as well help us go the rest of the way.
     
    dtaddis likes this.
  5. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    We have this on the radar. It's just lower priority than some other things we're doing right now.
     
    KimmoFactor likes this.
  6. ge01f

    ge01f

    Joined:
    Nov 28, 2014
    Posts:
    121
    Not trying to be antagonistic, but are these priorities set by users? Having the game run slow for a long time is basically a deal-killer. Are there many more important priorities than not having the software closed because its infuriately slow?
     
    Lars-Steenhoff likes this.
  7. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    Users' opinions are definitely considered when prioritising features.

    There are ways to do proper warmup manually. It's just not a very nice thing to implement.
    You could render each object in the scene manually positioning it right before the camera and hide that behind the loading screen. This would get all the necessary shader variants warmed up.
     
  8. peeweekVFX

    peeweekVFX

    Joined:
    Oct 19, 2015
    Posts:
    14
    Thanks for your answer ! I double down on feedback for this one, especially regarding shader warmup considerations for VFXGraph.

    From what I understand, Even though VFX Shaders are added to a variant collection, they will be loaded if the corresponding output context is rendered : https://forum.unity.com/threads/how-to-preload-shader-in-vfx-graph.1222602/#post-7795398

    @aleksandrk : I have made some testing using your manual warnup simple effects, but there are too many cornercases that could prevent particles from spawning, especially because of custom events, property interface or even other considerations that would make a VFX not working without its proper context in the scene.

    However it seems that it still tackles a majority of "simple" effects.

    @JulienF_Unity , @PaulDemeulenaere : I would really like to hear from VFX Graph team's some advice regarding such prewarming if it is currently possible, especially how to handle ShaderVariantCollections with VFXGraph.
    Is there some sample code with good practices available?
     
    Lars-Steenhoff likes this.
  9. Wolfos

    Wolfos

    Joined:
    Mar 17, 2011
    Posts:
    945
    Unity should *probably* warn developers this is a thing they need to do when they select DirectX12, Metal or Vulkan for their games. Learn from Unreal's mistakes and the dozens of high profile failures resulting from this very issue.

    Epic also let it fester for years. Should be a higher priority, and not buried in some obscure part of the documentation.

    Here's what Unity could do:
    - Give us a script that collects shader variants during gameplay and allow us to plug that data into the warmup function.
    - Instead of halting the render when a shader variant is missing, skip over rendering it until compilation is done (Unreal 5.2 does this).
     
    Last edited: Jun 19, 2023
    Lars-Steenhoff and dtaddis like this.
  10. unity_QJ7RazXzghZCzA

    unity_QJ7RazXzghZCzA

    Joined:
    Jul 9, 2021
    Posts:
    104
    You propose to create loading screen and hide all meshes behind it before starting the game?
    Im now working on optimizing RAM usage in my game for Android, I can easily tell you that this is not possible to just put everything into the memory and not get a crash on game bigger than hyper-casual. And also there is not only meshes but also particles. I noticed lag for them as well. First launch of the game was always a problem for my project, we even lost one publisher because of this issue game lags no matter what I do on first launch.
     
  11. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    You don't have to put all of them there at once.
     
    Lars-Steenhoff likes this.
  12. unity_QJ7RazXzghZCzA

    unity_QJ7RazXzghZCzA

    Joined:
    Jul 9, 2021
    Posts:
    104
    Question is why this is so tricky and complex to do? I think anything like that should be done on Unity-end and also async shader.warmupall would be really useful.

    Please, don't force peoples to do some tricky nightmare like that, this is not what we expect from game engine.
     
  13. Wolfos

    Wolfos

    Joined:
    Mar 17, 2011
    Posts:
    945
    The time required to load every object in a large game from disk is pretty huge.
     
    unity_QJ7RazXzghZCzA likes this.
  14. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,533
    That's Unity's modus operandi unfortunately.

    Instead of fixing and implementing a gazillion smaller features that would improve all Unity games, that most developers end up implementing themselves, (things that a good engine should handle for you, right? that's their reason for existing. Imagine a kick-ass, best in class, shader warm-up system. Hitching because of shader loading becoming a weird thing that only *lesser* engines do and that other engines look up to Unity for how it should be done), instead they are busy splitting the render engine 3 ways and discovering DOD and thinking it's pretty cool for a decade.

    Meanwhile the highest bar the rest of their feature set can reach is "eh, I guess it mostly works".
     
    Last edited: Aug 5, 2023
    unity_QJ7RazXzghZCzA likes this.
  15. unity_QJ7RazXzghZCzA

    unity_QJ7RazXzghZCzA

    Joined:
    Jul 9, 2021
    Posts:
    104
    Unity just should add label "Experimental" on their main page right before anyone will install it. When anyone try to download Unity it should show him popup with next message: "This engine is experimental and not ready for production use. Also it can deprecate any moment. Download at your own risk."
     
  16. Wolfos

    Wolfos

    Joined:
    Mar 17, 2011
    Posts:
    945
    Frankly, I think this issue is more on modern graphics APIs since Unreal is still struggling with the same issue.
    But Unity and Epic are members of Khronos, so at least they should be lobbying for a fix, right? (why has this taken 7 years wtf)

    Here's the situation: we can't sell constant stutter, nor a long (longer than a few seconds) shader pre-compilation screen to consumers. Please fix. We'll be using DX11 in the meantime.
     
    Kabinet13 likes this.
  17. shikhrr

    shikhrr

    Joined:
    Nov 19, 2013
    Posts:
    68
    Any update on this?
     
    DonCornholio likes this.
  18. DonCornholio

    DonCornholio

    Joined:
    Feb 27, 2017
    Posts:
    92
    Any update on this?
     
  19. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    714
    The only thing we need is some examples. In HDRP "unwarmed" shaders cause constant stuttering. On DX11, the shader collection we have refers to 1000 variants for a single scene. We need to prewarm these on Vulkan for the Steam Deck.

    I went ahead and logged all VertexAttributeDescriptor[] for meshes found in our scene and it's something like 19 unique sets. So starting from there it should be fairly simple to get things to work, but it's not...
    • How do we get VertexDescriptor from vfx graph? This should be really easy to answer, but difficult to guess. Unity must know what kind of mesh the VFX graph author?
    • I assume we should warm shaders using VertexAttributeDescriptor[] they are used on, right?
    Overall, it would be great to simply have a way to save an asset containing pairs of VertexAttributeDescriptor[] and ShaderVariantCollections.
     
    thebarryman, MVavrekPA and Peter77 like this.
  20. Kabinet13

    Kabinet13

    Joined:
    Jun 13, 2019
    Posts:
    93
  21. sebas77

    sebas77

    Joined:
    Nov 4, 2011
    Posts:
    1,639
    I stumbled upon this thread and I have my own question: why doesn't unity allow to cache compiled shaders on the device? Warming up is equal to compilation isn't it? Why should they be compiled each time?
     
    Genebris likes this.
  22. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,574
    I believe Unity does cache the compiled GPU programs, but when you restart the app/game, it will need to upload the cached GPU program to the graphics device for usage. The upload might be another cause of a hiccup, other than the compile, that you can workaround with prewarming.
     
  23. sebas77

    sebas77

    Joined:
    Nov 4, 2011
    Posts:
    1,639
    so this would happen also on mobile devices?
     
  24. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,574
    I'm pretty certain Unity does cache compiled GPU programs on Android, because the very first shader warmup takes significantly longer.
     
  25. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,073
    There'd surely be some example demos of the processes of the engine on a device, right?

    RIGHT?

    Oh... wait, they're waiting on you to provide this kind of insight into their software. Sorry, Pete. More work you shouldn't be required to do to keep this software honest.
     
  26. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    This is correct. One caveat - only on platforms that support it. Android is for sure among those.
     
    Genebris likes this.
  27. thebarryman

    thebarryman

    Joined:
    Nov 22, 2012
    Posts:
    130
    The ideal setup for me would be to simply add a vertex layout to the ShaderVariant struct within the ShaderVariantCollection.

    This way you could choose which variants need which vertex layouts. You could warm the same variant for multiple layouts, and you could also warm one layout per variant.

    The current API call leads to hugely redundant warmups. If a shader has 5 variants, each requiring a unique layout (eg. more and more vertex attributes for increasingly complex vertex animation), I should only have to warm each variant a single time with the layout it expects. Instead, because we are only given the option of warming a layout for an entire shader, we have to warm each variant with 4 unused layouts.
     
  28. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    The ideal setup would be to have the build system gather all the vertex layouts as needed and to have an API/UI to provide extra ones if one plans to switch materials at run time.
     
    Peter77 and Lars-Steenhoff like this.
  29. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    714
    A very good starting point for now would be to at the very least document how to warmup shaders on Vulkan and Metal using the existing API ;) Here and/or in the docs, but at least somewhere.

    At the moment if I run a script gathering layouts and use them to warmup a collection, the Steam Deck takes several minutes to warm them up. I must be doing something wrong.

    Then would also be great to know what to do with VFX graphs as they are the ones responsible for the most annoying hiccups.
     
  30. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,073
    Could Unity find the time to make demos of this for all major platforms?
     
  31. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,574
    This would indeed be very helpful in resolving a scenario like this one, where having some kind of hook to provide additional shaders/variants during the build process would most likely help.
     
  32. thebarryman

    thebarryman

    Joined:
    Nov 22, 2012
    Posts:
    130
    I already have a pretty elaborate tool set up to find all shader variants that are needed in the game. What you are proposing sounds similar to the tooling that I already have set up. While that would be great, it sounds like a much larger task than the more modest change to the API I am suggesting, which would provide much better functionality than the current experimental call.
     
    Last edited: Nov 22, 2023
  33. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    @thebarryman I would prefer to first design how we want to do warmup in the future before patching the API and then potentially changing it completely.
     
  34. TomTheMan59

    TomTheMan59

    Joined:
    Mar 8, 2021
    Posts:
    349
  35. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    714
    Unfortunately no. I asked again and again, but Unity staff just ignore it while answering to other questions. That's insulting. There is an api, but nothing to indicate how use the thing, so it is practically useless. The stutters on Vulkan HDRP are especially frequent and very intense.
     
    DonCornholio, Kolyasisan and pwka like this.
  36. Genebris

    Genebris

    Joined:
    Mar 18, 2013
    Posts:
    141
    Where is that stored on Windows? I just want to test my first launch time vs second but it looks like it's just not documented.
     
  37. Genebris

    Genebris

    Joined:
    Mar 18, 2013
    Posts:
    141
    OK, I managed to find them. For Windows with Nvidia:
    ...AppData\LocalLow\NVIDIA\PerDriverVersion\DXCache
    Hopefully some day it'll be documented.
     
  38. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    This is Nvidia driver cache, not ours.
     
  39. Genebris

    Genebris

    Joined:
    Mar 18, 2013
    Posts:
    141
    I understand, but there's no mention that prewarming is two step process with first one cached and you need to delete nvidia cache somewhere every time to test it. I've asked on forum and on reddit and nobody knew the answer.
     
    GordonNB likes this.
  40. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,960
    Ah, I see. I thought you were asking us to document the location of the NV driver cache :)
    I'll pass this request to the relevant people.
     
    Genebris likes this.
  41. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    714
    @aleksandrk can we just have a code snippet that shows how the new shader warmup api can be used? For instance how would it be done for the HDRP sample scene?
     
    DonCornholio likes this.
  42. thebarryman

    thebarryman

    Joined:
    Nov 22, 2012
    Posts:
    130
    In case anybody has a similar issue to mine, my workaround is to split each ShaderVariantCollection into a group of collections, one per each setup needed.

    So Collection1 -> Collection1_SetupA, Collection1_SetupB, Collection1_SetupC, etc.

    You have to generate many more assets, but you can avoid redundant warmups this way.
     
  43. ixikos

    ixikos

    Joined:
    Jun 21, 2013
    Posts:
    26
    Curious about this line in the docs -
    How would we go about finding this out and then setting this down the line when we are calling WarmupShader?

    Unfortunately, even with this API we are getting too long of a freeze on our app causing it to crash on Apple Vision Pro. Considering making lightweight meshes and rendering those offscreen with a mesh with the same vertex data layout and similiar material setup as what will exist later to potentially leverage the threading that we seem to get when approaching things this way - thoughts?
     
  44. Genebris

    Genebris

    Joined:
    Mar 18, 2013
    Posts:
    141
    I'm curious how would you add all materials in your game to the scene and not get 10 GB memory usage and a long loading time. Maybe this will be avoided with mip map streaming, but still sounds crazy to me that we are expected to do that.