Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Reducing HDRP/Lit RAM usage

Discussion in 'High Definition Render Pipeline' started by maxxa05, Sep 17, 2021.

  1. maxxa05

    maxxa05

    Joined:
    Nov 17, 2012
    Posts:
    186
    In our HDRP game, In a built player, the HDRP/Lit shader uses a ton of memory. After removing a bunch of features in our HDRP Asset (see screenshot at the bottom of this post), we're now at 1.25 GB usage only for the HDRP/Lit Shader, down from 2+ GB. It's something, but it's still quite huge.

    Now I wonder, is there a way to reduce this further? I have a hard time understanding exactly how the shader stripping works under the hood.

    Would removing all materials using a given keyword of the HDRP/Lit shader reduce the shader's size? So, if I found that there's a single material in a scene using a specific keyword, could I remove the HDRP/Lit memory usage by modifying that material? If that's the case, is there a known tool that I could use to track usage of keywords?


    upload_2021-9-17_11-9-8.png

    upload_2021-9-17_11-15-25.png
     

    Attached Files:

    Marald, valarnur and alexandre-fiset like this.
  2. Dominique0

    Dominique0

    Joined:
    Jun 8, 2015
    Posts:
    26
    The same thing happens on a PS4/PS5 build, the HDRP/Lit shader takes over 1GB which is a pretty ridiculous amount.
     
  3. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,136
    They really need to add texture streaming for built-in shaders. I've noticed all ambitious games made with HDRP suffer with VRAM usage. Pretty big issue.
     
    Deleted User likes this.
  4. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    516
    I agree, I have a RTX 3080 w/10 GB VRAM, and run out of vram pretty quick
     
  5. SebLagarde

    SebLagarde

    Unity Technologies

    Joined:
    Dec 30, 2015
    Posts:
    936
    Hi, we miss information to be able to provide help.
    What is your Unity binary version and hdrp version?
    Also have you modify hdrp? (maybe you have left #pragma enable_d3d11_debug_symbols
    lastly are you doing a heavy usage of shadergraph? (with a lot of unique shader graph) ?

    if you are on 20.3 or above it is possible to reduce variant futher by using shader frequency stage: https://github.com/Unity-Technologies/Graphics/pull/2996 (note: there is multiple PR for this one so best to check on master the correct keyword stage frequency to put) - This work have been done only for 21.x and above


    >They really need to add texture streaming for built-in shaders.
    This is call asset bundle in Unity.
     
    valarnur likes this.
  6. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    I believe what he means is to have Virtual Texturing support for HDRP Lit so that we aren't suffering too much for the outstanding size of HDRP Lit shader. As of now, VT needs to be manually added in shader graphs, does not work with built in shader, and does not work with Asset Bundles. This makes VT/Granite pretty much useless for any ongoing projet (and not a solution to fix high Unity memory usage).
     
    shikhrr and PutridEx like this.
  7. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,136
    exactly :)
     
  8. maxxa05

    maxxa05

    Joined:
    Nov 17, 2012
    Posts:
    186
    I'm using Unity 2020.3.17, HDRP 10.6.0. By 20.3, do you mean Unity 2020.3?

    I do have some Shadergraph shaders, some using the Lit material in the graph settings (see image), but I'm not too sure if they impact the actual HDRP/Lit shader memory usage in the profiler.

    upload_2021-9-27_11-56-54.png

    I don't think we've modified HDRP. Do you mean modifying the HDRP asset, or modifying files in the package itself? If it's the latter, we didn't modify anything. We do have #pragma enable_d3d11_debug_symbols in the Crest asset from the Asset Store, but it's on another shader, could this have an impact on the HDRP/Lit shader?

    As for asset bundles, I'm already using Addressables, is there a way I could minimize the size of the loaded HDRP/Lit shader with it?

    At the end of the day, I think the most useful way to address this would be a way to associate a renderer with a specific variant and, optionally, the memory it consumes. So if, for example, I have a single renderer which uses a specific variant taking a bunch of memory, maybe we could remove features from said renderer in order to save a bunch of memory. Is there any tool like this, and if there isn't, is there a way I could code something that would do something like it.
     

    Attached Files:

  9. maxxa05

    maxxa05

    Joined:
    Nov 17, 2012
    Posts:
    186
    So, I tried to remove the whole Crest asset and most shadergraph shaders, and it does not change anything on the memory taken by HDRP/Lit in a build, it stays the same. I guess I'll have to find another strategy to solve this issue.
     
    olavrv likes this.
  10. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    934
    Can anyone tell me how do you know which shader is specifically taking up the RAM, is this data available in memory profiler or the normal memory tab in profiler?
     
  11. Dominique0

    Dominique0

    Joined:
    Jun 8, 2015
    Posts:
    26
    Memory profiler -> Detailed -> Take Sample Playmode
    It'll be in Assets->Shader
     
    MartinTilo and Ruchir like this.
  12. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    1. Now that the information has been provided, can we get an answer on: Is it "as designed" for HDRP Lit shader to take more than 1 gb of memory at runtime?
    2. What are the most memory-impactful features on HDRP that could help us reduce its memory footprint on Gen8 consoles?
    3. Is there any ways we can have more info on untracked memory from the memory profiler?
     
  13. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,483
    Do you mean the Untracked Memory as shown in the Memory Profiler Module (2021.2+) or Memory Profiler package?
    If so:
    I'll leave the rest of the questions for @SebLagarde as they are more HDRP specific than profiling specific and I unfortunately have too few insights there to contribute.

    Edit: Not sure why I quoted myself there, as the info is also in the Package documentation, but I'm currently working on butting it directly into the tool.
     
    Last edited: Oct 6, 2021
    Ruchir likes this.
  14. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Thanks for the quick response Martin. We use the Memory Profiler Package. I did not know about the new module on 2021.2. Does it offer more insight on untracked memory?

    We are using DOTS so we're kind of stuck on LTS for now, but if there's more info on memory, maybe I hack things so it runs on 2021.2 to see if I can learn something or not.
     
  15. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,483
    No, the package has more info. The module is just visually adjusted for clarity and to match the new package summary UI. The package has more info in general and also more info integrated into the high level summary bars, i.e. ExecutableAndDlls and Mono Virtual Machine memory. Most data memory Profiler adjustment and additions are likely to get backported too, so I would upgrade due to that.
     
  16. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    This is on PC with the hightest settings for our game.

    upload_2021-10-6_13-10-32.png

    The game looks great and really benefits from HDRP, but right now it seems like porting to PS4 and Xbox One is nearly impossible. By stripping out almost all features, the HDRP Lit still takes a lot of memory and likewise for the Untracked Memory which can't be analysed.
     
    olavrv likes this.
  17. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,136
    Yeah, I hope this issue isn't ignored. Let alone consoles, even PCs can really struggle. It can cause massive performance issues.
     
  18. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,483
    Well, it can be, e.g. with Pix on windows and Xbox. And with that high amount of Untracked, I think it is warranted to do so too. It's just not (yet) readily possible with built-in Unity tools that give a higher level and contextual overview of that memory...
     
  19. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    PIX does not give that much info either. I mean, what it tells me is: "There is something taking a lot of memory, good luck with that!".

    I am aware I'm just there complaining but I've been using Unity for 10 years now and it is the first time I have to dive that deep to figure out how to save on memory. Usually we just lower our texture resolution and split our scenes in chunks, but now even an empty scene has difficulty running on Gen 8 hardware.

    Anyways. We're continuing our quest to find a way to reduce that huge memory footprint and if I find anything, I'll be happy to share it here for others.
     
  20. Dominique0

    Dominique0

    Joined:
    Jun 8, 2015
    Posts:
    26
    Is it normal that the HDRP/Lit shader use considerably more memory in a build? With the HDRP demo scene, I have 1.3MB in the editor and over 16MB in a build. Is the profiler memory usage in the editor reliable for Assets/shader memory usage?
     
  21. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Here is an update on my ongoing investigation.
    • I have used the sample scene for testing and Shader Control for analysis.
    • I only have looked at HDRP Lit because that is the problem we have.
    • On Unity 2020.3 (HDRP 10.6), 82 keywords, 62,000 variants were logged in the editor, 34 keywords in a build after stripping.
    • On Unity 2021.2 (HDRP 12.1), 44 keywords, 2,196 variants were logged in the editor, 13 keywords in a build after stripping.
    • The sample scene alone is not at all problematic in both cases, taking less than 10 mb in memory in a build.
    • Adding around 40 HDRP Lit assets from my game grew that memory usage to 180 mb, so I believe that this exponentially grow as we're using more keywords.Tomorrow I'll see if Unity 2021.2 does anything to that 180 mb. I'll update my post at that time.
    So my thought is that 2020.3 has serious issues with its keywords and variants. As we are using DOTS, we fear that upgrading to Unity 2021.2 might be way too risky as we are approaching age rating and certifications. Still, I found out that DOTS works in Unity 2021.2 as opposed to what Unity said, so we might just take that risk if that solve the whole thing.

    The big question now is: Is Unity aware of the major improvements between 2020.3 and 2021.2 and are they planning to backport these to changes to LTS?

    Our project is not super mega fancy; it uses Unity HDRP built-in features that are not ray tracing or anything fancy. 1.5 gb of RAM usage in a build for that is unacceptable, especially on Gen8 hardware.
     
  22. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    So we kind of solved the HDRP Lit memory usage now on 2020.3. HDRP Lit went from 1.4 gb to 300 mb. We'd need to investigate thoroughly, but it is related to one or many of these changes:
    • Stripping all instancing variants (as far as we know, SRP batcher mostly replace instancing)
    • Stripping all fog variants (as far as we know, these are built-in remderer related)
    • Stripping all lightmap variants except Directional
    • We found that our camera frame settings (in HDRP Default Settings) had Ray Tracing ticked on but greyed out. We then ticked it on in our HDRP Asset, ticked off the now available camera frame setting, ticked back off the asset, so that all potential ray tracing related calls were removed.
    I have now rolled back many of these changes and hopefully we'll soon find the real culprit.
     
    Last edited: Nov 1, 2021
  23. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Me again! So, @maxxa05 discovered something today: Even if you untick all quality settings except one, all of their references seem to be built. That means that if you have several quality settings with each their own HDRP assets, all of the keywords associated with them are built (and present at runtime) regardless of what is ticked on the chosen platform.

    The Scalability page for HDRP suggests to create a seperate asset per platform. At least on Unity 2020.3 LTS / HDRP 10.6, this is super wrong and will result in exponential shader memory usage at runtime. For us, keeping just one asset assigned in the Quality Settings reduced HDRP Lit from 1+ gb to 100mb.

    This also means that a project cannot scale much. If you want DXR as an option for your game, that means that non-DXR PCs will pay the memory cost for that option as well. Same goes for all memory-costly options. I consider this as a bug and will report it once I've reproduced the issue on a small project.

    Edit: Bug reported Case 1377684
     
    Last edited: Nov 2, 2021
  24. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,136
    The profiler with the memory package shows a different VRAM consumption than than task manager/afterburner.
    Am i missing something?
     
  25. PutridEx

    PutridEx

    Joined:
    Feb 3, 2021
    Posts:
    1,136
    How do you strip all the built-in stuff?
     
  26. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    You keep one asset assigned in your Quality Settings + you strip all Instancing variants in the Graphics Settings.
     
    PutridEx and Lex4art like this.
  27. iamarugin

    iamarugin

    Joined:
    Dec 17, 2014
    Posts:
    887
    Any comments from Unity on this problem would be great.
     
    Deleted User likes this.
  28. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    Yup, got a 2020.3 project using HDRP that needs to be ported to consoles and two lit shaders alone are using over 2GB of RAM due to the enormous amount of variants.
     
  29. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Like I said, you can reduce that 2 gb down to ~500 mb by keeping only one referenced HDRP asset in your quality settings and by unticking debug aov ap on the asset.
     
    Ruchir likes this.
  30. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    We'll have to add more pre-build scripts to swap between the HDRP assets depending on the target platform. I swear, feels like 70% of my work in Unity is working around Unity's problems.
     
    Tetsuya_OYA likes this.
  31. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Yes and I really can't see the point of these assets if they just stack up in memory.

    This part of their doc is just super wrong as doing what they suggest will have serious impacts on memory. It took us weeks to find that out and it is a major problem as it simply makes HDRP not scalable at all unless all your platforms have infinite memory.

    upload_2022-3-6_15-37-55.png

    We'd love to offer DXR as an option in the future but eh! Doing that will just increase memory consumption for all our players.
     
    DavidDreamer and Neto_Kokku like this.
  32. slime73

    slime73

    Joined:
    May 14, 2017
    Posts:
    107
    Did you make sure to limit the platforms used by the Quality settings associated with those HDRP Assets?
     
  33. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Yes. If you were to have PC Low / Medium / High assets, the HDRP Lit shader always included in memory would have all the variants. If Low has no Depth Prepass but High has it, then both Depth Prepass and No Depth Prepass variants are compiled and loaded at all time.
     
  34. slime73

    slime73

    Joined:
    May 14, 2017
    Posts:
    107
    Yeah that's sort of expected IMO (although it's not good design on HDRP/Unity's part, to put it mildly, but maybe not a bug per se). I don't think Unity has a way to selectively load/unload shader variants at runtime, it can only strip them at build time, so for a given platform it has to load all variants that were kept by any HDRP Asset associated with that platform.

    I just thought from your earlier posts that you were getting variants loaded on non-PC platforms that shouldn't be there at all, because they're from HDRP Asset settings which were enabled in your PC Asset files but not your other asset files. If you're only talking about memory use on PC then it's definitely an issue but thankfully not as catastrophic as it could have been.
     
  35. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    I'll test later today to confirm if limiting quality levels enabled for the platform actually reduce the memory consumption, or if everything is included regardless.

    Still sucks if you want to provide different profiles for PS4 base and PS4 pro, though: double the (already ridiculously high) shader memory.

    500MBs worth of shaders on a platform where you can only use 4.5GB of RAM is already a massive number. With built-in I rarely saw that number go past 200MBs.
     
    nehvaleem and slime73 like this.
  36. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    No deal, removed all quality levels except for one, the lit shader still uses 1GB of RAM. Time to dig into variant stripping.
     
    futurlab_peterh likes this.
  37. slime73

    slime73

    Joined:
    May 14, 2017
    Posts:
    107
  38. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    In built-in it was possible to generate a shader variant collection with only the variants actually used by playing the game in the editor, is this no longer possible with HDRP?
     
    Deleted User likes this.
  39. Deleted User

    Deleted User

    Guest

    @SebLagarde will this issue be fixed soon??? this issue looks very serious
     
  40. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    If that happens, it might be assigned elsewhere or has some memory-heavy options ticked on, notably:
    • Runtime Debug Display
    • Runtime AOV Debug Display
    The more you remove features, the less the asset takes. Then you should untick all of these unused:

    upload_2022-3-31_8-51-22.png

    We also found ShaderControl on the asset store particularly helpful to list all variants built. Sometimes there are variants just in used once that is just a matter of toggling a boolean on the material inspector to get rid of it.

    In the end you can shring HDRP Lit to ~150 mb and still have decent graphics. Not ideal, but at least not a showstopper like a 2 gb HDRP Lit always there in memory.
     
  41. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    Just FYI, in Unity 2021, especially 2021.3.15, major improvements can be noticed. More information on the said improvements can be found there.

    If we do all the magic tricks I explained above, we can get HDRP Lit down to 150mb with two assets with most non-debug and non-DXR features on.

    The real godsend is that I've tried building with the three assets, same context as OP up above, and the runtime usage dropped from 2gb+ to 320mb. That's some massive gains. Furthermore, our first build with that worst case scenario went from 12h+ to around 3h.
     
  42. Marald

    Marald

    Joined:
    Jan 16, 2015
    Posts:
    42
    I was able to fix this (after reading various comments here) in Unity 2021.3 when it suddenly happened in my project.

    I added an extra option in my URP quality levels and added it to the renderer list and building times went from minutes to hours instantly.

    removing quality levels in URP (and even deleting the universal render data objects in library) brought the build down from 2 hours back to 10 min or so..

    just referencing more than 2 gets unity into problems, I guess.. now stripping only takes a fraction of the time.
    Good luck everyone!
     
  43. HazimMadbox

    HazimMadbox

    Joined:
    May 5, 2018
    Posts:
    4
    Can you share your HDRP asset , cuz i did what you said and i had no luck in decreasing Ram usage
     
  44. alexandre-fiset

    alexandre-fiset

    Joined:
    Mar 19, 2012
    Posts:
    717
    There is nothing special about my HDRP asset, and there is one per platform except PC where we afforded three. Did you remove all references to other assets from your quality settings list? You should only reference one if you desire low memory usage. Another thing we found is that we rendered the game journal on 4096px Render Textures: This caused all render target to be scaled to 4096 as soon as we opened the journal, and that scaling persists. And now there is shader variant loading, an option in the player settings, that can also reduce the size of HDRP Lit even more.