Search Unity

  1. Looking for a job or to hire someone for a project? Check out the re-opened job forums.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Help Wanted ShaderCache.db file is extremely large

Discussion in 'Editor & General Support' started by orville_redenbacher, Feb 5, 2021.

  1. orville_redenbacher

    orville_redenbacher

    Joined:
    Sep 9, 2018
    Posts:
    13
    Two days ago I had to wipe my Library folder in an attempt to fix ~20 minute loading times when opening my Unity project. I deleted 2.5 million files totaling 175GB. I only have 9GB of assets so I'm unsure how this could be a normal occurrence. Today I noticed the same slowdown when opening the project and sure enough, my Library folder is now 81GB with ShaderCache.db eating up 63GB. I'm only using a few shaders (mainly variations of Lit) in my scenes. Is this normal?

    Edit: No, this isn't normal. Performance slowly deteriorates with every Android build until the editor literally becomes unusable. I'm nuking 80GB of garbage data every day to maintain my sanity. Task Manager shows me the editor reading up to 15GB/s from disk when idling.
     
    Last edited: Feb 20, 2021
    Danielpunct likes this.
  2. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    I have a similar issue, where the size of the ShaderCache.db grows exponentially when building to iOS or Android.

    Tested on two Win10 machines, the problem occurs using the 2020.2 releases. Using just the example URP project to test.

    It is fine when building to the Windows platform, and the ShaderCache.db remains a few kb in size.

    Using the 2019.4 and 2020.1 releases, the problem is not present at all: building to iOS or Android works fine and the ShaderCache.db remains just a few kb.

    So it appears to be an issue with 2020.2 and the latest Universal RP 10.x packages when building to mobile. The file size increases during the 'compiling shader variants' process.

    Can somebody confirm if this is a know issue? I can't find any mention of it in the tracker.

    Thanks
    Phil
     
  3. orville_redenbacher

    orville_redenbacher

    Joined:
    Sep 9, 2018
    Posts:
    13
    Exact same setup here - URP building for Android using 2020.2 on Windows 10.
     
  4. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    Thanks for confirming, Orville.

    Is there a fix coming for this?

    Thanks
    Phil
     
  5. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    Quick update: tested in the newly released 2020.2.4f, and the shadercache.db issue described above is still present.

    Any suggestions much appreciated.

    Thanks
    Phil
     
    Danielpunct likes this.
  6. Danielpunct

    Danielpunct

    Joined:
    May 9, 2013
    Posts:
    15
    Hey guys,
    Here a possible workaround: I just deleted the ShaderCache.db file from Library.

    On my side that file alone had 4Gb and after I saw that the reason for Unity hanging was that my Hard Disk was in 100% writing that file I figured it must have something to do with it.

    After deleting it and doing some development in Unity and also a successful apk build that file is 200 Kb. I imagine it will go up again in which case I am prepared to delete it again


    - let me know in case you discover it does any harm
     
  7. Simianosaurus

    Simianosaurus

    Joined:
    Feb 14, 2013
    Posts:
    14
    My shadercache.db was 21GB!!!
    It was stopping my project from opening as it just sat there saying 'Importing' (I'd left it for an hour).

    This is for a small Oculus Quest 2 (Android) test project with about 20 objects. Unity version 2020.2.4f1
    I have built it many, many times though as my PC is really underpowered and can't manage to just run on the device linked to Unity, but has to do a full build and run it standalone.

    Deleting the shadercache.db allowed my to open the project again.
     
  8. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    Thanks both.

    Daniel, when I delete the shadercach.db file and rebuild to a iOS/Android, the file immediately starts to grow again.

    Using only the default sample URP project, this is what happens:

    - 2019.4 - before build: 3kb, after a mobile build: 3kb.
    - 2020.1 - before build: 12kb, after a mobile build: 12kb.
    - 2020.2 - before build: 24kb, after a mobile build: 750MB.

    In 2020.2, it appears deleting the shadercache.db file will make your editor responsive again if it's causing slowdown, which is particularly evident on a larger project. But from my testing, the shadercache.db will quickly re-grow when you compile to mobile. This means you may need to delete it after every build as a workaround. Obviously this not a solution. The bug only appears to exist in the 2020.2.x series.

    Can anybody at Unity comment?

    Thanks
    Phil
     
    Danielpunct likes this.
  9. Shuyin76

    Shuyin76

    Joined:
    Aug 1, 2014
    Posts:
    5
    Hello Guys !
    Sorry to hear that you have the same problem than me !
    I also have a wooping ShaderCache.db of 12Go as I'm compiling my project for android...
    The bad news is that I'm using Unity 2021.1.0b4.1811 so it seems this bug is going to exists for some time.

    I'll open a ticket in hope they will be able to fix this issue ASAP !

    Have a good day,

    Clément.
     
    Picmo and Danielpunct like this.
  10. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    Hi Clément

    Thanks for the info, and thanks for submitting a ticket!

    Hopefully it will get fixed soon - please report back here if there's any news.

    Cheers
    Phil
     
  11. melterx12

    melterx12

    Joined:
    Jan 20, 2019
    Posts:
    16
    I just deleted the shadercache after tracking it down as the cause for my 10+ minute wait times when saving or reloading my unity project on an NVME SSD. It was 72GB.

    What the hell is going on here?
     
  12. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
  13. orville_redenbacher

    orville_redenbacher

    Joined:
    Sep 9, 2018
    Posts:
    13
    Picmo likes this.
  14. StaggartCreations

    StaggartCreations

    Joined:
    Feb 18, 2015
    Posts:
    1,459
    Looks like this has been fixed in 2021.1.0b12+!
     
    Shuyin76, Danielpunct and Picmo like this.
  15. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    This issue is reported as fixed for 2020.3 LTS. Having just tested the new release, it appears to be still broken.

    - 2019.4 - before build: 3kb, after a mobile build: 3kb.
    - 2020.1 - before build: 12kb, after a mobile build: 12kb.
    - 2020.2 - before build: 24kb, after a mobile build: 750MB.
    - 2020.3 - before build: 24kb, after a mobile build: 750MB.

    Has the 2020.3 release fixed things for anyone else?

    Thanks
    Phil
     
  16. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    My error, the bug is not marked as 'fixed' for 2020.3 as I mistakenly thought, it is actually listed under 'known issues'. Sorry for the noise, hopefully it will be sorted soon.

    Cheers
    Phil
     
  17. concave

    concave

    Joined:
    Aug 28, 2012
    Posts:
    49
    same here - building for oculus quest 1 file has 39.4GB o_O
     
  18. Picmo

    Picmo

    Joined:
    Feb 27, 2015
    Posts:
    10
    It's now marked as fixed in 2020.3.1 so try that.

    From my testing, it is much, much better. It's still grows a little though, so I would say it's not completely fixed compared to 2020.1 and earlier.

    Using the default URP test scene, this is how things now stack up:

    - 2019.4 - before build: 3kb, after a mobile build: 3kb.
    - 2020.1 - before build: 12kb, after a mobile build: 12kb.
    - 2020.2 - before build: 24kb, after a mobile build: 750MB.
    - 2020.3 - before build: 24kb, after a mobile build: 3MB.

    So it only grows a little using the default URP scene, but what is it like on a larger project?

    Cheers
    Phil
     
  19. tildebox

    tildebox

    Joined:
    Nov 6, 2018
    Posts:
    38
    not fixed in 2020.2.6 (Android), the file is 19.12 GB big
     
  20. StaggartCreations

    StaggartCreations

    Joined:
    Feb 18, 2015
    Posts:
    1,459
    There's a good chance this is related to the async shader compiler, introduced in 2019.1. It can be disabled in the Editor project settings
    upload_2021-3-28_14-18-37.png
     
unityunity