Resolved ShaderCache.db size is getting extremely inflated

IMPORTANT NOTE: We do not have any reproduction steps, and this issue is persistent but we were unable to locate any culprit - it happens around 2-5% of the time.

Some time ago our project got an issue on CI that we run out of space, for no reason at all. Looking at the file sizes we found that ShaderCache.db has grown to 1TB of size, for a project size of around 5GB of assets.

We took nothing from it, just removed the file, and moved on - we thought this was a one-time off.

After a while, we found that on our development machines (not on all machines) that isses exist as well. (Example of inflated ShaderCache.db file size from today)

(Size: 112GB)

Today I discovered that the file is growing a lot at two points of the editor's lifetime. "On editor start" and "On editor quit"

On Editor Start it never grew past around 100MB which is understandable - even if the file was already bigger it didn't change its size significantly.

On Editor Quit, the editor (when this issue happens) took around 5-6 minutes to close, and while that process ShaderCache.db is written and read to a lot! (Sorry for the truncated screenshot that lacks Write speeds)

System ShaderCache.db Read - 338 376 192 B/sec | Write - 125 151 124 B/sec
Unity.exe ShaderCache.db Read - 2 066 432 B/sec | Write - 912 511 B/sec

It averages at my SSD Read/Write speeds - and my usage of the SSD reaches 100%.

Findings:
- Removing ShaderChache.db does not decrease or increase loading time of the editor.
- It seems to not happen on HDD drives
- It only happens on Editor closing

Log dump from closing the editor

Code (Log):

1. Unloading 707 Unused Serialized files (Serialized files now loaded: 0)
2. Loaded scene 'Temp/__Backupscenes/0.backup'
3.     Deserialize:            2.093 ms
4.     Integration:            47.725 ms
5.     Integration of assets:  0.004 ms
6.     Thread Wait Time:       16.259 ms
7.     Total Operation Time:   66.081 ms
8. System memory in use before: 0.72 GB.
9. System memory in use after: 0.55 GB.
10.  
11. Unloading 3461 unused Assets to reduce memory usage. Loaded Objects now: 9381.
12. Total: 83.948000 ms (FindLiveObjects: 1.757300 ms CreateObjectMapping: 1.151900 ms MarkObjects: 73.640200 ms  DeleteObjects: 7.396500 ms)
13.  
14. Uploading Crash Report
15. Unloading 20 Unused Serialized files (Serialized files now loaded: 0)
16. System memory in use before: 0.61 GB.
17. System memory in use after: 0.61 GB.
18.  
19. Unloading 21 unused Assets to reduce memory usage. Loaded Objects now: 9432.
20. Total: 79.600600 ms (FindLiveObjects: 1.707500 ms CreateObjectMapping: 0.724000 ms MarkObjects: 77.075200 ms  DeleteObjects: 0.092000 ms)
21.  
22. UPID Received '108f880f-71a0-431f-8f59-8ce8eed85972'.

Unity version - From 2020.1.5f1 to 2020.3.0f1

Questions!
Did any of you had such issue with their project?
@unityteam are there any additional log files that I can provide for you to help us solve this issue and find the cuprit?

 

I've seen this reported a few times perhaps six months ago or so. It's likely been fixed already... perhaps sniff around the change notes for the versions just after some of the versions you're using?

https://unity3d.com/get-unity/download/archive

 

Upgrade to 2020.3.1 or later. You probably will want to delete the ShaderCache first, but not sure. Always backup your project before a Unity version upgrade.

https://issuetracker.unity3d.com/issues/shader-error-db-grows-on-each-build
https://unity3d.com/unity/whats-new/2020.3.1

 

Menawhile searching my projects for the shadercache.db size expecting the worse

 

Thanks for pointing me to update the Unity.

https://unity3d.com/unity/whats-new/2020.3.1 - Shaders: Fixed issue where the ShaderCache.db was growing in size after each build. (1317744)