Question Vulkan Shaders: Why Are They So Large?

Hey everyone,

So I've been tasked with investigating the feasibility of moving a large app from from DirectX 11 to Vulkan. This app is cross-platform between Windows and Android, and the Android builds already use Vulkan. It would simplify production if both plaforms shared the same graphics API, not to mention the performance benefits and increased feature set that should come with a modern API.

However, I discovered a major roadblock. For some reason, the compiled Vulkan shaders Unity produces are simply enormous, being between 5 and 10 times the size of their DX11 counterparts. For example, our project uses a cut-down version of URP Lit with a fraction of the variants. This shader is about 300mb when compiled for DX11. On PC, Vulkan produces a file that is a whopping 2 gigabytes! This is unusable, as in the version of unity our project is on (2021) loads all variants into RAM regardless if they are used. Just loading in all the shaders for an average scene with Vulkan consumes more than half of our memory budget! Even if the fix for only loading the in-use shader variants into RAM was back-ported, our app was made for user created levels and mods made with the unity editor. Expecting users to upload/download hundreds of megabytes to gigabytes of shaders with their mod for what would otherwise be a less than a hundred megabyte file is not practical.

The shader size issue also applies to Android as well. On Android, we stripped out 5 keywords from Lit, reducing the number of variants by a factor of 32. Despite this, Lit on Android is only 1/3rd the size vs PC, at 100MB. Since memory is at such a premium on that platform, reducing the shader size even by a factor of 5 would be a huge win (assuming the apparent issue can be fixed).

I'm not sure whether this is the fault of Vulkan, or Unity's implementation, but I'm leaning toward something being wrong on the Unity side. I've talked with other non-Unity devs familiar with Vulkan who say there's no reason the shaders should be that large. As far as I can tell, the only reason that Vulkan shaders might be larger is the fact that Vulkan doesn't separate the vertex and fragment program, so any savings from using multi_compiles/shader_features exclusive to one program won't apply. However, vertex programs tend to be tiny and most of the time the fragment-exclusive keyword is the type used. So in practice this saves very little. For example URP Lit compiled in a test project goes from 2,162 KB to 2,180 KB when all program specific keywords are replaced with universal ones. Interestingly, compiling the same modified Lit with Vulkan resulted in a ~15% decrease in size, despite the fact that on Vulkan program specific keywords shouldn't behave differently than universal ones.

I've also experimented with the DXC compiler. It seems to be very hit or miss whether the shaders get larger or smaller when compiled with DXC. Sometimes it halves the size, but most of the time the shader gets only slightly larger or smaller. Even if every shader got halved, it still isn't enough to make up for the difference between DX11 and Vulkan. The 2 gigabyte URP Lit shader was compiled with DXC, so it might have been as big as 3 gigs if it was compiled with FXC+HLSLCC!

As an experiment, I created an empty base URP project in 2023.1.0a18 and slapped some cubes with different variants of Lit on them into an empty scene. I then built the project in debug mode with DX11, Vulkan, Vulkan with Lit modifed to use DXC, DX11 with Lit modifed to have no program-specific keywords, and Vulkan with Lit modifed to use DXC and have no program-specific keywords. In each case the number of variants compiled was around 17,000. Here are the sizes of Lit in each case:

DX11 ---------------------------- 2.16 MB
DX11 universal keywords ---- 2.18 MB
Vk -------------------------------- 10.25 MB
Vk DXC --------------------------- 5.60 MB
Vk DXC universal keywords - 4.86 MB

The space-savings with DXC are much more dramatic with the small number of variants of default Lit in this test project than what I've gotten in full builds with our own shaders. I'm not sure why the size difference varies so wildly. Being just twice as large as in the best case seen here would be acceptable. In practice, a significant portion of our shaders jumped from less than 30MB on DX11 to around 300MB on Vulkan with DXC, a 10x increase. Also, since unity decided to drop support for DXC on android, it can't be used to solve the size of shaders on that platform anyways.

Does anyone know why Vulkan shaders are so large? Is there a way to mitigate the issue other than switching to a different API?

 

Thank you, this is the information I needed.

I didn't stop to think about it until just now, but debug build asset bundles aren't compressed so I've been judging the uncompressed size this whole time. I'll have to check how much larger a full vulkan build is post compression.

I never saw anything saying it wasn't until recently. The doc sheet with all the info about the compiler mentioned that android had yet to be tested thoroughly. I got the impression android support was in the works. Also, I posted some bug reports many months ago about the DXC compiler and android, and got some replies about fixing the issues. Then last month the issues were closed with "will not fix" saying that DXC was not supported on android.

I wasn't aware the dynamic variant loading was already finished and backported, we'll definitely try updating versions.