Has anyone done a multi-threaded project build comparison?

I'm trying to decide if it's worth the extra money to get a 5900X or 5950X instead of a 5800X. I've been able to find some threads that show how a 5900X or 5950X maxes out all the threads during a project build, but for some reason I can't find any actual specs on how much faster the extra threads make the process than a CPU with less threads. Has anyone built the same project on the same/similar machine with both a 5700X or a 5800X vs a 5900X or 5950X? How much benefit did all the extra threads actually deliver on a Project Build? 10%? 100%?

Thank you so much!

 

Didn't record any real data but from my personal experience adding more RAM has improved build time much more than a faster CPU.

Changing RAM from 16gb -> 32gb decreased build time by 50%~ (from 8 mins to under 4 mins)

Building on i7-9700K -> i5-1038NG7 (from 2020 MBP) when both are at 16gb ram only increased build time by 10%-20%. Note that the i7-9700K benchmarks 50%~ better than the i5

Checking resource usage during my builds also mostly showed RAM as the bottleneck.
Going from 32GB RAM -> 64GB RAM did not result in much improvement though.

 

Type of HDD, SSD or faster will also play significant role, how fast data is handled.

The game build time is the least of the concern, as you don't do that every day in most cases. You can enable build and go for a dinner. After back, it will be probably done in your case.

It is the time you spent working with assets, scripts compilation, baking, entering runtime, etc. that should be considered most.

 

Build time is a killer when you're doing device specific optimisation, or tracking down build-only bugs on device.

 

Xjjon, that's super helpful, thank you! So it sounds like maybe all the extra threads in the 5900X or 5950X wouldn't actually matter for build times. Very interesting.

 

Antypodish, that's also good to know. I don't suppose you have any data to pin down HOW much a faster SSD might be able to help, do you? I was able to get the WD_Black SN850 on a deal yesterday and my motherboard should be able to support it at full speed so hopefully I've maxed out SSD speed potential. But it would be really helpful to get some stats on how much a fast SSD might improve build times over a slow one.

(And you're right that for most people Build Times aren't the priority. But I happen to be aggregating code updates from multiple devs for different builds and often we have priority bugs that only manifest in full WebGL builds, so it's important in my case that I slash build times as much as is possible. If an extra $500 would slash all my build times in half for the rest of my life, that would be dollars well spent.)

 

Oh yeah, I should mention all of my builds were done on NVMe SSDs (average 3300MB/s read, 2000 MB/s write). On HDD it will be way slower. I cannot even stand developing on HDD anymore these days. Everything is slower, not just the build, but re-compile, editor loading times, etc.

That being said, disk speed will have diminishing returns too. So anything faster than a HDD will be a huge improvement and then not much more afterwards.

This is just for android builds though so your results may be different. I have to do 3-6 builds per day so anything over a few minutes build time is really noticeable. Being able to keep it under 4 minutes mostly has been great for my productivity.

 

I've tested the build time differences on HDDs and SSDs a few months ago, the difference between a SATA and NVMe SSD was minimal, and the difference between a NVMe SSD and a ramdisk wasn't existant, so you can't really go faster than your SN850, unless you are working on very huge projects or something.

If you are using Mono, more threads shouldn't really matter anymore once the first build was finished and cached.
For IL2CPP (and WebGL) more cores are good. I wouldn't expect 50% less build time for 100% more cores, but about 20% sound reasonable.
We can make sure if you tell me which Unity version you are using. I could try to disable some cores in the BIOS then to compare the build times when I have some free time.

Another thing you might want to look at is build automation, be it in the cloud or on a secondary computer in your home network. If you are really spending a lot of time on building, it's probably better to let another machine do the building while you keep working on your actual game or other stuff without blocking your computer.

 

Can you check your drive speed utilisation during build? Does it utilises anything close to 100%?

Check also CPU utilisation.

I don't have specific benchmarking, but @Armynator provided best answer so far on that.

Jus on side note, I have assumed you may be single dev. So that my bad on that part.

Other than, yeah having dedicated remote machine may be a solutions. I am aware of cloud services for builds. But don't know anything more boyond that. So you would need look for more information.

 

Here was a quick test (total build time 3min 40s)

CPU touches 100% a few times but mostly stays at 50% or lower.
RAM will slowly keep creeping up. On a 16GB system the ceiling was obviously there as Unity would not utilize more than 12gb~ due to system overhead. So more RAM here alleviated that.

From these rough tests though, CPU seems to be the next place to improve. A faster CPU may help more than more cores though.

For reference the project size is 34GB~ the output APK is around 140 MB. I don't expect output disk speed to make a difference here due to the small file size. Read speed may be more important but the overall project size is still rather small compared to other platforms, so your results may be different.

See:
Unity spins up 130-150 threads but seems most work is done a few.



--

We have some cloud build / remote jobs for targeting different store build configurations (like google play, amazon, etc) but those are pretty slow compared to local build. The local build use case is mostly for device specific debugging or profiling on specific hardware.

Curious if there are ways to improve the build time though, from these tests it does not seem apparent where there are bottlenecks.

 

Wow, that would be awesome to see the difference when you just disable some cores. That's a great idea. I'm on 2019.4.28f1. Although it would also be interesting to clock the build difference between 2019 and 2020. I've heard they made improvements to Build Times in 2020, but I've never seen them quantified.

That would be great to get a separate rig doing my building. A lot of times I just want the fastest possible build because I'm waiting to test something and it can be a pain to go start something else then come back to it. But a lot of the time a remote build would be great. I remember looking into getting Team City or something to help with that, but can't remember why I stopped pursuing it. I'll dig back into it, especially now that my old computer could make a decent drone.

 

2020 is faster than 2019. I don't know why but it does build faster (using same project). I upgraded it from 2019 LTS -> 2020.3 LTS to test.

Second computer is good to build on too. That way you can keep doing work in your main Unity.

In my opinion, for development using remote(cloud) build is not really good unless your builds are huge. For deployment and distribution it saves a lot of time though if you automate it.

 

Your comments made me realize I should scrutinize my own builds and see where the bottlenecks are. Unfortunately it wasn't super helpful. The build took 13.5 minutes. I couldn't see any bottleneck on hard drive or RAM. About 4 minutes of the total build process showed all the threads maxed out. So it looks like 30% of the build process could be accelerated with more threads. Other than that, I have no idea what the bottleneck is for me then.

Ah, I should mention this is on an i7-8700, Samsung SSD, 32GB RAM, GTX 1060

 

I don't have 2019.4 installed, but I've quickly tested building an empty WebGL project on Unity 2020.3.6f1 LTS with 8, 16 and 24 cores on a 3960X. 106 seconds for 24 cores, 103 seconds for 16 cores and 112 seconds for 8 cores.

The results are a bit unexpected, but I made sure that every step was the same after every PC restart to disable the cores. (Create new project, add scene to build, switch to WebGL, set API compatibility to .NET 4.x, build into an empty folder)

I think deeper testing is required for a final answer, as creating and building an empty project isn't really anywhere near an actual daily usecase. Also keep in mind that I've tested this with a Threadripper using quad-channel memory. The regular Ryzen 5000 CPUs all have dual-channel memory, and I have no idea how to check if that could be a bottleneck.
Maybe someone with a 3950X or a 5950X has a big WebGL project and a bit free time to test this.

In the meantime I'd suggest udating to a newer Unity version - there have been some nice build time improvements lately. Just take a look at this 2021.1 feature for example - in theory this should make quite a difference for build times.

 

Wow. That is indeed very surprising.

BTW, I found this old thread where @Arowx put together some excellent sleuthing about bottlenecks and possible solutions.

 

@xjjon, do you happen to remember how much faster it might have been? Like 10% or 50%?

 

Not as much improve as going from 16gb ram to 32gb ram.

The upgrade from 2019-2020 gave like 20%~ improvement. Not a huge improvement, but the workflow greatly improved for mobile as there are lots of known bugs in 2019 for gradle and android sdk not back-ported to 2019 LTS.

 

So it's murky, but the best data we have for now shows that the extra threads of a 5900X and 5950X, or probably even a 5800X, really don't help much with Project Build times. Sufficient RAM and SSD speed are essential, but otherwise it seems largely limited by single-thread speed. The latest 2020 and 2021 builds are adding features that seem like they might help more than a new CPU.

 

I'm a bit late to the thread and I don't have the exact build times but I went from a 3600 (6C/12T @ 3.6GHz) to a 5950X and the time to make the initial build for a platform (ie with a fresh library folder) went from 2 hours down to 45 minutes. Subsequent builds went from around 8 minutes down to 3 minutes.

My project has to make thousands of shader variants since it uses the HDRP and there are no assembly definitions set up to ease the time it takes to make builds since they didn't come out until a while after the project started and it was going to be too much of a headache to implement them.

My complete system is the 5950X, 64GB DDR4-3600, 2x 2TB PCIe 3 SSDs, and an RTX 3070. Shorter build times are not the only benefit I've seen though. Thanks to all of those cores my computer remains responsive regardless of what task I have Unity performing allowing me to continue working in other apps.

Craziest thing I was able to do was play Cyberpunk 2077 on completely maxed out settings while making a build without the game making so much as a twitch from the background processing. I would absolutely buy it again.

 

Wow, that's huge. On paper it should only be about 35% faster. Do you happen to remember the RAM/HD setup you had in the previous rig? Or was it only the CPU that you upgraded?

Are your 2 SSDs in a raid configuration? I wonder if that might be part of it . . .

I wonder if there's something specific to your type of build allowed it to be benefited a lot more by the extra threads. Have you ever looked at how much of your build time is spent with most of the threads maxed out?

In the end, maybe I'll have to just buy two CPUs and try my exact build both ways. Amazon actually has the 5950x for sale at MSRP today, but it wouldn't ship for a few weeks. Hmmm . . . super helpful data point, THANK YOU!!

 

Only upgraded my CPU. I don't have the two drives in RAID.

I haven't taken the time to perform a thorough analysis. My data is from opening the Task Manager and looking at it at various points during the build process. The CPU is almost always sitting at 100%.

 

I recently went from an i7-3820 to a 5600X. I made a few other updates along the way, including going from a mid-range SATA SSD to a high-ish end NVME drive and getting more, faster RAM. I don't remember the difference in time for initial builds, but subsequent builds have gone from ~10 minutes to ~2 minutes. While the system stays responsive throughout, the CPU does get 100% utilisation during the build.

 

FYI, we were finally able to do a direct test where the only thing we changed was the CPU, from a 5800X to a 5900X. In my particular build case the 50% more threads made no difference. It was actually a bit slower. I was hoping that the larger L3 cache might help, but it didn't. Note that it didn't help in my case because my build type spends very little time in the threaded aspects of the build, normally only 1 or 2 minutes. The test was in 2020.3.13f1.

I have noted that keeping the CPU very cool helps a lot, allowing the CPU to stay maxed out.

So one of the takeaways is that the impact of more threads will greatly depend on your particular build. Pull up the resource monitor and see how much time your build spends maxing out all the threads. Part of "Compile WebAssembly" is threaded, part is not. Same with "Building native binary with IL2CPP." Unfortunately, Brotli compression is not threaded. If a lot of your time is spend in those threaded aspects, then more threads will help you a lot. If not, then it would be overkill.

Overall, we're still about twice as fast as we were before, and I wish I knew what exactly made the difference, the Gen 4 SSD, the 3600Mhz RAM, maybe the B550 motherboard, maybe the Noctua cooling, or the jump from an i7-8700 to a Ryzen 5800X. I really wish I knew how much of the improvement each components are responsible for.