Over time, my compile times increase. Restarting Unity fixes it, briefly. Why? A fix?

I've been having an issue, which I know I've seen others mention online a while back when I was looking into how to decrease my compile times. Did some googling, learned about Assembly files, and implemented them. My compile times dropped from like 28 seconds to about 2.8 seconds. Woohoo!

But - the longer Unity is open, the more updates I do, the compile times each time that I save/update a script and hit play increases. Yesterday I was at about 9 seconds. Now, after saving a file, I'm around 11 seconds.

If I take the time to shut down Unity, and re-open it, my compile time will reset to about 3 seconds again. But tomorrow it'll be up to around 5... then 7 or so... and in a few days it'll become unbearable and I'll need to restart Unity again.

I've tried googling this problem. The issue is Google is trying to be SO helpful that I get pages upon pages of hits on how to speed up my compile times using Assembly definitions. Yes, thank you Google. I already know that. Almost all those links are highlighted as visited, because I did all that a few months ago.

While I was doing that original research I do remember seeing people complaining about this exact issue, and that restarting Unity would (temporarily) clear it up. Thats how I knew to try a restart.

Anyone know why this happens, and if there's an easier/faster way to stay on top of it, aside from a restart?

I'm using AsmdefDebug, and it's telling me that of my last 11.93 second compile:
myAssembly.dll is .58s
Assembly-CSharp.dll is .43s

And Assembly Reload Time is: 10.918s

It's the assembly reload time that keeps creeping up.

Any help?

 

Trying to find a solution I came across this link which I read a few months back, one of the many detailing the same issue and recommending restarting unity as the "solution".

https://answers.unity.com/questions/1628931/reloading-assemblies-is-extremely-slow-how-to-fix.html

 

As far as I know, this issue was introduced in the first 2020 version of Unity.
I'm experiencing it as well and haven't encountered it in 2019 or earlier versions. Unfortunately, I don't have a solution to offer; I've just been restarting Unity as well.

I don't know if the compile time increases the longer Unity is running, or if it increases every time scripts have to re-compile, or maybe it's both. Either way, it's become pretty annoying and I can't find a solution for it anywhere either.

 

Nah I've been experiencing it with 2019 the whole time I've used it. I'm thinking that it MIGHT have something to do with some [InitializeOnLoad] attributes. I've seen a few posts pointing towards that and I found a few really unnecessary instances of it in scripts I've gotten from the asset store (making sure the asset's readme file was in the editor's window menu, etc).

Unfortunately, I had planned to deep dive this issue today and see what I could do, but Windows decided that last night was a good time to forcibly reboot my computer. So I had to start Unity anew, and my compile time is 3.2 seconds. I'll be a few days before it creeps back up and profiling will be of any use. I'll post any updates if I find anything useful.

 

Not sure about that. I don't have any scrips with any [InitializeOnLoad] attributes in my project.

 

Thats good to know. There are a few attributes that work like that, but all I've seen have "InitializeOn" in them somewhere.

Once I get back to the point where it's taking way too long (and I can look into where that time is being spent) I'll be able to find out more.

 

Haha good guy Windows. Always comes to the rescue!

I'm interested to see how this saga unfolds. Poetry aside, there are some bizarre issues popping up occasionally and even weird external factors at play, and I hope issues such as this one will be ironed soon enough. Probably completely unrelated, but I had a rather problematic slowdown when editing texture import settings that rendered my 2020.1 completely unusable. You wait 5s to click anything, you wait 5s for that click to settle down and reflect back on the inspector, the selected texture appears to be compressed every frame, the asset database rebuilt every time, and all of that without hitting the Apply button at all. It worked like this with any texture, and when I'd unselect the texture, everything's fine again. So I manage to close it eventually and try to dig deeper. But then, after I decided to update my NVidia driver, just in case (for a card that's too old for this to matter), and when I rebooted, good guy Windows got to finish some freaking update, as if I haven't had to let it update 3 days ago. And now it all works just fine. I don't know what to make of it.

Related to this -- because I was afraid this is what hit me -- I've seen an article of a change in Unity that made it import/compress textures incredibly slowly (2-3x times slower than versions 4.x), and this was going on for years. It turned out that the file stream was reading 3 bytes at a time, due to a change that should've improved the speed of import, for thousands of users, version after version. Whaddyaknow. Year 2020. *shrug*

 

Yes, I've noticed this as well, but it's not only related to compiling. The longer Unity stays open and the more you switch around (maybe open / close other programs) Unity seems to slowly get out of control of it's "internal flow". I was once at a point where Unity was kinda stuck in an "Application.Reload" cycle, even when idling inside Unity. Every few seconds the busy dialog pops up for a few seconds, even though I do nothing, To me that looks like some sort of background task is spawning several times and they seem to "pile up". This might also be the reason why Unity looses track of the VisualStudio instance from time to time and decides to open a new one. It's all very inconsistent.

It's hard to pinpoint the issues, so it's difficult to actually formulate a bug report properly. I do also suspect win10 and it's glorious memory management might play a role in the issue. I have 12 gigs of memory but windows decides to swap out some applications even though half the physical memory is empty. Not having an SSD doesn't help here ^^.

In general it seems Unity locks up way more often and longer since they introduced the busy dialog. I didn't care if Unity got stuck for 3 sec and did not respond. Those 3 seconds can now easily get up to 15 or much more in some cases. Also there are some obscure reasons given in the busy dialog

 

This is the feeling I also had with my seemingly unrelated problem. But there is this underlying process that is quick to resurrect after waiting for something anonymous and silently failing, and leading to some sort of memory leak or process leak over time, I'm not sure how to call it without proper analytics, but it incrementally constricts operating resources. I was just "lucky" to see the full magnitude of this, but sadly didn't do much to inspect what the hell got loose before it got fixed.

It appears that the (un)expected weird win10 update thing fixed things on my end, while my codebase is still not big enough for me to be able to report slowdowns when compiling.

I also have this feeling of uneasiness with the busy dialog. I've increased the delay to 4 to no avail. 2020.1 is definitely around 30% slower for me than 2019.1 was. All around. It's easier for me to perceive these things because my current comp is kinda potato-ish (but I use an SSD, and this makes all kinds of bad things unnoticeable, but the 3-byte read I mentioned in the previous post makes me sad).

 

So its been a few days since my Windows10 forced reboot, and my compile times are not yet horrible. They've gone up a tad, but in general are still within what I consider "reasonable".

I just made a small change in a file and saved it, switched back to unity. 5.01 seconds compile time. 1.75 seconds recompiling my main assembly and assembly-csharp, and 3.26 seconds reloading assemblies.

Since I never actually benchmarked how long it took for the compile time to get annoyingly long, it's hard to say if this is normal until a few more days pass.

I did make some changes that could be affecting it. After reading about the InitalizeOnLoad family of attributes, I did comb through my code and found a few possible culprits which I removed.

I had previously installed a free motion animation asset by Invector, which put in a menu item, and spawned a splash screen when unity started up. Removed that. Have Asset Hunter Pro which was adding some menu items and seemed to be doing way too much processing setting up a readme file. Just removed that entire package. A few other packages were helping themselves to setting up menus with a bunch of things that were InitializedOnLoad. Anything that I knew was unnecessary or harmless I stripped out.

So far it SEEMS to have had an effect. But again, its hard to say, since I never benchmarked how long it took before the compile times got out of hand.

I believe my assembly reload time was around 3 seconds after the reboot. It's 3.26 just now. After a few days thats... tentative good news? We'll see if it creeps back up.

 

So it's been a few more days. Yesterday I did not touch my app at all, and left it running in another Unity instance as I wasted an entire day trying to get google Firebase to work correctly. That was fun. But for another post.

While I worked on it I ran a second and third instance of Unity doing testing on other projects, with my main app minimized in the background.

This morning, realizing that Firebase is more trouble than it's worth, I came back to my app intent on designing my own login system. I'm doing this in a new scene - so none of the multitudes of gameobjects and active scripts on my main app scene are loaded or active. My Assembly Reload time is now up to about 4.5 seconds.

Thats up 1.25 seconds from 2 days ago at 5:43pm, and for a 24 hour span in the middle, I didn't touch the code at all. That leads me to believe that something is "getting worse" even when I'm not actively using that instance of unity, compiling code in it, or changing anything.

I'm going to wait another day or so (or maybe tonight, if the delay ramps up to unbearable levels) before doing any actual testing. But the fact I'm losing 1.25 seconds more than I was 43 hours ago, without even touching it for over 24 hours is telling me something's going on that has nothing to do with anything I'm actively doing.

 

And after a day of heavy coding and work, my assembly reload time is up to 10.85 seconds. Diving into the profiler, I'm not seeing a ton of useful information. Its incredibly frustrating. I'm seeing a lot of time used in EndReloadAssembly... 10.61seconds to be exact.

ProcessInitalizeOnLoadAttributes gets 5.3 seconds
AwakeScriptedObjects gets 2 seconds
RestoreManagedReferences gets 1.6
then after that it's a lot of small stuff, starting with didReloadMonoDomain.Invoke getting 434ms

Drilling down ProcessInitalizeOnLoad, the biggest culprits are:
PluginContainer..cctor at 3 secs and
RiderScriptEditor at 1.6 seconds.

Not sure what I can do about the RiderScript Editor. I'm real close to rebooting Unity as I can't handle this 11 second wait every time I change a script then another 11 seconds when I hit play to test it. I'll check right after a reboot to see what Rider is doing, but I cant imagine thats growing a ton.

PluginContainer is a bunch of stuff, the biggest being PeekConfiguration..cctor at 1.5 seconds. Removing Peek to save 1.5 seconds is something... but there's just a big pileup of other stuff.

No one thing in particular screams that it's causing an issue. It looks like everything is creeping up in time equally.

Ok, I give up. I saved that frame data and I'm gonna reboot Unity and take another deep profile of the startup and see what the two frames look like, especially those big frames.

Ok. Restarted. Did an identical frame grab from the same scene with no changes to anything.

EndReloadAssembly taking 3.5 secs

Inside there:

ProcessInitializeOnLoadAttributes: .94 secs
RestoreManagedReferences 612ms
AwakeScriptedObjects 581ms

Inside ProcessInitalizeOnLoad:
PluginContainer..ctor: 552ms
RiderScriptEditor..cctor: 158ms

Way down inside PluginContainer... PeekConfiguration..ctor - 164ms

After a few days of use, most of this stuff is taking 10x as long to run. There's no reason whatsoever that something that takes 164ms to run on a fresh boot should take 1500ms+ to run just a few days later. I absolutely guarantee that I'm not changing the code inside Peek. And some quick googling of Peek causing slowdowns and memory leaks in Unity reveals... absolutely nothing.

If Rider was causing it (that also went up by 10x the load time) there'd also be mention of it online somewhere. There's not.

Something inside Unity itself is causing this memory leak, or disk leak... or something. Nothing stands out as EVERYTHING takes progressively longer to load once Unity has been open a long while.

 

A small update for anyone watching the thread. Last week I added a new feature to my app that allowed for some modular stuff. I didn't want to mess with defining a bunch of stuff, so I went to add it as a Resource or Addressable Asset. Since they were prefabs, and I'm not super familiar with all that, I googled to see which was the best way to go.

And I found on the Unity page for Resources, that the best practices for using Resources is, in bold italics, do not use them.

Oh. Well. S***. If it's so absolutely bad, why the hell are they so easy to use? I understand backwards compatibility, but jeez... I load icons on the fly based on text data from an external source, and they are all in a Resources folder (and I added that code MONTHS ago when I was still seriously learning Unity). Did some research and saw Resources = BAD and addressables = good.

So I revamped all my code that loaded icons from resources, hacked together something that allowed for addressables annoying non-blocking async behavior to still work and replace my old synchronous Resources.Load code, and moved all 200+ icons out of the Resources folder and deleted it.

I've now beed coding for oh... 4-5 days now? And my compile time is hovering around 5 seconds. I have assets that still use Resources - if those are the cause of the problem there's not much I can do about that. But if the resources were somehow memory or disk leaking icons all over the place, at least that hole is patched?

I can't say for sure thats the issue, but doing my research I did see a bunch about Resources needing to be unloaded correctly. I didn't read too much into it since I was dumping the system. But I have noticed my compile time appears to be NOT as bad as it was before. In another few days will really tell the tale.

Anyone else in this thread whos experienced the issue use Resources?

 

Not sure about this one either.
I'm still having the compile-time issue & don't have a Resources folder in my project.

Maybe something else worth noting: my current project is pretty much nothing at the moment; I've so far been spending all my time writing custom components and editor scripts. The project itself is just one scene with like, 40 GameObjects in it. I don't think project structure is related to this issue.

 

Yeah, big fail on that hopeful thought. Had to restart today when my compile time finally crept up to 11 seconds. I think that was confirmation bias... I WANTED it to be the fix so it felt like the compile time growth wasn't as bad.

I might see about setting an external program to watch key directories, and see if they grow over time (in any rate that matches my compile time) and what happens when I reboot.

The worst part about this is it takes literal days to notice an increase - so debugging whats wrong can take forever. I can't speed the process up. I'm considering making an AutoHotKey script to change a source file, save it, tab into Unity, hit play, hit stop, tab back, change the file back, save, etc etc and see if I can speed up the process. Even then it'll still take hours to get a big enough change. My coding style is to make small changes, try it, fix bugs, try it, add more code, try it... so I am constantly saving, recompiling, entering playmode, etc, and even then it takes 5-6 days before it gets unbearably long.

 

If I have to guess I would bet it's their implementation of Mono.

Mono links the c++ engine to the C# world, but they went as far as building big chunks of the editor itself on C# side.

I would guess Unity generates 2 app domains, one for the editor and another for when you enter play mode. When leaving play mode, it swaps back to editor domain and cleans up the play domain... However, the editor mono domain is never shutdown until you close the editor. So tools are used and garbage memory piles up, every assembly reload the editor have to scan mono objects to reserialize data and refresh their displayed info on inspectors, but the amount of mono objects just increase overtime regardless of garbage collection, you cannot completely clear an assembly image until the mono domain is shutdown, so every compilation is going to be slower and slower.

A definite solution to that would be to reload the editor domain from Mono API, but I guess if they do that they kill the entire UI since they have decided to write it in C#

Just a thought from a guy who have implemented mono based applications.

 

I have the same problem, but when I restart my Razer RGB light controlling bloatware the compile time drop from a minute to almost instace, I think Unity don't work well with certain other softwares for some unknown reasons.

 

Still having this issue in 2022.3!