Official Improving iteration time on C# script changes

Did you file a bug report? Awesome that you identified an immediate soluton.

 

You cannot fix what you do not actively track.

The Editor Iteration Profiler is not actively maintained and no longer works as expected.
However, the Standalone Profiler is more than adequate.



It seems as though the CompilationPipeline does no caching? For me, this would be the single most significant change. Am I naive to think that it could be cached within a single ReloadAssembly?

@TheVirtualMunk finding a bottleneck where script compilation is now dependent on network latency really highlights how much of a blindspot iteration time is. Is there a way that there could be a benchmark for Unity's iteration time, and variances from there could be tracked?

Similar to Firefox's "Are we fast yet?" project.

_______

After restarting the editor, the profiler reads very differently. Seems like with each iteration, ReloadAssembly takes more time...

 

No I did not

 

I would honestly be good with automatically submitting details on every compilation and playmode-enter to Unity if it helps with iteration times.

 

Here's an interesting article on the subject of domain reloads in Unity.
It's an insightful short read that mentions a lot of what is being talked about in this thread!

 

Thanks for the article! I'm using 2021.2.*, and so this profiler is integrated under the standard profiler, correct? The article might be out-of-date? When playback is stopped I take a sample and it reports the following --



It isn't clear to me how to tie this information to app domain reload or activation of game objects. What does one do with this knowledge?

 

:

Yes, it is slightly out-of-date because like you say the article mentions that you need to download it in a separate package, but that's no longer necessary. However, I don't think that makes the whole process out-of-date.

I haven't gotten around to doing this yet. I'm looking into it right now.

Here's what I understood from reading the article:
The memory profiler is a place where you can judge what files are taking your memory which is linked to longer assembly reload times. (High memory usage => Longer assembly reload).

More concretely, he mentions how he fixed an issue with the example of a file that took up a lot of memory space. Not sure how he dealt with that file: either by deleting the file or making sure that this file does not get loaded.

The article states:
"This explains why Domain Reload times tend to grow as the Editor stays open: as more objects are created, Unity must save/load more state."
=> This could mean that memory profiling is most important for people experiencing longer loading times as they keep the Unity Editor open. But I don't have that problem myself, so I can't say for sure.

In your case, what you would do to optimize would be to unfold Assets in the memory profiler, and to starting looking at what takes up the most amount of memory, if there's something that has a high memory footprint, make it lower (modify or delete the file).
To not do efforts for nothing, I myself am first going to try this by copying my project and deleting those high memory files and see if I even get any significant performance gains.

For example, in my personal project, I can see there's a shader that I don't use that's taking up quite some memory space. Maybe you have similar things in your project?
https://i.imgur.com/h3S4Na4.png

Lastly, I find this point in the article interesting:
"You don’t need to worry about asset size. Native assets like textures, models, audio, or any other data that is stored in the native layer (ie. the C++ core of Unity) do not get serialized during this phase. What matters is the size of the C# classes.
At the end of the day, this step is likely going to take a second or two. But at least it can be lowered with a bit of careful work."
=> He says this step properly handled would take around 1-2s.
However, I don't know where to benchmark that step, certainly it's in the profiler but I don't know what is its name.


Anyway, asking the man who wrote the article on Twitter is surely better than me because I'm mostly doing conjecture here.

 

Any effort to tackle this problem is appreciated! Not shooting the messenger here but I am going to poke holes in the author's assertion that slow editor iteration time is an avoidable problem ... if we merely take the appropriate steps to address it. I'll tweet the author for an opportunity to respond and I'm appreciative of the post sharing this article.

I'm not angry at the guy -- I just greatly disagree with his assertion:

That is quite a statement. Let's evaluate each recommendation to achieve that sweet sub-5 second iteration time. I have reload times that are 12 seconds minimum and up to 35 seconds ... for single line script changes on high powered hardware.

This is on for users by default, isn't it? Let's move to the next suggestion.

Most users in this thread report the issue is not compile time but assembly reload time. The creation of many fine-grained asmdefs apppears to increase app domain reload time. Be careful before embarking on such an effort -- you'll want to track timing before and after.

The suggestion to minimize serializable state in the scene hierarchy is helpful but many of us who suffer greatly are already doing this. The author asserts that the existence of serialization state increases domain reload time proportional to the time the editor stays open. Why should domain reload time grow just because the Unity editor remains open? This would appear to be a Unity bug and not a user-space issue.

My game's scene memory is just 2MB.

This is helpful and a consideration that many of us have addressed. We can't remove the initialize on load callbacks in Unity packages, however.

The article asserts that most projects should be able to achieve 5 second reload time regardless of overall size. I couldn't disagree more.

The issue of editor iteration needs to be a continued focus of the Unity dev team. Iteration worked quite well in 2019.x and now it doesn't and their own benchmarking shows as much.

 

That makes sense, and I've had similar experiences.
Still I'd try anything I can to make my project reload assemblies faster.
There might be things we don't understand so it's a good initiative to have contacted the author. The article is from 2020 so he might have been on Unity 2019 as well.

Regarding memory:
Looking at my project, I just don't have that many specific C# classes that participate in having a high memory footprint, so there doesn't seem to be much that can be done looking at the memory profiler. (in my project at least).

 

The memory profiler that the article links to is a package and in contrast to the built-in one also shows the managed memory and how much memory the managed fields take up for each of managed object.

 

Yeah, the memory profiler is an interesting tool.
For example, I saw that there were many duplicates of string being loaded up in memory from Unity packages.
Looked like a bug so I reported it.

 

I came back to Unity (2022 beta) after working on other projects for two years, and now an empty project created from the 2D mobile template (with unnecessary packages disabled) with one script file takes ~10 seconds to reload assemblies after every source change.

Unity used to be quite fast at this so this concerns me greatly. Is Unity maybe traversing ALL of the assemblies when they look for script attributes? If so, you should limit the scope (i.e. don't traverse assemblies you know won't contain said attribute). I am sure there's a ton of optimisations that could be done on iteration time if there was a will to optimise the FUNDAMENTALS before working on esoteric things like DOTS.

Thank you.

 

Unity 2020.3.31f1 seems to have a related change (since this topic is about iteration times, of which entering play mode is a part of):

In my case, though, "can significantly" turned out to be "slightly".

 

What the heck has happened to Unity?
The iteration times have become impossible to manage.

If i merely touch anything in the inspector I have a 10 second wait on my hands while the rainbow spinning wheel spins.
If I ever try to save anything I have a 10 second wait on my hands while the rainbow spinning wheel spins.
If I adjust a variable, I have a 10 second wait on my hands while the rainbow spinning wheel spins.
If I start to play a scene, I have a 10 second wait on my hands while the rainbow spinning wheel spins.
If I start to pause a scene, I have a 10 second wait on my hands while the rainbow spinning wheel spins.
If I try to stop a scene playing , I have a 10 second wait on my hands while the rainbow spinning wheel spins.
If I adjust one small thing in a script I have a 30 second wait on my hands while the rainbow spinning wheel spins.

If I breathe anywhere near the keyboard, I have a 10 second wait on my hands while the rainbow spinning wheel spins.

Perhaps no one at Unity thought that this would be a problem.
I would politely disagree.

Every year it seems to become worse and worse to use.

 

It'd greatly benefit Unity if they had a large project with say tens of thousands of textures and tens, if not hundreds, of source files to simulate a real world project... and force every developer to use that project in their daily testing of features while working.

I am concerned that the Unity developers may be detaching from reality working in their small islands of empty projects now. Rule number one is always to use your own tools.

 

The only advice that reliably works is to remove as many packages as you can afford to, even if it means recreating the functionality of some packages yourself.

Sadly, Unity gets better when you use less of it…

 

One answer to this question can be the practice of marking editor-only Objects with HideAndDontSave. Those objects seem to survive domain reload. I think it's known that they should be destroyed manually, but I've found that it's still common to forget destroying when using static fields to store them. Even in Unity's code there are cases of this. It can happen like this:

Code (CSharp):

1. public static class SomeClass
2. {
3.     private static SomeUnityObject s_AnObject;
4.     public static SomeUnityObject anObject
5.     {
6.         get
7.             {
8.                if (!s_AnObject)
9.                {
10.                    s_AnObject = CreateAnObjectInstance();
11.                    s_AnObject.hideFlags = HideFlags.HideAndDontSave;
12.                }
13.                return s_AnObject;
14.             }
15.     }
16. }

One could think that the object assigned to s_AnObject doesn't need to be destroyed, because it's only assigned once and then reused, but a new object will be created every domain reload, while the previous instances will still exist.

I don't have a prefect solution for this problem. For cases where it's Unity's code, there's not a lot that one can do. Maybe reporting it I suppose, but it's a bit tricky to make the effects clear and evident.

Sometimes the HideAndDontSave flags aren't really needed. They prevent objects from being destroyed when opening a scene or changing play mode in the editor, and they stop an object from being saved when it's not an asset and it's referenced on a serialized field in a scene. So, if the object doesn't need to survive that long, and they won't be assigned to a scene field, they could do with those flags.

Other times, there are ways of reassigning an existing instance to the static field after a domain reload. For example, a custom ScriptableObject singleton could assign itself to the static field in its OnEnable.

I think these problems need a long time and/or a lot of objects to be really noticeable. I've found it a couple times after using Unity without closing it during several days, by making my computer sleep instead of turning it off. It's a bit rare, but that's a way in which it can happen.

 

LOL
More like from 2s to 30-35s

Development has become completely impossible
If its not endlessly waiting, its freezing up.
Didn't anyone at Unity consider the possibility of testing the software on actual projects before performing a rollout?
As far as I can tell Unity is finished as a company as its software has become COMPLETELY F***ING UNUSABLE

 

I'm using 2020.3.30f1 and the wait times have come right down - I'm not sure if this was in line with going from Visual Studio Code (which I've used for years) and now using Visual Studio Community 2022 (17.1.1), which is much better.

Not saying this is a solution, but what is everyone using?

I was getting quite awful wait times on two seperate projects (one heavy and the other much less bloated), now reduced to a couple of seconds.

 

I am using 2021.1.19f1 on a M1 mac
Whatever you do, make sure you do NOT update to 2021
Its complete S H I T

 

Current primary project is 2020.3.31f1. On an i9 12900kf, the wait times are sub 1 second. Said project is neatly organised into a number of assemblies.

In the two times so far I attempted the update the project to 2021 (unsucessfully), I didn't notice a difference.

 

I'm seeing similar improvements going to VSC 2022.

 

updating to VCS2022 sadly didn't improved anything for me.

honest question: unity 2019 LTS support runs out in june, is there any way to continue the LTS support until this is fixed in 2020 onwards? to me 2019 is the last useable unity version if i am honest

 

Why would updating VS improve anything inside Unity?

 

¯\_(ツ)_/¯ just grabbing every straw i can find to get rid of...

 

The amount of stuff Unity does when reloading assemblies has grown so much it's probably affected by the direction your computer is facing and how well organized your desktop is.

 

Proper feng shui affecting Unity makes more sense to me than installed IDE version.

 

You’re using a CPU that has the fastest single core performance of anything sold on the market. I would hope that it’s quick!

I’m using a i9 9900K and find that it can fluctuate between 2-15 seconds. Depends on which direction I face my computer.

Neat assemblies cut down on compile time (which is sub 0.02s for me). Domain Reloading is the problem. Could this ever become a threaded task?

 

Please, Dear God, fix this ASAP. This "feature" (bug!) is causing me to smoke more cigarettes!

"Reload Script Assemblies..." for 2+ MINUTES??? CPU load is carried by one core.

30 seconds is aspirational
1 minute is typical
2 minutes is not unusual
My record is somewhere above 5 minutes.

 

What about incremental compilation? Could Unity add this (If it makes enough difference)? There was an experimental asset for this a while ago: https://docs.unity3d.com/Packages/com.unity.incrementalcompiler@0.0/manual/index.html

Nowadays PCs are so extremely fast, compiling shouldn't take so long, when changing one line of code.
Even though people are complaining (tbh me too), Unity is trying to fix these problems, that users have been fighting with for a while, so let's try to appreciate these efforts.

 

Compilation is not the problem, that's actually pretty fast. Domain reload is what takes the longest.

 

exactly, script compilation (empty project) is just 1.3 seconds as you can see here.
asset database v2: refresh internal/ import out of date assets is where it gets nasty. almost 2 seconds
reload assembly is 90% of that.
leading to 4 seconds of staring at a bar (to be fair, its 1 second less if i turn off the profiler, so 3 seconds)

if you open up unity 2019 and switch from asset database 1 to asset database 2 (in u2019 you still had the choice) you see your compile times go double

 

In my projects I always have domain reloading and scene reloading disabled. Unless one uses static classes, I don't see any reason to leave it (domain reloading) on.

 

But domain reload only happens when you hit play, or? This whole thread is about script compilation, not entering playmode

 

It does happen when the play button is pressed, but it also happens whenever there are script changes.

 

tbh i dont see any difference between turning reload domain on or off

 

Pretty sure this setting is only for entering playmode. You can't get around domain reload when changing scripts.

 

Just an idea

 

I am sure the internal assemblies are looking up attributes and stuff, but don't limit the assemblies at all so they all traverse through every assembly. Please add normal optimisations such as culling assemblies you know aren't relevant etc. It's funny how they focus on some gimmicky custom burst compilers while just taking a pass of a single week on the editor internals and optimising how the vanilla objects are loaded in the runtime would yield a benefit 1000x higher than the theoretical things they've been working on for the last 5 years.

 

Eh in some other thread it was said that they focused on a number of optimizations that are also included in the newer alpha builds. Unfortunateky without really massive improvements. They certainly spent more than a week on it however. So guess there are no low hanging fruits left.

Btw. I'd definitely not trade in-game performance foreditor performance. The former is way more important and the burst compiler is quite a gem - my current project would not be possible without it.

 

Hey everyone, author of that article here. Glad the article was at least a little useful, although I wish it had better suggestions -- there's really no silver bullet, other than continuing to push Unity to prioritize things that will remove domain reloading entirely (ie. DOTS, .NET Hot Reload). They're working on that, at least.

The '5 seconds' is perhaps a little optimistic, although I do still believe it's possible with enough work. In our case, our reload times have crept up to around 8 seconds, which is passable, but not amazing.

Unfortunately the work required to get these times down tends to be very heavy on the engineering side, so you may have to find some help if you're not comfortable using profiling tools.

In my case, the asset was a ScriptableObject that was storing a massive string. To reduce the memory size, I compressed the string with the .NET compression libraries before storing it into the ScriptableObject. This helped a lot for us, because the asset was highly compressible, but probably won't apply to most cases.

If domain reload time is slowly increasing as the editor stays open, that's a bug. I haven't seen that, though. In the article I was just referring to the fact that the longer you use Unity, the more objects you tend to create (you open more windows, load bigger scenes, etc, etc).

----

Overall, though, if you're seeing reload times of more than a minute, something is *seriously* wrong with your project. I'd bet you have an asset from the asset store that is causing issues with domain reloads. An easy-ish way to track this down might be to make a backup, slowly delete files from your project, and track how much faster the reload times are with certain folders deleted.

Keep in mind you'd have to be careful to keep the game compiling, if you want to do a good test.

 

Keep in mind ReloadAssembly is just the top call, alot of store assets (popular ones I might add), do sneaky things on domain reload. I just cleaned up 12 seconds of assembly reload by editing asset code to not do these things or just outright removing them from my project.

If you dig deep enough you'll see full script loads, iteration of all assemblies and classes looking for some silly attribute they add. Non lazy loading of icons / textures and other random stuff that makes dev / publishing easier but poor performnace at user expense.

Example code I stripped out, it loads all scripts looking for attributes to assign script execution order - which by the way doesn't make sense because that is stored in the .meta file anyway...

Code (CSharp):

1. foreach (var monoScript in MonoImporter.GetAllRuntimeMonoScripts())
2.             {
3.                 if (monoScript.GetClass() == null)
4.                 {
5.                     continue;
6.                 }
7.  
8.                 foreach (var a in Attribute.GetCustomAttributes(monoScript.GetClass(), typeof(someattribute)))

 

If your project is sufficiently large (based on number of scripts, number of assets stored in asset database) ... none of the steps proposed across the five pages of this thread, and 20+ pages of other threads, will achieve editor iteration below 5 seconds. Having literally worked through every optimization proposed here I have no remedy beyond Unity fixing the severe regression in iteration performance. @xoofx

 

It's taking upwards of 20 minutes for Reload Script Assemblies to finish and I'm losing my mind.

For example, it's taken me an hour to get the Editor Iteration Profiler imported and enabled.

20-minute Reload Script Assemblies when the files were imported.
20-minute Reload Script Assemblies after import.
20-minute Reload Script Assemblies after opening up the Editor Iteration Profiler.

 

Just tried the new 2021 LTS version, now I need to wait for this annoying popup whenever creating a script.

I don't get it why Unity need to reload for such an empty script? It doesn't do anything!

 

Creating/deleting a script is still introducing/removing a class from whatever assembly it lives in, regardless if there's any logic inside of it. Of course it has to recompile.

 

This popup does not show on older Unity version so i didn't notice that, thanks for clarify this!

 

20 minutes means YOUR project has an issue. It's highly likely that this is due to a script in your project that is doing something during the Reload Script Assemblies.
You would do well to read this thread and optimize that/those scripts.

 

I like how we went from the (probably very optimistic) promised 500ms iteration from Unity which would mostly solve this for everyone -> to “no actually it is the users’ responsibility to fix this, here are a million things they need to change to maybe shave a second or two”

 

Upgraded from 2019 LTS to 2021 LTS yesterday. I feel like it was a mistake. Everything is slower. The Tilemap tools are still garbage after how many years? Can Unity please put someone on this?

And we still have this gem in the log on every launch.

In order to call GetTransformInfoExpectUpToDate, RendererUpdateManager.UpdateAll must be called first.
UnityEngine.GUIUtility: ProcessEvent (int,intptr,bool&)
​

Long time supporter and champion of Unity. I'm really fairly disappointed in them now. I can surely appreciate how hard it is to maintain a beast like Unity and serve all the different needs of developers (and the corporate weasels that push things in the wrong directions). Too many things take too long to address. Core tools like Tilemap tools are completely in a "programmer made" state vs being a serious dev option.

Iteration speed is key and has always been good in the past. Even on simple, small, 2d tilemap projects, the iteration times from 2021 to 2019 are longer and noticeable and frustrating.