Official Incremental script compilation

-If you change a script in an asmdef, only that asmdef or any code depending on that asmdef gets recompiled.

Wasn't it always like this? I thought that was the point of using asmdefs when they were introduced

 

I remember something about an assembly triggering compilation of other assembies only if the public API changes. Is that part of this?

 

From what I've heard, yes. In practice, that horrendous freeze increased for me. Compilation is what, less than 10% of the freeze? It's domain reload that's the issue. {Unity .NET Core rant}

EDIT: In other words, does this compilation pipeline rewrite really matter right now?

 

What happened to AssemblyLoadContext? Is there some sort of misunderstanding regarding what it can do?

 

This is great news, very much appreciated to see Unity push hard on iterative workflows! I miss Unity 4.3 days where, nostalgia or not, everything felt snappier, a few more of these and Unity will feel snappier than ever

 

You stated that CompilationPipeline.assemblyCompilationStarted, and implied that CompilationPipeline.assemblyCompilationFinished will be deprecated.
Does this also affect CompilationPipeline.compilationStarted and CompilationPipeline.compilationFinished?

If so this is a problem for a tool I'm working on, I specifically need to know when compilation completes so I can overwrite certain files.

 

Do u have an ETA when this domain reload only the assembly that has code changes? I think I already asked this question quite a few times but until now this feature still haven ship yet.

 

What about incremental script compilation for Burst? Couldn't static methods that are completely independent be recompiled safely even in runtime? That'd be similar to Unreal LiveCoding feature.

 

Will any of these changes affect editor iteration speeds or is it only standalone that benefits at the moment?

 

I've been working on a tool to assist in modding unity games that just helps sets up the environment for a given game. In Order to do this, after compilation completes, I need to overwrite files in ProjectRoot/Library/ScriptAssemblies in some cases, such as Assembly-CSharp and Assembly-CSharp-firstpass in order to ensure that the libraries correctly get loaded and are usable in the unity environment.

Obviously this is a giant hack, and its for type of work that is largely giant hacks.... so take that as you will.

Edit: Just wanted to add that, this obviously does cause various problems. For Example, If you create a script in Assets which isn't managed by an AssemblyDefinition then it would get compiled by default into Assembly-CSharp, but since that gets overwritten the script isn't loaded correctly and Unity throws various errors.
This is honestly expected, you also run into issues with various assemblies still not being loaded correctly. This is more or less acceptable since the goal isn't to allow someone to completely boot up a deployed game inside unity, some imperfection is warranted, and possibly even wanted.

 

My feedback so far is that this popup modal:


needs to go away and the stuff it's reporting on should be relegated to the Background Tasks window as soon as possible. However you guys manage to solve this, it should be a priority because it's a very stifling experience iterating on script changes currently.

The Flax 1.0 engine editor shows its script compilation progress in the bottom-right status bar (like Unity 2021.1 does) but they don't block their editor. Best I can tell is they just prevent entering play mode until script compilation is complete but their editor remains responsive to input.

I do notice overall that the time it takes to compile a simple edit to one script file in my current project reduced by a noticeable amount. It wasn't a lot but it was something. So this is promising, just please......get rid of that modal!

Thank you.

 

I would love it if they could get rid of the stalling on script change completely too. When I first started using Unity years ago I was kind of surprised that it did that, was very jarring.

On a positive note I have noticed in the recent betas that the 'compiling scripts' is now no longer locking up the editor and appears to show as a progress bar in the bottom right corner. Then the domain reload (I assume) happens after, which locks up the editor for a few seconds. Slightly better experience than before. Maybe I'm only noticing this more now that I manually trigger the refresh...

I think I remember hearing before that the domain reload UI lock-up is a fundamental part of how Unity / the editor work and the isn't going away anytime soon. I guess it needs to do it if you have made script changes that affect the editor or something? [shrug emoji]

 

Yeah, it's pretty fundamental. Since the editor UI can be driven by user scripts, they have to reload the UI when user scripts change.

I'm not sure if it'd be possible to skip reloading the domain the editor lives in. Say your inspector is showing the script you just changed, and that script has a method that's called by a custom inspector's code. When you reload your script's assembly, what does that inspector do?

 

One solution would be not to reload the UI if the serialized properties or editor related scripts are unchanged. I think most people don't change these that often. But I guess that would be complicated to implement.

They already did something similar for not recompiling other Assemblies if the referenced assembly's public API didn't change. I guess @jonas-echterhoff would have a better idea of how complicated this would be implement, if at all.

 

I thought it would be easy to implement using Burst Modding support in 2021 https://docs.unity3d.com/Packages/com.unity.burst@1.5/manual/docs/ModdingSupport.html but it requires domain reload meaning that it can't be done in runtime :/

 

If it's true that the domain reloading is only necessary to reflect changes to exposed script variables in the inspector or editor windows, then an idea would be to provide a sort of "Refresh" or "Reload Domain" button that's easily visible and pronounced, that when clicked, Reloads the Domain (make it show a slick animating progress popup modal right away and dim the editor behind it).... That way (hopefully) coders can make quick simple script changes, hit Play, and see results without needing to reload the domain.

If you *do* make a script change that affects an exposed or serialized variable, then you can reload the domain when you want to instead of being always forced to wait.

Obviously still not ideal, especially for new users learning Unity who won't know that they have to click "Reload Domain" in order to see the new property they want to show up in the inspector, but......if this could be provided as an advanced option (since it's for advanced users) I feel this could go at least some way towards speeding up the iteration process.

 

Yep I like the idea @amisner2k suggests.

~98% of my DOTs based coding has nothing to do with inspector/editor script values nowadays anyways.

 

Does this mean it would be possible to avoid recompiling if the C# doesn't affect the generated IL? ie. The file change is whitespace only?

 

Hi

These changes ruined my Jenkins build flow.

Switching the editor platform causes script recompilation and (unlike Unity 2019) it doesn't wait until the compilation end but continue to go down via function. Right after the platform switch, I have Addressable build (called when editorPlatformSwitched event is fired) and it always fails because Editor.isCompiling is true.

I can't use semaphores, because this causes deadlock because Unity is still semi-async. I can't just subscribe to assemblyCompilationFinished event, because the function ends before this event will be fired.

What do I suppose to do, @jonas-echterhoff ?

My code:

 

This change causes a very painful performance regression across a wide range of projects, both simple and complex. I'm seeing on average 15 second delays every time a minor script change is made.

15s of waiting because I added a carriage return to a game script.


It raises questions concerning Unity's internal QA process and how such a huge hit to productivity was introduced into the product. Does Unity test releases with commercial grade solutions of sufficient complexity?

More concerning is that there has been no public acknowledgement that this issue is serious and that it is being given proper attention. Maybe it is being worked on and we will learn the outcome in this thread.

I appreciate Unity. And I appreciate the effort of Unity employees, including @jonas-echterhoff. But I don't appreciate that this serious performance regression has been known since December 2020 and eight months later it still exists with no discussed timeline for resolution.

 

For us it gets worse as I keep Unity open for more time, up to minutes of waiting. So I just restart the editor every few hours to make the script recompilation faster.

 

The unity team is gathering feedback in this thread.

https://forum.unity.com/threads/editor-progress-bar-stalls-request-for-bug-reports.1137817/

This is a known problem that is affected by several reasons, some are fixed, some are not.

 

I don't think it's the same issue. There are no unresponsive stalls here -- the Unity editor assembly load/recompilation process was rewritten and it consequently takes longer in Unity 2020.2 and 2021.x.

 

Hi @jonas-echterhoff , I migrated recently from 2019 lts to 2021lts editor, but now I'm having this issue on Android builds, where the build freezes at a random file, during the CopyFiles step of the Incremental Build. I tried to change the folder's permissions, reinstall everything, but it is still happening every time.

 

Hi, is it freeze on operation something like "Copy from Library/Android to something blablabla"? If yes, I have the same problem with the 2021 version: near the end of the build process, it's like "stuck" for no reason.

However, after waiting for around 30 minutes (on a computer with Ryzen 7 3800X, 32GB of RAM, and Samsung 980 Pro SSD and for the empty URP project), the process completes with success and the build worked as expected. For the next builds, this problem does not exist anymore.

Likely this is a problem with the incremental build system.

 

Hi there, we recently upgraded our Unity project from version 2020.3.36f1 to version 2021.3.11f1, and one of the things we've found recently is that the build.log file looks significantly different from how it used to. Namely, after an "Assembly Updater Post Process Assets" line in the logs, we used to get information about each of these .dll files compiling:



These would then appear for us over in GitLab in our CI/CD pipeline console (which uses fastlane and a u3d plugin) in a format that would let us quickly and easily identify compilation errors:



Since upgrading, we no longer get these file-by-file compilation logs in the console, and while we initially thought this was due to the fastlane plugin we're using we took a look at the build.log file and it appears these aren't getting logged in the raw output either (or, if they are, they're in a different format):



As a result, we're currently in a state where, if there are compilation errors, these don't appear for us in our CI/CD pipeline and we instead need to grab the raw ouput/build.log file to search for errors manually.

What we're currently hoping to find out is whether the changes to the log files and standard output during builds is due to these changes to the script compilation pipeline, due to some other change, or whether we've most likely got some quirk in our project's setup. If the former, we can perhaps look into adjusting our CI/CD pipeline to accommodate these changes.

Many thanks for any help or advice anyone can offer with this.

 

@jonas-echterhoff is ithis still working on Unity 2022.3.0 ? because our android I2LCPP build times are being EXTREMELY slow (30-40 min) and with the same machine iOS I2LCPP builds take less than 5 min...
Whats going on?