5.6.0b2: (Case 863912) Editor performance of Open/SaveScene and EnterPlaymode

I used Unity 5.6.0b2 to run a few editor performance tests to find out how much impact the number of empty game objects in a scene has on the time that the editor requires to open and save a scene, as well as enter and exit playmode.

The results seem to show that the time does not scale linearly with the number of game objects.

For example:
The editor requires 3.6 seconds to enter playmode from a scene with 10000 empty gameobjects on my computer. Having a scene with 50000 empty gameobjects requires 171 seconds to enter playmode. It's 5 times more empty game objects, but almost a 50 times longer wait.

I tested the project on a SDD and non-SSD hard-drive, as well as with enabled and disabled anti-virus software. The results were pretty similar at all times. Unity disk-activity during the Open- and Save-Scene tests was also very low, I don't think it's IO bound.

The project and tools that I used to run the tests are attached to Case 863912, in case someone wants to take a look at it.


Open Scene
Represents the time, in seconds, how long it takes the editor to open a scene with N objects. These game objects are either empty GameObjects (earlier created via new GameObject) or empty prefab instances (earlier created via InstantiatePrefab). Empty in this case means the game object has only the Transform component. The duration has been measured in Force-Text and Force-Binary asset modes.




Save Scene
Represents how much seconds the editor requires to save a dirty (EditorSceneManager.MarkAllScenesDirty) scene. Notice how fast "Prefab Instance (Binary)" saves, I'm not what happens there.




Enter Playmode (aka Play)
Measured using the EditorApplication.playmodeStateChanged callbacks. It's how long it takes the editor to switch from edit mode to play mode (Press Play).



What's interesting with the "EnterPlaymode" thing is that it seems the editor has twice as many "AwakeFromLoad" calls than game objects in the scene. For example, the following image shows a profiler snapshot of a scene with 20002 game objects in total, yet it displays more than 40000 AwakeFromLoad calls.




Leave Playmode (aka Stop)
Represents the amout of time the editor requires when pressing Stop. That is the time to switch from playing to edit mode.

 

Filesize comparison
The following image shows a filesize comparison of scenes with empty game objects or empty prefab instances in Force-Text and Force-Binary asset modes.
It's interesting to see that prefabs seem to add a good amount of data to the text-based scene files.

 

By any chance did you also test the equivalent compiled runtime impacts for comparison?

(I've never personally had a scene anywhere close to tens of thousands of GOs, but I'm curious.)

 

I did not, but I can do this during the following days. I'll update this thread then. I guess the only useful information I could profile in the Player would be loading times, but that definitely would be interesting to see if and how much of a difference there is!

Thank you, for going to improve it!

 

It would also be interesting to double check that 'AwakeFromLoad' count and see if its an editor thing or if it happens in a build too.

Either way I was hoping a Unity Dev might jump in and say if that is expected behavior or not - seems a little weird and might be worth entering a bug report for it just in case?

Overall though there generally seems to be a lot of overhead in playing in the editor, which might just be unavoidable, but still it is interesting to look at it.

 

Here is Unity's response to the bug-report I submitted.

 

Thanks for posting that, helps clear up a few things.

 

I profiled how long the Unity Player, or runtime, requires to load each of the scenes I measured earlier in the editor. I've also profiled asynchronous loading, but it doesn't really make a difference.

However, I've a few learning's taken from that.

1. Asynchronous loading does not work in the editor. The editor turns unresponsive while loading a scene asynchronously. It's pretty much like blocked loading.
   
2. Asynchronous loading in the built player kind of works, but still has at least one significant stutter for each scene I tested. My best guess would be that it stutters a lot worse when having actual content in the scene, making smooth async loading not possible.
3. The Unity profiler does not work in 5.6.0b3, it can't connect to the Player and outputs "Sending message header failed" every frame. Which means I can't check the AwakeFromLoad calls.
   
4. Closing the Player causes it to freeze forever and needs to be terminated using the Windows TaskManager.

I'm going to submit bug-reports for these issues.

The "Unload" time in the chart is how long Unity needs to load an empty scene. I tried SceneManager.UnloadSceneAsync, but Unity always complained "Unloading the last loaded scene <name>, is not supported". So my workaround was to create an empty scene and load this one. My assumption was that I can use this to measure how long Unity needs to destroy the previous loaded scene.

Spoiler: Loading code

Code (CSharp):

1.  
2. // Unload blocking
3. {
4.   long ticks = DateTime.Now.Ticks;
5.   SceneManager.LoadScene("Empty");
6.   yield return null; // wait one frame to allow unity to awake loaded objects
7.  
8.   _UnloadPeriod = TimeSpan.FromTicks(DateTime.Now.Ticks) - TimeSpan.FromTicks(ticks);
9. }
10.  
11. // Load blocking
12. {
13.   long ticks = DateTime.Now.Ticks;
14.   SceneManager.LoadScene(sceneName);
15.   yield return null;
16.  
17.   _LoadPeriod = TimeSpan.FromTicks(DateTime.Now.Ticks) - TimeSpan.FromTicks(ticks);
18. }
19.  

The timings make perfect sense to me. The time Unity requires to load a scene increases linearly with the number of objects in that scene. I'd expected this in the editor as well.


EDIT:
Here is the Unity project:
http://www.console-dev.de/tmp/perftest_unity_project.zip

Here is the built windows player (it's a development build, because I wanted to connect the profiler).
http://www.console-dev.de/tmp/perftest_loadscene_x86.zip