[Question]How to improve scalability of creating a massive amount of objects (in the editor)?

I'm currently working on a project where I have to load a massive amount of (edit: non-static) objects. These objects are loaded via editor scripts, so this is not in-game. The amount of objects I have to create are in the range of 50.000 - 100.000. At run time I hide about 99% of all objects through a simple LOD system, so it actually runs pretty fast.

The problem I'm having however is that Unity takes a massive amount of time to load all the objects in the first place. It seems that Unity takes a bit longer every time I add an object to the scene. Near the end of loading all these objects, Unity took the same amount of time to add a single object as it took to add the first 1000 objects. The scalability is less than favorable, in other words.

Here's an image to demonstrate what I mean. These are 4 time measurements, each representing the time it took to add 1000 objects to the scene. Note how the time it takes increases quite dramatically as more are added. After 3000, it already takes over 4 times as long to add the objects as the first 1000.



At lower amounts this is neglectable, but with the many objects I create, this becomes a serious problem. If I take just the time it takes to add the first 1000 objects and extrapolate, the loading should take no longer than 10 minutes. Instead, it takes 2 hours.

My suspicion is that unity does some unnecessary calculations for the sake of run-time optimization, such as adding the object to an octree or something similar.

Is there a way to temporarily disable these kind of optimizations, if that is in fact what's happening here? And if not, is there anyway I can optimize the scalability?

 

I also saw this non-linear slowdown while making a ton of objects at once.

However, this was in-game, not in-editor.

The solution I used was to make the loading function a coroutine and yield for zero seconds every hundred or so game objects. It does seem like perhaps there is a linear search going on for each initial object, and the more just-added objects there are, the worse it is. Once the objects are processed for one frame, they seem to move out into the main scene graph and performance is restored.

Perhaps you can do it in an editor script by having a separate control boolean that marks the entire editor pane as invalid until all objects have gone through?

 

I will try this, thank you.

edit: Not really seeing any changes. I think it's similar to what I was already doing. After every 1000 objects I already let the editor do an iteration. Otherwise Windows starts complaining that Unity stopped responding.