Truly async scene loading.

Hello everyone,

I asked about this in Unity 4 to 5 era and the answer was a big hard NO while having YES just winking from afar. A lot being changed in the decade, I thought why not, let's discuss again if it's possible. Here's the question;

Is it possible to load scenes without any kind of freeze (or block) whatsoever even if it would take triple the time than normal? A great example is Destiny 2 (Bungie's own engine). If you check the loading screens of that game, you'll see what I mean. You can check your inventory, do stuff with it, or just sit and watch the 3D animation play while you wait for a world to load without a single frame drop.

Right now, we have SceneManager.LoadSceneAsync but it's definitely not that async as it still freezes very often (almost all the time) for long times for very large scenes. Let's say for some bizarre reason a project needs to load gigabytes of data at start. Unity would just freeze to death in this case.

So is this possible? I'm guessing it might have something to do with threading (and unity not being thread safe) but now with burst+jobs+dots and tons of other changes in our hands, maybe there's a way?

I know this may be a lost cause but still, thanks for any answers.

 

The built in behavior is still basically as you describe but you could use other custom approaches such as loading an empty scene that uses a world streaming tool which will load in the content slowly and properly.

It affects your workflow in a bigger way, and I agree it would be much better if the built in API just had more customizability for lazy loading and such.

 

Thanks for the answer. I agree to that. If something is claimed to be 'async' and practically isn't, that's a bit odd to me.

Still tho, I've tried something different like you've suggested too but I can never ever get rid of the tiny hiccups during the initiation of the loading of the files. The only possibility I could find was loading everything including textures and the models during runtime from a folder structure with your own loading system or with something like Asset Bundles. And even that is still not perfect and does not prevent hiccups as you're forced to do many things on the main thread.

While that is a way that you can get around with what I describe, it's a whole different approach and brings in many tedious work and almost completely removes many useful features like referencing things from the inspector as they're not loaded with the scene. Also you have to deal with garbage collection at times too.

In short, it's possible that way but it takes a one hour, junior scene design job to multiple days with very deep skill level requirements and that just sucks don't you think?

 

Hiccups aren't just about whether or not it's truly asynchronous. At some point a bunch of things have to be added to the active scene, and Unity can't make assumptions about how that can be broken up, so there's a bunch of work it just has to do all inside a single frame. If you load a scene with 1,000 colliders and 500 scripts in it, it can pre-load all of the data for those things, but at some point in a single tick Unity has to make all of the colliders active and call Awake() and then Start() on each of those things. If it does them a few at a time things could break.

I would consider writing Editor tools so that this stuff didn't have to be done by hand. I've not tried this, but one thing that springs to mind is using the OnBefore[De]Serialize() callbacks to move stuff out of the base scene and into a custom data structure at save time, and back at load time. This way you can still use the Editor with all its niceness, but add extra data at save time which allows your scenes to be broken down into smaller chunks for effective stream loading.

Exactly how to break them down into smaller chunks would be entirely dependent on the specifics of your game.

(I haven't tried that approach. Just splitting stuff into smaller scenes and loading them hierarchically has worked for me so far.)

 

I understand how it can't make assumptions but it could very easily allow us a lazy loading order list for any given scene so that it manages everything internally without having us to inject code at every part of it. Script Execution Order is a very different thing but has the same base idea for example. At the very least, one tick with Lazy Load could allow us to load the scene on a hierarchy respect and wait for every single gameobject there to be loaded before loading the next. If you're aware of what you're doing, that shouldn't be a problem for the project as in many things with Unity are already extremely experimental.

That's what I'm considering too but yeah a bit of too much work for me for now.

JSON could be a solving mitigator here tho. Maybe even replacing the default Start and Awake functions with a new one that allows you to respect a certain loading order. This tool would also have to inject itself into building pipeline as well so that it builds things accordingly. But then again that would probably be un-testable since it would require actually building the project and all.

Anyways. Seems like it's still impossible from within Unity Thanks for the answer.

 

maybe using a ECS kinda-approach, loading all the entities into the scene (this can be stretched across frames, a few entities per frame), but the systems being disabled (yet).

then initializing the systems (like Awake()) and spread this across multiple frames, with regards to the load order, same again vor a Start() alternative.

when this is done, the scene load can be considered done.

 

Yeah, I'm building my stuff with barebone scenes in mind (loading additively and async) and loading the object in from Addressables over time. Although I don't have very populated scenes yet, so I have nothing tangible to report, just sharing the idea I came up with to avoid hiccups around scene loading.

 

Yeah I think I saw a post on the same thing you describe (which was posted by you actually I guess? ). Adressables seems like one of our best bets yet still, to my coder mind, having to replace a deeply embedded in-engine default loading system entirely sounds a bit unprofessional.

Nonetheless, would love to hear your reports if you have any success!