About 2D open world streaming... (?)

Now I do want to make an open world 2D top-down space game taking place entirely inside a huge (but not insanly huge) handcrafted open world, probably containing somewhere between one and ten thousand gameObjects. And for me there is no way to go around it being open world anymore cause the space-world being open and dooming the player to temporarily get lost at some point is a key motif of the game.

Firstly I thought I might leave the world streaming to some third party tools however there are some problems with that:
Sectr Stream seems to only work with 3D worlds and the World Streamer works with rectangular chunks but I'm working with some giant 2D wall geometry that would for sure expand far beyond just one of these chunks in most cases. So either I'll find a valuable way to slice my walls into pieces automatically or the World Streamer is out too.
And also there's always the problem of NPCs since the player might get chased by other spaceships from time to time and both 'ships getting unloaded near the player because their original chunk/sector has been unloaded' and 'spaceships flying through not loaded areas off-screen and eventually getting stuck or reaching places they're not supposed to' are scenarios I need ta avoid. But the solution to this doesn't seem obvious with the World Streamer, too.

Now if I were about to code a 2D streaming solution myself the first thing that came to mind was manually grouping all my gameObjects into (let's just call them) sectors and activate/deactivate adjacent sectors via Unity's GameObject.setActive() method.
But how will that affect memory usage and performance? And is coding such a system actually worth it in the end?

If not, my last chance would be to write a tool that makes each sector its own scene and streames these scenes at runtime but also allowing NPCs to traverse between scenes and I really doubt I could ever do that...

Anyways I'm curious about what you think and what experiences you've had with this topic and I'd really appreciate any feedback you have

 

You store your data as an array. GameObject is just like decorator, optional. You create object if you need only to display, or make interaction with block. Array should stores data of blocks or chunks, whatever you need. This way you have also data ready to transmit, without additional conversions involved. Providing arrays are small enough.

 

If in doubt - use additive scene loading in background. Split your game by "sectors" and stream them as soon as player gets nearby one. Same with unloading. That way you could disable and enable entire sections of the global "map" without enabling / disabling lots of objects. Plus, background thread streaming. Firewatch did it the same way.

Works pretty well for complex 3D scenes, would work even better for 2D environment (less data to load), doesn't require specific bounds splitting, but a bit of tooling and proper level design. Well worth it if done correctly.

 

I know, I know, Firewatch used (probably a modified version of) the Sectr Stream system from the Asset Store, however the way portals work pretty much intends it should be used for 3D games. Not a problem for Firewatch but definitely a reason for me to not buy it until I know for sure I can also use it for 2D game development

And for @Antypodish you mean I should create a system that stores all the data from my gameObjects in a seperate set of arrays and then Instantiate() empty gameObjects and add all the components and variables from these arrays at runtime? Nice idea actually, but why not avoid most of the trouble and make some arrays holding gameObjects in the first place? That'll save a lot of calculations about who's what and where which values should be set to 0.26 or false.
But thanks for reminding me about the array thing, I've actually heared about array-based open world streaming in the past the last time I researched this topic but totally forgot about it by now.
But I'm still wondering whether or not permanently instantiating and destroying gameObjects (of course, by destroying the sectors these gameObjects are children of) causes a lot of hiccups

 

You are likely want an array with a class type of block/chunk, to store any type of data, Yes you can store Game objects too, but I would rather keep them either in class, which is type of array, or separate array. The reason is, you can not easily serialize GameObjects, if you want to save or send over the network etc. Anyway that would be unnecessary burden. It can be list as well mind.

 

You don't need to split content by sectors, you can do it on scene basis. Everything that can be put inside scene can be streamed in and out. Without portals.

 

Correct. Now there would only be the problem left that NPSs need to be able to move between scenes.
Anyway just found out I don't even need portals at all with Sectr Stream, there is also the option to load sectors using Unity's trigger colliders - I just hope it also works with 2D triggers

 

You can move objects between scene by using https://docs.unity3d.com/ScriptReference/SceneManagement.SceneManager.MoveGameObjectToScene.html