Question Design issue with subscenes and persistence (serialization)

Hi!

I'm currently prototyping a game with DOTS for an upcoming project and I have trouble to deal with subscenes and entities' persistence.

My game is sort of a very classic "open world", with a grid of levels (subscenes here) streaming when the player is near. Nothing fancy.

In this world, I have a couple of thousands objects (placed by hand) I need to update every five minutes through jobs. I do not need this update to run in a single frame: I just need this update to be done within this five minutes interval between two updates (so scheduling can be progressive).

I need to update loaded and unloaded objects. These objects are entities with a few components and everything to render a mesh. The update may change the mesh. However, the position is guaranteed (at this moment at least) to remain the same and it is also guaranteed that two objects can't share the same position.

Eventually, I need to save and load the state of these entities, but that's not the real issue here.

I figured out two main solutions for this problem. However, none of them seems fully satisfying and I think I misunderstood something in the whole DOTS ecosystem.

Solution A (but feels like a hack):

I have a manager with a dictionary (or some sort of container). Keys are Translation type, values contains data. My game loop only updates the dictionary so nothing happens in the world. Then, through a system, I update loaded entities with a query, getting the appropriate value from the dictionary for each object.

Saving and loading is trivial as it is just a standard container to save.

Pros:
- Very efficient to iterate through all the data
- Easy to work with
- Trivial to save and load

Cons:
- Not flexible at all (and even though it is not a problem at this moment, it could be at some point)
- Feels like a hack and definitely not the way I should do it
- Defeats the principles of ECS with components owning the data, since the data is centralized
- A dictionary is not at all the most efficient container for huge amounts of data

Solution B (but it is very, very inconvenient):

This time, I have to set up my subscenes in two parts: one part holds the entities I will update, and another part holds the rest of my objects.

At first, I load the subscenes with the entities I need to update. I update them, I serialize them (subscene by subscene), then I unload them.
When I need to update them again, I deserialize them, I update them then I reserialize (still subscene by subscene).

Pros:
- Uses ECS and components are components (with actual data, yeah!)
- ... That's pretty much all?

Cons:
- Ridiculously inconvenient
- I need to serialize over and over again
- Probably very, very bad performance
- Difficult to set up as I have to create twice as much subscenes
- I need to load meshes associated with the entities even if I do not need to render them
- Probably more but that's way enough to disqualify this solution

Solution A version 2.0 (but still feels like a hack):

Same solution but with an editor script to automatically assign unique IDs to entities with a component. It allows the manager to use this ID as a key, and therefore a simple list.

This is more flexible and probably more performant at runtime, but I'm not so fond of the idea of a script editor to set up IDs. I feel like it is too complicated for what I'm trying to do, but perhaps it's the way to go...

Thank you a lot for reading this (longish) post! Would you have some recommandations, guidelines to help me solve this problem? Experiences to share with similar design issues in ECS?

Thanks a lot!

 

Some people have discussed simply copying the world into a saving world and using the utilities that exist in the Entities package to serialize the world and write it to disk. This would be extremely fast, but I personally don't like this approach for a number of reasons with the biggest one being that updates to your game and updates from the Entities package could break your saves.

IMHO save data must be tolerant to changes to game code. I try to design save data to be "upgradeable" when components change / are replaced.

The save scheme I currently have is to have a singleton monobehaviour that loads / writes save data to and from disk. A series of systems iterate over all entities with a persistent Id and save data that they are specifically written to handle. Data is serialized using JSON. When loading a saved game each load system finds all entities with a persistent Id component and updates the loaded data to reflect what was loaded from disk. Another system adds a Loaded Tag to all entities with a persistent Id so that they appear "unloaded" for exactly 1 frame after they are instantiated. My save scheme does not currently work with dynamically spawned entities nor entities that were destroyed (these are not cleaned up when loading a game), but I am working to rectify these issues and other disadvantages (such as having to write a serialize / deserialize system for every component you want to save) through code generation and custom attributes.