Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

Serialization questions

Discussion in 'Data Oriented Technology Stack' started by pointcache, Jun 1, 2018.

  1. pointcache


    Sep 22, 2012
    1. Can i have access to all entities and components before and after serialization?
    I dont use unity editor or interface to store game data, everything is loaded from disk.
    When serializing currently many reference objects (like current graphic, current reference to "database" model describing an entity) are converted to simple strings before serialization, and on deserialization are retrieved back from database as references.

    2. How deserialization of entities is handled? Let's say i have a custom EntityFactory, describing how to construct each type of entity in the game (say Furniture, Weapon) using my own definition file. Now i expect when i deserialize the state that my Factory will create a new entity using the stored definition id, and overwrite components with serialized.

    3. Is it possible to mark fields to be ignored by SerializeUtility.SerializeWorld ?

    4. Also where can i get a tutorial on how to use SerializeUtility.SerializeWorld ?

    I really would like to have control over how i serialize things. I want to be able to serialize it into a generic format, say json, and deserialize into a temporary object which i can then manually initialize (recreate entities from those serialized using new / changed / versioned versions of factories, change component values (like apply hotfix)) , or simply change data and feed into the EntityManager .

    In theory i could implement a save as ecs system, getting all entities that have some base RuntimeData
    ` public struct FurnitureData : RuntimeData, IComponentData `
    and then get it through GetComponent<> , if ecs supports that.
    And then creating a wrapper for each entity with ID + data, and load system would simply destroy all entities and recreate them from factory with that data.
    Now i have one more question

    5. If i want to do the above, and i serialize Entity.Index, can i feed it back to ecs so that it creates entity with this exact index? Since we need to have persistent entity ids, so that we can have collections, inventories, etc.

    6. Are entity indexes sequential? Am i guaranteed to not have entity 3, 5 and no 4?
    If so i could just sort the serialized data before hand by id, and i would assume that the state would be restored.
    However im assuming some entities wont be serialized, thus it would create gaps, and break the relation.

    Thank you for moving unity towards the golden age <3
    Last edited: Jun 2, 2018
  2. SubPixelPerfect


    Oct 14, 2015
    I'm a bit disappointed about the current state of serialization
    1) you can't choose what to serialize and what not to serialize, only entire world can be serialized
    2) you can't merge saved data with the existing scene - "loading into non-empty entity manager is not supported"
    3) you can serialize data only to file, you can't serialize to memory (without writing you own BinaryWriter)
    4) there is no inspector - you can't tweak serialized data in editor

    Code (CSharp):
    2. private string file = Application.persistentDataPath + "/test-save.dat";
    3. public void SaveWorld(){
    4.       var sharedComponents = new int[]{};
    5.       BinaryWriter writer = new StreamBinaryWriter(file);
    6.       SerializeUtility.SerializeWorld( EntityManager, writer, out sharedComponents );
    7.       writer.Dispose();
    8. }
    9. public void ClearWorld(){
    10.       var entities = EntityManager.GetAllEntities();
    11.       EntityManager.DestroyEntity(entities);
    12.       entities.Dispose();
    13. }
    14. public void LoadWorld(){
    15.       BinaryReader reader = new StreamBinaryReader(file);
    16.       SerializeUtility.DeserializeWorld( EntityManager, reader );
    17.       reader.Dispose();
    18. }
    Last edited: Jun 3, 2018
    DwinTeimlon and Afonso-Lage like this.
  3. Joachim_Ante


    Unity Technologies

    Mar 16, 2005
    Its very much work in progress. Right now its a binary only format. It's very fast and provides no backwards compatibility if component data changes. It is only intended as a runtime loading format. Scene editing is still recommended to be done via game objects and IComponentData attached to them.

    Generally you load into an empty world. (The idea being that you can do that async in a job via ExclusiveEntityTransaction)

    Once all entities have been loaded into the other world (Lets call it construction world), you use EntityManager.MoveEntities to move the entities to the new world. This way you can modify entities / components before activating them in the main world. Also you can either load from disk or procedurally generate entities. Essentially unifying streaming procedurally or from data.

    We will also provide a EntityManager.CopyEntities function so you can add / remove any components / entities you don't want before writing it to disk.
  4. SubPixelPerfect


    Oct 14, 2015
    Thank you for your answer, going to try multi-world setup
  5. SubPixelPerfect


    Oct 14, 2015

    EntityManager.MoveEntities moves all entities from one world to another, how to move one or some subset?

    Also, it'll be nice to be able to copy a single entity from one world to another.
  6. Hans-D


    Sep 24, 2013
    If you serialize and then load those entities into a world, are their entity IDs consistent? This would be very helpful to keep server/client worlds synchronized while having components reference other entities through entity ID.