Search Unity

  1. Unity 2018.3 is now released.
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. Want more efficiency in your development work? Sign up to receive weekly tech and creative know-how from Unity experts.
    Dismiss Notice
  4. Build games and experiences that can load instantly and without install. Explore the Project Tiny Preview today!
    Dismiss Notice
  5. Nominations have been announced for this years Unity Awards. Celebrate the wonderful projects made by your peers this year and get voting! Vote here!
    Dismiss Notice
  6. Want to provide direct feedback to the Unity team? Join the Unity Advisory Panel.
    Dismiss Notice
  7. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice

Serialization questions

Discussion in 'Entity Component System and C# Job system' started by pointcache, Jun 1, 2018.

  1. pointcache

    pointcache

    Joined:
    Sep 22, 2012
    Posts:
    461
    1. Can i have access to all entities and components before and after serialization?
    I dont use unity editor or interface to store game data, everything is loaded from disk.
    When serializing currently many reference objects (like current graphic, current reference to "database" model describing an entity) are converted to simple strings before serialization, and on deserialization are retrieved back from database as references.

    2. How deserialization of entities is handled? Let's say i have a custom EntityFactory, describing how to construct each type of entity in the game (say Furniture, Weapon) using my own definition file. Now i expect when i deserialize the state that my Factory will create a new entity using the stored definition id, and overwrite components with serialized.

    3. Is it possible to mark fields to be ignored by SerializeUtility.SerializeWorld ?

    4. Also where can i get a tutorial on how to use SerializeUtility.SerializeWorld ?

    I really would like to have control over how i serialize things. I want to be able to serialize it into a generic format, say json, and deserialize into a temporary object which i can then manually initialize (recreate entities from those serialized using new / changed / versioned versions of factories, change component values (like apply hotfix)) , or simply change data and feed into the EntityManager .

    In theory i could implement a save as ecs system, getting all entities that have some base RuntimeData
    like
    ` public struct FurnitureData : RuntimeData, IComponentData `
    and then get it through GetComponent<> , if ecs supports that.
    And then creating a wrapper for each entity with ID + data, and load system would simply destroy all entities and recreate them from factory with that data.
    Now i have one more question

    5. If i want to do the above, and i serialize Entity.Index, can i feed it back to ecs so that it creates entity with this exact index? Since we need to have persistent entity ids, so that we can have collections, inventories, etc.

    6. Are entity indexes sequential? Am i guaranteed to not have entity 3, 5 and no 4?
    If so i could just sort the serialized data before hand by id, and i would assume that the state would be restored.
    However im assuming some entities wont be serialized, thus it would create gaps, and break the relation.

    Thank you for moving unity towards the golden age <3
     
    Last edited: Jun 2, 2018
  2. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    159
    I'm a bit disappointed about the current state of serialization
    1) you can't choose what to serialize and what not to serialize, only entire world can be serialized
    2) you can't merge saved data with the existing scene - "loading into non-empty entity manager is not supported"
    3) you can serialize data only to file, you can't serialize to memory (without writing you own BinaryWriter)
    4) there is no inspector - you can't tweak serialized data in editor



    Code (CSharp):
    1.  
    2. private string file = Application.persistentDataPath + "/test-save.dat";
    3. public void SaveWorld(){
    4.       var sharedComponents = new int[]{};
    5.       BinaryWriter writer = new StreamBinaryWriter(file);
    6.       SerializeUtility.SerializeWorld( EntityManager, writer, out sharedComponents );
    7.       writer.Dispose();
    8. }
    9. public void ClearWorld(){
    10.       var entities = EntityManager.GetAllEntities();
    11.       EntityManager.DestroyEntity(entities);
    12.       entities.Dispose();
    13. }
    14. public void LoadWorld(){
    15.       BinaryReader reader = new StreamBinaryReader(file);
    16.       SerializeUtility.DeserializeWorld( EntityManager, reader );
    17.       reader.Dispose();
    18. }
     
    Last edited: Jun 3, 2018
    DwinTeimlon and Afonso-Lage like this.
  3. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,201
    Its very much work in progress. Right now its a binary only format. It's very fast and provides no backwards compatibility if component data changes. It is only intended as a runtime loading format. Scene editing is still recommended to be done via game objects and IComponentData attached to them.

    Generally you load into an empty world. (The idea being that you can do that async in a job via ExclusiveEntityTransaction)

    Once all entities have been loaded into the other world (Lets call it construction world), you use EntityManager.MoveEntities to move the entities to the new world. This way you can modify entities / components before activating them in the main world. Also you can either load from disk or procedurally generate entities. Essentially unifying streaming procedurally or from data.

    We will also provide a EntityManager.CopyEntities function so you can add / remove any components / entities you don't want before writing it to disk.
     
  4. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    159
    Thank you for your answer, going to try multi-world setup
     
  5. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    159
    @Joachim_Ante

    EntityManager.MoveEntities moves all entities from one world to another, how to move one or some subset?

    Also, it'll be nice to be able to copy a single entity from one world to another.
     
  6. Hans-D

    Hans-D

    Joined:
    Sep 24, 2013
    Posts:
    24
    If you serialize and then load those entities into a world, are their entity IDs consistent? This would be very helpful to keep server/client worlds synchronized while having components reference other entities through entity ID.