Search Unity

  1. Unity 2018.1 has arrived! Read about it here
    Dismiss Notice
  2. Scriptable Render Pipeline improvements, Texture Mipmap Streaming, and more! Check out what we have in store for you in the 2018.2 Beta.
    Dismiss Notice
  3. If you couldn't join the live stream, take a peek at what you missed.
    Dismiss Notice
  4. Improve your Unity skills with a certified instructor in a private, interactive classroom. Learn more.
    Dismiss Notice
  5. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  6. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

Making a data structure git friendly

Discussion in 'Scripting' started by joshcamas, Jun 13, 2018.

  1. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    212
    Hello friends :)

    I have a list of objects that need to be serialized. Currently, they're all stored in one big dictionary (I'm using Odin Serializer to serialize the dictionary).
    However, this means that if a developer edits this dictionary (adds or removes), and another developer also edits this list, then there would be a git related issue. Bad beans.

    Luckily, my world is split into many pieces, called chunks. Since each of these objects have a position / cell stored, (they aren't game objects, just a basic class) I figured it'd be cool to just create a scriptable object container for each dictionary of objects per cell. This would mean a dev could edit the dictionary in one cell, while another edits the dictionary in the other. Cool beans.

    BUT... then if I need to search for a certain object using a string (which is the key in the dictionary), that would mean going through every single cell scriptable object to find it. Which doesn't sound very nice.

    Should I just do it anyways? Or is there another way to make things git friendly other than having separate files?


    Thanks!
    Josh
     
    Last edited: Jun 13, 2018
  2. Kurt-Dekker

    Kurt-Dekker

    Joined:
    Mar 16, 2013
    Posts:
    2,414
    I've dealt with things like this. I think the best engineering approach is to select a size of datablob that is large enough that you don't have a bazillion of them (which as you point out you must search linearly), and yet small enough to avoid merge issues.

    One way to drastically reduce data merge issues is to make sure your serialization is stable, that is the iterated keys are maintained in some kind of order, such as lexicographic order, when they are serialized. That way inserts to your dict are all physically localized in the serialized output, as are deletions. Not sure if Odin is already "stable" in this regard, but it really helps with merging.

    And keep in mind even if you have a million subcontainers that you have to search, a clever local indexing scheme on top of it that you update once a day can go a long way towards speeding up searches. You could probably elect NOT to commit this index to source control but have each developer keep their own local index that they can rebuild on demand. That way it reduces chuff in your source control, and people who don't need it don't bother.
     
    joshcamas likes this.