Search Unity

  1. Curious about what's going to be in 2020.1? Have a look at the 2020.1 beta blog post.
    Dismiss Notice
  2. Want to see 2020.1b in action? Sign up for our Beta 2020.1 Overview Webinar on April 20th for a live presentation from our evangelists and a Q&A session with guests from R&D.
    Dismiss Notice
  3. Interested in giving us feedback? Join our online research interviews on a broad range of topics and share your insights with us.
    Dismiss Notice
  4. New Unity Live Help updates. Check them out here!

    Dismiss Notice

[Solved] Storing skill values in ECS

Discussion in 'Data Oriented Technology Stack' started by Guedez, Aug 26, 2019.

  1. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    Edit: Seems like NativeHashMap does everything I need in a performant way
    Edit2: I will just create a whole bunch of Skill1, Skill2, Skill3, etc structs at runtime

    Context:
    My game will feature tons of skills, each have a total exp earned and total bonus exp earned, so I can use those values to figure out it's current level (it's done backwards to enable me to chance the needed exp to level seamlessly if needed)

    The issue that the total amount of skills is "procedural", it's determined by some XML files and can be easily increased or decreased depending on the mods you install. I will order them and attach a integer to each so I can easily access regardless of it being in OOP or ECS. I know full well that I will need some conversion if a mod adds a new skill​
    So here I come to ask:
    Is a DynamicBuffer performant if the literally only thing I am doing is getting the whole buffer so I can get a single value out of it? It is never used as a list, it is only ever being used as a container for an unknown number of values.With the DynamicBuffer, every character that have skill have to have all of them, even if 90% of them is at level 0, otherwise I can't random access them (not gonna do the whole ordering them by likeliness of existing and only indexing as much as needed)

    What I would really love is some sort of "variant component" thing which is most likely not implemented yet. Something where you could store an arbitrary number of the same IComponentData and index them by a int/short.

    Another crazy idea is to create a new entity for each skills that the character have, and add a ISharedComponent to it that is it's ID, so that I can index them and pick one from another, so to access I would go "Gimme the <Component> from the (entity which has IShared<Character> and IShared<SkillIndex>)". After writing it out it became so crazy that it actually might work and be more performant than I otherwise though. (Edit: Thinking about it again, even though it would probably be really fast, it would also take a whole chunk for every single skill for each character, pretty crazy bad memory wise)​
     
    Last edited: Aug 27, 2019
  2. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,813
    This skills mutable at runtime itself? Or after loading game they immutable and only player get/remove skills from himself?
     
  3. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    after loading they are immutable, but it can change between playthroughs or even between loads/saves. Just not during the game itself
     
  4. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    I am not sure why you need same variant of components, if that what buffer does for you. Buffer is as fast, as NativeArray. So no reason to not use it. And is flexible in size. Plus is applicable per entity (character).
     
  5. siggigg

    siggigg

    Joined:
    Apr 11, 2018
    Posts:
    247
    I'd recommend looking into blob assets then, they are great for immutable data and passed around as a pointer so if two entities reference the same blob only one is stored in memory.
     
  6. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    Also there are hash maps as an alternative.
     
  7. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    The amount of types skills is imutable during play, but the level of each skill changes, I am also considering the character not having the skill if it's level 0 to save up memory too, as there will be tons (500+) of them. It's like:
    Skills: Blacksmith, Mason, Cooking (each recipe have it's own skill)
    But the character only knows blacksmith.at lv3 and omelete at lv12. The character can learn the other at any time.
    After closing the game and changing the mods/updating the game, the next time the game loads, there might also be a Lumberwork skill, or new recipes may have been added, which also adds a skill for each

    The issue is that any skill I don't have will still be using up memory if I just use an array that tells the level of each skill by index, good to know that the buffer performs well. If there was some sort of loose buffer, then it could work.
    Every character in the game will have skills, if it was just the main character, I wouldn't care, but I don't want every single NPC storing 500 skill levels when they only know and use 3-4

    I was not aware that you could have hashmaps in ECS. If you are talking about ComponentObject, then I will just store the skills outside of the ECS
     
  8. digitaliliad

    digitaliliad

    Joined:
    Jul 1, 2018
    Posts:
    63
    Why do you not just assign skills as components on character entities when they have achieved a nonzero level of that skill? Saves you the memory of tracking all possible skills for all characters. Also, it would allow you to query for all "Blacksmiths", or whatever, returning only characters with the "Blacksmith" skill; and you could keep the character's skill level as the int value field of the component data.
     
  9. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    That would require making a struct for each skill, whose names and amount I don't know at compile time. I have no idea how many skills there will be because mods can add them.
     
  10. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    You store you skills set somwhere. I.e. hashmap, or NativeArray whatever suitable.
    Then you can just use buffer per character, to store index to skill type, and properties for that skill.
    So if your character has 3 skills uses 3 elements. If uses 100, then 100 elements, etc.
    Then based on index to type, you can execute relevant behavior.
     
  11. digitaliliad

    digitaliliad

    Joined:
    Jul 1, 2018
    Posts:
    63
    I'm not sure how you plan to implement that: if mods can add meaningful skills to the game, how do those same mods add the systems that will be necessary to implement the verb-forms of those skill-nouns? What's the point of somebody adding an "Archer" skill if there's no ArcherySystem to actuate that skill? How do you plan on implementing the qualitative behavior that mod-introduced skills imply? More to the point: if mods can create/alter behavioral systems, they can certainly declare the component types those systems will operate on; if not, how is this more than just aesthetic?
     
    Antypodish likes this.
  12. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    Unless op has internal scripting / parsing functionality, which don't require of creating new systems.
    I did something similar as modding feature, which converts JS like script, into methods, which can be added at runtime, and can be executed in burst.
     
  13. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    I don't know about other skills, but for all the cooking recipe related skills, the skill is just a level value and it is just some multiplier on the cooking system, but it is a skill nonetheless. In the end, mods will probably only be able to add subskills (where cooking a omelete is a subskill of cooking) related to content. Although I do want to load mod assemblies, they are not my worry at the moment, if a mod add a cooked rice recipe, then a cooked rice skill will also be created to govern how good the cooked rice the character cooks is.

    You mean to give a unique ID to each skill per character?
    Because if (as I intend to do), I give one ID for each skill at runtime, the issue with buffers is that if a character have the 500th and the 2nd skill, then he will store 500 worth of skills for only 2 of them


    Somewhat back to my preferred solution that seem to not be implemented:
    The same way we can add an arbitrary number of components indexed by their type, I wish I could add an arbitrary number of same-typed components indexed by an int.
    Since this feature does not exist, and there don't seem to be any equally good workaround, I will just store them in a hashmap outside of the ECS and use non ECS code to deal with them. plenty of my code is hybrid already
     
  14. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,813
    You have some dictionary (NativeHashMap for example) which populates when you want (at start, at reload), where key is your unique id per skill (some hash value or just integer ID. You can pass this dictionary everywhere you want (systems, jobs etc.) it’s burstable, it’s some type utility data. Every skills consumer hold DynamicBuffer with hash map keys which correspond to skills for this entity. It’s fast, it’s reduce memory cost, it’s reduce wasting chunk space. Relationship with level accessibility can be in other HashMap where key is skill and value is level what ever you want to use (level of access, level of something etc.)
     
    andywatts, Guedez and Antypodish like this.
  15. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    What I mean is, if you got n skills
    0-cut bread
    1-boil water
    2-cook sup
    3-fry meat
    4-cook carrot
    5-make poncho

    Now character has two skills
    Cut bread (0) and make pinch (5)
    So you populate character's buffer with 2 elements.
    First element has type set to 0 and second to 5. And whatever other values may be needed.

    Now you run system, and iterate through 2 skills of selected character, rather than full set of 5, or 500.
     
  16. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    I was under the impression that it would perform just as bad as a ComponentObject storing a normal HashMap for some reason. Seems like it solves all my issues
     
  17. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    Actually, I am searching the documentation, but I can't seem to find how you add a NativeHashMap to an entity. I doubt this is bittable. Turns out I have to add it as an object with AddComponentObject?
     
  18. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    You can not add NativeHashmap(s) to entities.
    Not sure why you need that for?
    You store it as variable outside job, allowing any system/job to access it.
     
  19. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,628
    Couple of thoughts. First off it sounds like you are jumping ahead a bit, worrying about performance for a design that isn't even fully fleshed out with no prototype in place. Get it working first which always surfaces other stuff, then once it stabilizes fine tune for performance.

    Generally, most games have a notion of static vs instance data for stuff like this. Static being defined rather loosely. Ie you could add/remove from static data via mods, data only updates to the game, or entirely new builds.

    Instance data is then what is variable per player/character, and is generally sparse. Like you might have a hash map of some type with a skill name/id as the key and experience and maybe calculated level in the value.

    Bigger picture though for a real game you have other important concerns. Like what can modify experience? In a real game the answer is potentially anything can. Recognizing that implies a significantly different flow for your data and logic, and it will have a significant impact on the specific implementations you create in ECS.

    For example our skill system in a complex Rpg isn't aware of experience at all. We have a generialized progression abstraction that takes event messages. So that any feature can progress the player using any/all of our progression stats.
     
  20. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    If I want to know what are the skills of my entity, I want the thing that stores which are the skills of the entity on the entity.
    There is no need to make some system that I send an entity and get the NativeHashmap when AddComponentObject already lets me do that, and it's easily accessed through ForEach. It's not nearly as fast as accessing an IComponentData (afaik, since the ComponentObject is on managed memory), but it seems no solution will be.

    Since I am already spending time planning how it will work, I might as well try to make it performant, so that I don't have to toss the whole thing away when I need to optimize it because the "whatever way I did just to get it working fast" is incorrigibly bad. I am already doing everything in a pretty "whatever fast way" by using ForEach instead of burst jobs, but since I am actually spending time attempting to make as much as possible in ForEach, if any of those turns out to be slow and needs actual optimization, it's clear and easy how to do it, just a bit time consuming, since everything is more or less ready for jobification.
     
  21. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    Worse of all. I could solve all of those if I were to use Reflection, I could have an IComponentData for each skill and everything would be solved, except I would also need Reflection to access those IComponentData, thus making he whole effort moot. (Unless Reflection is not nearly as slow as I think it is)
     
  22. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    That why you use buffer for.
    After received directions, what stopping you from using buffer on entities?
    Maybe there is something, which you are not familiar with?
    What makes you stopping accessing hasmap, with stored skills sets, from jobs?

    I suspect, you still think in terms of OOP, rather DOD, while trying use ECS.

    Take advice from @eizenhorn. Is very good at this stuff.
    If you have toughs of doubts, here is what he is working on
    https://forum.unity.com/threads/unity-ecs-and-job-system-in-production.561229/
     
    Last edited: Aug 27, 2019
  23. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    Because I will have to iterate through the buffer to get the skill I need.
    The Unity ECS system already had to do some cache-space-magic stuff to get the buffer for me as fast as it could, now I am either iterating over it or accessing an per-entity-index-mapping when I want only a single value from it.
    I want to use the cache-space-magic stuff to get the value I want directly, as I would easily be able to do if I knew at compile time all of my skills.
    I doubt anything related to skills will ever be a performance bottleneck, but I want to know how to do this right when I stumble on something similar that actually will be a bottleneck

    Way too much to list here


    Thinking more deeply how and when I will access those values, I probably am never going to access them inside a job or a system, but on interfaces and other non-ecs code. Yet, for the sake of ease of access and simplicity, i'd want the information of the skill on the entity, not on some outside handmade system.

    Turns out, according to (https://stackoverflow.com/questions...c-considered-a-bad-practice/32817143#32817143), MethodInfo.CreateDelegate call is only 2,13 times slower than a conventional method call, so it's totally viable to use GetComponent<T> on a type created at runtime. I think I will just create a whole bunch of Skill1, Skill2, Skill3, etc structs at runtime and use reflection to get them from the entity. And I get the irony of using runtime code generation and reflection for the sake of simplicity
     
  24. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,249
    Only profiling and testing, equivalent to your project needs will tells you the through.
    Do your stress tests.
    Compare reflections results vs proposed DOTS approaches, if feeling need for it.
    Come back with conclusions. Less speculations.
     
  25. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    2,060
    I haven't read much through this thread but if you're trying to create new types of IComponentData at runtime you can't do this. They need to be defined at compile time.

    You can't even use generic components at runtime without first registering them in your AssemblyInfo with

    Code (CSharp):
    1. [assembly: RegisterGenericComponentType(typeof(YOURTYPE<int>))]
     
  26. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    I disagree. Works flawlessly. I can get and set them via an integer ID upload_2019-8-27_21-28-5.png
    Edit: Unless it works on editor but not on play mode, I didn't check play mode
     
  27. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,813
    You shouldn’t store HM on entity. You store it in system, utility struct, etc. On entity you store only used by this entity skills (HM IDs in DynamicBuffer)
     
  28. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    340
    Important to note, this was actually correct, I just had my ECS package outdated, after updating it stopped working until I """Fixed""" it
    See: https://forum.unity.com/threads/re-...e-decide-when-typemanager-initializes.763325/
     
unityunity