[Solved] Storing skill values in ECS

Edit: Seems like NativeHashMap does everything I need in a performant way
Edit2: I will just create a whole bunch of Skill1, Skill2, Skill3, etc structs at runtime

Context:

My game will feature tons of skills, each have a total exp earned and total bonus exp earned, so I can use those values to figure out it's current level (it's done backwards to enable me to chance the needed exp to level seamlessly if needed)

The issue that the total amount of skills is "procedural", it's determined by some XML files and can be easily increased or decreased depending on the mods you install. I will order them and attach a integer to each so I can easily access regardless of it being in OOP or ECS. I know full well that I will need some conversion if a mod adds a new skill​

So here I come to ask:

Is a DynamicBuffer performant if the literally only thing I am doing is getting the whole buffer so I can get a single value out of it? It is never used as a list, it is only ever being used as a container for an unknown number of values.With the DynamicBuffer, every character that have skill have to have all of them, even if 90% of them is at level 0, otherwise I can't random access them (not gonna do the whole ordering them by likeliness of existing and only indexing as much as needed)

What I would really love is some sort of "variant component" thing which is most likely not implemented yet. Something where you could store an arbitrary number of the same IComponentData and index them by a int/short.

Another crazy idea is to create a new entity for each skills that the character have, and add a ISharedComponent to it that is it's ID, so that I can index them and pick one from another, so to access I would go "Gimme the <Component> from the (entity which has IShared<Character> and IShared<SkillIndex>)". After writing it out it became so crazy that it actually might work and be more performant than I otherwise though. (Edit: Thinking about it again, even though it would probably be really fast, it would also take a whole chunk for every single skill for each character, pretty crazy bad memory wise)​

 

This skills mutable at runtime itself? Or after loading game they immutable and only player get/remove skills from himself?

 

after loading they are immutable, but it can change between playthroughs or even between loads/saves. Just not during the game itself

 

I am not sure why you need same variant of components, if that what buffer does for you. Buffer is as fast, as NativeArray. So no reason to not use it. And is flexible in size. Plus is applicable per entity (character).

 

I'd recommend looking into blob assets then, they are great for immutable data and passed around as a pointer so if two entities reference the same blob only one is stored in memory.

 

Also there are hash maps as an alternative.

 

The amount of types skills is imutable during play, but the level of each skill changes, I am also considering the character not having the skill if it's level 0 to save up memory too, as there will be tons (500+) of them. It's like:
Skills: Blacksmith, Mason, Cooking (each recipe have it's own skill)
But the character only knows blacksmith.at lv3 and omelete at lv12. The character can learn the other at any time.
After closing the game and changing the mods/updating the game, the next time the game loads, there might also be a Lumberwork skill, or new recipes may have been added, which also adds a skill for each

The issue is that any skill I don't have will still be using up memory if I just use an array that tells the level of each skill by index, good to know that the buffer performs well. If there was some sort of loose buffer, then it could work.
Every character in the game will have skills, if it was just the main character, I wouldn't care, but I don't want every single NPC storing 500 skill levels when they only know and use 3-4

I was not aware that you could have hashmaps in ECS. If you are talking about ComponentObject, then I will just store the skills outside of the ECS

 

Why do you not just assign skills as components on character entities when they have achieved a nonzero level of that skill? Saves you the memory of tracking all possible skills for all characters. Also, it would allow you to query for all "Blacksmiths", or whatever, returning only characters with the "Blacksmith" skill; and you could keep the character's skill level as the int value field of the component data.

 

That would require making a struct for each skill, whose names and amount I don't know at compile time. I have no idea how many skills there will be because mods can add them.

 

You store you skills set somwhere. I.e. hashmap, or NativeArray whatever suitable.
Then you can just use buffer per character, to store index to skill type, and properties for that skill.
So if your character has 3 skills uses 3 elements. If uses 100, then 100 elements, etc.
Then based on index to type, you can execute relevant behavior.

 

I'm not sure how you plan to implement that: if mods can add meaningful skills to the game, how do those same mods add the systems that will be necessary to implement the verb-forms of those skill-nouns? What's the point of somebody adding an "Archer" skill if there's no ArcherySystem to actuate that skill? How do you plan on implementing the qualitative behavior that mod-introduced skills imply? More to the point: if mods can create/alter behavioral systems, they can certainly declare the component types those systems will operate on; if not, how is this more than just aesthetic?

 

Unless op has internal scripting / parsing functionality, which don't require of creating new systems.
I did something similar as modding feature, which converts JS like script, into methods, which can be added at runtime, and can be executed in burst.

 

I don't know about other skills, but for all the cooking recipe related skills, the skill is just a level value and it is just some multiplier on the cooking system, but it is a skill nonetheless. In the end, mods will probably only be able to add subskills (where cooking a omelete is a subskill of cooking) related to content. Although I do want to load mod assemblies, they are not my worry at the moment, if a mod add a cooked rice recipe, then a cooked rice skill will also be created to govern how good the cooked rice the character cooks is.

You mean to give a unique ID to each skill per character?
Because if (as I intend to do), I give one ID for each skill at runtime, the issue with buffers is that if a character have the 500th and the 2nd skill, then he will store 500 worth of skills for only 2 of them


Somewhat back to my preferred solution that seem to not be implemented:
The same way we can add an arbitrary number of components indexed by their type, I wish I could add an arbitrary number of same-typed components indexed by an int.
Since this feature does not exist, and there don't seem to be any equally good workaround, I will just store them in a hashmap outside of the ECS and use non ECS code to deal with them. plenty of my code is hybrid already

 

You have some dictionary (NativeHashMap for example) which populates when you want (at start, at reload), where key is your unique id per skill (some hash value or just integer ID. You can pass this dictionary everywhere you want (systems, jobs etc.) it’s burstable, it’s some type utility data. Every skills consumer hold DynamicBuffer with hash map keys which correspond to skills for this entity. It’s fast, it’s reduce memory cost, it’s reduce wasting chunk space. Relationship with level accessibility can be in other HashMap where key is skill and value is level what ever you want to use (level of access, level of something etc.)

 

What I mean is, if you got n skills
0-cut bread
1-boil water
2-cook sup
3-fry meat
4-cook carrot
5-make poncho

Now character has two skills
Cut bread (0) and make pinch (5)
So you populate character's buffer with 2 elements.
First element has type set to 0 and second to 5. And whatever other values may be needed.

Now you run system, and iterate through 2 skills of selected character, rather than full set of 5, or 500.

 

I was under the impression that it would perform just as bad as a ComponentObject storing a normal HashMap for some reason. Seems like it solves all my issues

 

Actually, I am searching the documentation, but I can't seem to find how you add a NativeHashMap to an entity. I doubt this is bittable. Turns out I have to add it as an object with AddComponentObject?

 

You can not add NativeHashmap(s) to entities.
Not sure why you need that for?
You store it as variable outside job, allowing any system/job to access it.

 

Couple of thoughts. First off it sounds like you are jumping ahead a bit, worrying about performance for a design that isn't even fully fleshed out with no prototype in place. Get it working first which always surfaces other stuff, then once it stabilizes fine tune for performance.

Generally, most games have a notion of static vs instance data for stuff like this. Static being defined rather loosely. Ie you could add/remove from static data via mods, data only updates to the game, or entirely new builds.

Instance data is then what is variable per player/character, and is generally sparse. Like you might have a hash map of some type with a skill name/id as the key and experience and maybe calculated level in the value.

Bigger picture though for a real game you have other important concerns. Like what can modify experience? In a real game the answer is potentially anything can. Recognizing that implies a significantly different flow for your data and logic, and it will have a significant impact on the specific implementations you create in ECS.

For example our skill system in a complex Rpg isn't aware of experience at all. We have a generialized progression abstraction that takes event messages. So that any feature can progress the player using any/all of our progression stats.

 

If I want to know what are the skills of my entity, I want the thing that stores which are the skills of the entity on the entity.
There is no need to make some system that I send an entity and get the NativeHashmap when AddComponentObject already lets me do that, and it's easily accessed through ForEach. It's not nearly as fast as accessing an IComponentData (afaik, since the ComponentObject is on managed memory), but it seems no solution will be.

Since I am already spending time planning how it will work, I might as well try to make it performant, so that I don't have to toss the whole thing away when I need to optimize it because the "whatever way I did just to get it working fast" is incorrigibly bad. I am already doing everything in a pretty "whatever fast way" by using ForEach instead of burst jobs, but since I am actually spending time attempting to make as much as possible in ForEach, if any of those turns out to be slow and needs actual optimization, it's clear and easy how to do it, just a bit time consuming, since everything is more or less ready for jobification.

 

Worse of all. I could solve all of those if I were to use Reflection, I could have an IComponentData for each skill and everything would be solved, except I would also need Reflection to access those IComponentData, thus making he whole effort moot. (Unless Reflection is not nearly as slow as I think it is)

 

That why you use buffer for.
After received directions, what stopping you from using buffer on entities?
Maybe there is something, which you are not familiar with?
What makes you stopping accessing hasmap, with stored skills sets, from jobs?

I suspect, you still think in terms of OOP, rather DOD, while trying use ECS.

Take advice from @eizenhorn. Is very good at this stuff.
If you have toughs of doubts, here is what he is working on
https://forum.unity.com/threads/unity-ecs-and-job-system-in-production.561229/

 

Because I will have to iterate through the buffer to get the skill I need.
The Unity ECS system already had to do some cache-space-magic stuff to get the buffer for me as fast as it could, now I am either iterating over it or accessing an per-entity-index-mapping when I want only a single value from it.
I want to use the cache-space-magic stuff to get the value I want directly, as I would easily be able to do if I knew at compile time all of my skills.
I doubt anything related to skills will ever be a performance bottleneck, but I want to know how to do this right when I stumble on something similar that actually will be a bottleneck

Way too much to list here


Thinking more deeply how and when I will access those values, I probably am never going to access them inside a job or a system, but on interfaces and other non-ecs code. Yet, for the sake of ease of access and simplicity, i'd want the information of the skill on the entity, not on some outside handmade system.

Turns out, according to (https://stackoverflow.com/questions...c-considered-a-bad-practice/32817143#32817143), MethodInfo.CreateDelegate call is only 2,13 times slower than a conventional method call, so it's totally viable to use GetComponent<T> on a type created at runtime. I think I will just create a whole bunch of Skill1, Skill2, Skill3, etc structs at runtime and use reflection to get them from the entity. And I get the irony of using runtime code generation and reflection for the sake of simplicity

 

Only profiling and testing, equivalent to your project needs will tells you the through.
Do your stress tests.
Compare reflections results vs proposed DOTS approaches, if feeling need for it.
Come back with conclusions. Less speculations.

 

I haven't read much through this thread but if you're trying to create new types of IComponentData at runtime you can't do this. They need to be defined at compile time.

You can't even use generic components at runtime without first registering them in your AssemblyInfo with

Code (CSharp):

1. [assembly: RegisterGenericComponentType(typeof(YOURTYPE<int>))]

 

I disagree. Works flawlessly. I can get and set them via an integer ID
Edit: Unless it works on editor but not on play mode, I didn't check play mode

 

You shouldn’t store HM on entity. You store it in system, utility struct, etc. On entity you store only used by this entity skills (HM IDs in DynamicBuffer)

 

Important to note, this was actually correct, I just had my ECS package outdated, after updating it stopped working until I """Fixed""" it
See: https://forum.unity.com/threads/re-...e-decide-when-typemanager-initializes.763325/