Search Unity

Help Needed: Assigning a single component from multiple threads

Discussion in 'Data Oriented Technology Stack' started by PublicEnumE, Nov 24, 2019.

  1. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    Not sure of the best way to solve this problem:

    I have multiple threads (from a single job) which all might need to set the same component on the same, target entity.

    In this case, the data of the component being set doesn’t actually matter. It can be an empty component. I just want to assign it, to increment its “changed” version, so that that component will be picked up by an EntityQuery (with a change filter) in the next system.

    (Imagine 10 assassins seeing the same target, and the target just needs to know that it was “seen”, to be worked on by the next system.)

    But I’m pretty sure that having several threads assign the same component value in parallel, even if that component is empty, is dangerous if not unsupported. And even if it is, it sure smells bad.

    Would you have any advice on how I should be approaching the problem? Thanks for any help.
     
    Last edited: Nov 24, 2019
    florianhanke likes this.
  2. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    I could really use some advice about this if you smarter people have any thoughts.

    I have what I suspect are ‘bad’ ideas of how to do this (like storing everything in a NativeHashMap<Entity, bool> and running an ‘AlwaysUpdate’ system to iterate over the map). But everything I’ve come up with seems unsafe or inefficient.

    if you have any experience in this area, I’d really appreciate the help.
     
  3. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    760
    ChangeFilter only works on chunk granularity, so you will need a different solution.

    If you run into issues with atomics, you can use a NativeQueue, sort the results, and then use an IJobParallelForFilter to remove any entity that is the same as the prior index.
     
    PublicEnumE likes this.
  4. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    Thank you for the advice.

    I am having quite a hard time figuring out how to use IJobParallelForFilter, or finding documentation for it. Even the various forum posts which mention it are confusing for some reason - some of the basic details of its functionality don't seem to be talked about. Sorry to ask for this, but would you mind explaining what it does? Or pointing me to an explanation?
     
  5. vestigial

    vestigial

    Joined:
    May 9, 2015
    Posts:
    109
    I think for this we are supposed to use an entity command buffer. So your parallel jobs all check which assassins can see the target, and commands are scheduled to mark the target as seen. Depending on which entity command buffer system you created the buffer from, eventually the system runs and performs the scheduled commands, in order, on the main thread. So that would be where the target actually gets the "seen" component added or iterated or whatever you do so your next system can operate on it.
    Note I'm new to ECS so I might be wrong on some or all of this, so check out the ECSSamples project, HelloCube section, SpawnerSystem_FromEntity.cs for an entity command buffer example. The one problem I can think of is depending when the systems update, there might be one frame of lag between when the ninjas spot the target, and when the next system picks it up.
     
  6. vestigial

    vestigial

    Joined:
    May 9, 2015
    Posts:
    109
    Although actually it might be a lot easier to break it up so the parallel ninja jobs don't try and modify the target, but instead modify the ninjas themselves to toggle a CanSeeTarget bool. Then the subsequent IsTargetSeen system can just get the count of entities where CanSeeTarget is true, no need for the command buffer.
     
  7. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    I really appreciate the input. But I feel like I'm just not finding the efficient approach that Unity intends. The best idea I've got so far is to:

    1. Write each "seen" target entity to a NativeQueue (this is my first, main job which figures out which assassins have seen which targets).
    2. Schedule a second job to sort that NativeQueue.
    3. Schedule a third, IJobParallelForFilter job to cull repeating Entities from the Queue, and assign them to a NativeList.
    4. Pass that list into a fourth IJobParallelForDefer, to assign the "seen" ComponentData for each target
    5. Schedule a fifth job to clear the NativeList.

    What am I missing here? This job chain seems like it would create a long sync point in my update loop, where subsequent systems are waiting for at least steps 1-4 to happen in sequence each frame.

    Is there a better way?

    Fwiw, I can already write each of the target Entities to a non-repeating NativeHashMap in 1 step. Is there some way for me to loop over those Key/Value pairs in a job?

    Many thanks for any comments.
     
    Last edited: Nov 26, 2019
  8. temps12

    temps12

    Joined:
    Nov 28, 2014
    Posts:
    25
    Totally depends on your use case but probably the fastest way in many cases are just to add to a queue in one parallel job and then just set the component in another job. Seems like you only want to set a bool so it doesn't matter if the bool is set multiple times on the same entity. Depending on how many entities we are talking about maybe it will be faster to sort first but simplest is to not care about that and then make a use case to optimize for later. Then you can compare simplest approach to the new maybe more optimized approach.
     
  9. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    Is this true? I had been assuming it was still unsafe (potential issues with data corruption if two threads tried to write to the same mem address at the same time).
     
  10. temps12

    temps12

    Joined:
    Nov 28, 2014
    Posts:
    25
    If the second job that consumes the queue isn't parallel it won't be any problems.
     
  11. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    But what about the chunk’s component version? Each time a component is assigned, that version number is incremented. If two threads try to increment that int at the same time, it could lead to incorrect results.
     
  12. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    You’re saying that the job which assigns the componentdata shouldn’t be parallel? Won’t be significantly slower?
     
  13. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,956
    I've been working on (well it's been finished for a month now) a very powerful general purpose vision library and it supports observations but I do it in reverse.

    For your example, instead of having the assassins tell the target it has been seen. I have targets check assassins if it can be seen. This completely avoids any threading issues. (Technically assassins generate a vision map and the observable entities check the map if they are within the observed area.)
     
    temps12 likes this.
  14. temps12

    temps12

    Joined:
    Nov 28, 2014
    Posts:
    25
    Yes, that's what I mean. It totally depends on how many entities we are talking about.

    Edit: Actually setting the component data can't be done in parallel with ComponentDataFromEntity
     
  15. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,692
    It can with NativeDisableParallelForRestriction attribute
     
  16. temps12

    temps12

    Joined:
    Nov 28, 2014
    Posts:
    25
    It comes down to if it is worth it to do the extra job of sorting and removing duplicates vs just setting the data multiple times on the same entity. And as tertle said, changing to a pull approach instead of push you avoid threading issues. Great to keep that approach in mind when coming up with solutions.
     
  17. temps12

    temps12

    Joined:
    Nov 28, 2014
    Posts:
    25
    Yes, but in this case there's a possibility of race conditions that could make it more unpredictable. The correct data will be set though. But yes, you can add that attribute to write to it parallel.

    I'm quite sure it's not incremented each time a thread access it, it's set to the component system version that scheduled the job. So if two jobs that were scheduled from the same system access the same chunk component data array with write access they will set the same version.
     
    PublicEnumE likes this.
  18. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    Looks like there may be a simple solution I was missing:

    Using NativeHashMap.TryAdd() to filter when I set new ComponentData, in my very first job:

    Code (CSharp):
    1. if(seenTargets.TryAdd(targetEntity, true))
    2. {
    3.     seenDataFromEntity[targetEntity] = new SeenData();
    4. }
    based on the NativeHashMap source, this appears to be atomic safe. And it does seem to function correctly.

    Am I missing something that will bite me in the ass later? Thanks for any advice.

    (fyi: There is no game with assassins or targets. This is an abstracted example.)
     
  19. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    162
    Wondering if anyone can back up this solution. It seems to work great, but it feels to easy to be true.
     
  20. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    760
    There are definitely ways your solution could come back to bite you. But it could also be the most optimal solution to your problem. I don't know. I don't know how many entities you are processing, how many results you are generating, and how complex the result-generating job is.

    Atomics don't scale well. They handle the "lots of computation, sparse results" case well, which might be your use case. But the more frequently the atomic operations are used, the more likely you are to create a memory traffic jam. Profile your code and see if you are happy with it.

    Others have also suggested inverting the point of view of the problem such that the targets self-determine if they are seen. Normally, I would suggest the same thing. But I bit my tongue in this case because I actually wouldn't use that technique here. I have an algorithm that lets me plop in two arbitrary groups of entities with spatial data, and I receive pairs of entities in an interface callback where I can safely write to a ComponentDataFromEntity to both entities in the pair despite the fact that this algorithm creates these callbacks in parallel.
     
    temps12 and PublicEnumE like this.
unityunity