[SOLVED-ish] Using ParallelWritter within Entities.ForEach

In the old

IJobForEachWithEntity

familiy there was a possibility to pass a Concurrent reference to Native collections in order to write into them. That was renamed at some point to

ParallelWritter

(or something of the sort, never really got time to use it), but I haven't been able to find how to use that with the

Entities.ForEach

costruction.

Can anybody point me in the right direction here? It would be greatly appreciated.

For a bit more of context (maybe I don't even need this actually). I am trying to populate a

NativeMultiHashMap

with some data I want to have available for a few different systems so that I don't have to calculate it every time. I could indeed not use a parallel job, but where would the fun be in that?

Anyhow, thanks again to anyone who gives me a hint.

 

I think I figured it out, I need to use

Entities.WithStructuralChanges().ForEach()

, at least I don't get any errors

 

Status update .. structural changes only work with

Run

, not with

Schedule

or

ScheduleParallel

.
So better than before, but less than ideal (provided off course there is a way of achieving this with

Schedule

or

ScheduleParallel

...)

 

WithStructuralChanges suggests you are touching EntityManager or some other managed type. What errors were you getting?

 

Not really (I think).

Basically I have a static reference to a

NativeMultiHashMap

holding specific values related to specific entities. Said reference is refreshed in every system update (instantiated on create, disposed on destroy).

There is this particular system that populates the values once every system update so that a few other systems that run afterwards can access it without having to calculate it more than once per update. I also take care of orchestrating the systems so that no system that relies on this runs before, off course.

So the problem is, without using the

WithStructuralChanges

with

Run

, the error I get is that the system that is attempting to populate the

NativeMultiHashMap

static reference is attempting to write on it, but it has read only access.

What I am aiming at, is being able to populate the

NativeMultiHashMap

in a

ScheduleParallel

. For now I'm settling with the current fix, as it allows me to move forward. But it would be great to understand how to do it the other way.

 

show code

 

Here you go. I removed some of the logic for the sake of readability.

In a nutshell, without the

WithStructuralChanges

you see there commented out, Unity complains that the system is trying to write on

hashMap

, but it is read-only.

Code (CSharp):

1. [UpdateAfter(typeof(PreviousSystems))]
2. public class PopulateHashMap : SystemBase
3. {
4.     public static NativeMultiHashMap<int, Data> hashMap;
5.  
6.     protected override void OnCreate()
7.     {
8.         base.OnCreate();
9.         hashMap = new NativeMultiHashMap<int, Data>(0, Allocator.Persistent);
10.     }
11.  
12.     private static int calculateKey(float3 pos)
13.     {
14.         return (int)(pos.x + pos.y + pos.z);
15.     }
16.  
17.     protected override void OnUpdate()
18.     {
19.         hashMap.Clear();
20.  
21.         var queryDesc = new EntityQueryDesc
22.         {
23.             None = new ComponentType[] { typeof(ComponentX) },
24.             All = new ComponentType[] { typeof(ComponentY) }
25.         };
26.         EntityQuery query = GetEntityQuery(queryDesc);
27.         var count = query.CalculateEntityCount();
28.  
29.         if (count > hashMap.Capacity)
30.         {
31.             hashMap.Capacity = count;
32.         }
33.  
34.         Entities
35.             // .WithStructuralChanges()
36.             .ForEach((Entity entity, int entityInQueryIndex,
37.                         ref ComponentA a,
38.                         in ComponentB b, in ComponentC c) =>
39.         {
40.             int key = calculateKey(b.Value);
41.             a.Value = key;
42.  
43.             hashMap.Add(key, new Data
44.             {
45.                 Entity = entity,
46.                 b = b.Value,
47.                 c = c.Value
48.             });
49.         }).Run();
50.     }
51.  
52.     protected void onDestroy()
53.     {
54.         hashMap.Dispose();
55.         base.OnDestroy();
56.     }
57.  
58. }
59.  
60. public struct Data
61. {
62.     public Entity Entity;
63.     public float3 b;
64.     public int c;
65. }
66.  

 

Does it work better if you assign the hashmap to a local var right before the Entities.FoeEach and use the local var version? I could see this combo throwing codegen a curveball.

 

Hmm .. that's interesting, and then assigning the local reference to the static variable before deallocating it after the job's done.

Will try that and get back to you. I guess I was so focused on getting the "parallel writer" working out I did not stop long to consider workarounds ....

Thanks @DreamingImLatios

 

As DreamingImLatios said, you need to assign it locally.

Code (CSharp):

1.  
2. var map = hashMap.AsParallelWriter();
3.  
4. Entities
5.     .ForEach((Entity entity, int entityInQueryIndex,
6.                 ref ComponentA a,
7.                 in ComponentB b, in ComponentC c) =>
8. {
9.     int key = calculateKey(b.Value);
10.     a.Value = key;
11.  
12.     map.Add(key, new Data
13.     {
14.         Entity = entity,
15.         b = b.Value,
16.         c = c.Value
17.     });
18. }).ScheduleParallel();

Should work fine. You can't capture managed classes in lambda.

 

Ok so, after a quick rewrite, it complains the static field is deallocated for some reason... these are the relevant changes.

Code (CSHARP):

1. ...
2.     protected override void OnUpdate()
3.     {
4.         hashMap.Clear();
5.  
6.         var queryDesc = [URL='http://www.google.com/search?q=new+msdn.microsoft.com']new[/URL] EntityQueryDesc
7.         {
8.             None = [URL='http://www.google.com/search?q=new+msdn.microsoft.com']new[/URL] ComponentType[] { [URL='http://www.google.com/search?q=typeof+msdn.microsoft.com']typeof[/URL](ComponentX) },
9.             All = [URL='http://www.google.com/search?q=new+msdn.microsoft.com']new[/URL] ComponentType[] { [URL='http://www.google.com/search?q=typeof+msdn.microsoft.com']typeof[/URL](ComponentY) }
10.         };
11.         EntityQuery query = GetEntityQuery(queryDesc);
12.         var count = query.CalculateEntityCount();
13.  
14.         if (count > hashMap.Capacity)
15.         {
16.             hashMap.Capacity = count;
17.         }
18.         NativeMultiHashMap<int, Data> tempMap = new NativeMultiHashMap<int, Data>(count, Allocator.TempJob);
19.  
20.         JobHandle handle = Entities
21.             .ForEach((Entity entity, int entityInQueryIndex,
22.                         ref ComponentA a,
23.                         in ComponentB b, in ComponentC c) =>
24.         {
25.             int key = calculateKey(b.Value);
26.             a.Value = key;
27.  
28.             tempMap.Add(key, [URL='http://www.google.com/search?q=new+msdn.microsoft.com']new[/URL] Data
29.             {
30.                 Entity = entity,
31.                 b = b.Value,
32.                 c = c.Value
33.             });
34.         }).Schedule(Dependency);
35.  
36.         handle.Complete();
37.  
38.         hashMap = tempMap;
39.  
40.         tempMap.Dispose();
41.     }

Gonna call it a night .. thanks everyone

 

wow .. I think you just gave me the answer I was looking for.. thanks @tertle !!

 

Yup ... worked like a charm! Thanks again to both of you! @tertle and @DreamingImLatios

 

Hmm .. now I'm getting a new issue.

After a while running, this pops up when trying to update the map (line 12 on @tertle's code sample):

InvalidOperationException: HashMap is full

 

without code can't see issue.

are you making sure there is enough space each frame or recreating it?

 

Well .. I do get the parallel writer reference after I have updated the capacity of the map .. do I need to do something else?