Can't share containers between systems

I have 2 systems (let's call them A and B) where System A processes some data and stores it on a native array that will be accessible from other systems (System B in this example). System B will update after System A and reads from System A native array.
I'm getting an exception complaining about the need of calling complete on the jobs that read from the native array.

Code (CSharp):

1. InvalidOperationException: The previously scheduled job SystemB:ReadJob reads from the NativeArray ReadJob.Data. You must call JobHandle.Complete() on the job SystemB:ReadJob, before you can deallocate the NativeArray safely.

If i'm correct about the error is that System A is deallocating the native array but that only happens on the next frame and feels like there shouldn't be a problem.
Is it possibile to achieve this without calling complete() on those jobs that read from the System A native array?

Here are my systems:
System A

Code (CSharp):

1. public class SystemA : JobComponentSystem {
2.   public NativeArray<int> ProcessedValues {
3.     get;
4.     private set;
5.   }
6.   public JobHandle InputDeps {
7.     get;
8.     private set;
9.   }
10.  
11.   Random m_random;
12.  
13.   protected override void OnCreate() {
14.     base.OnCreate();
15.     m_random = new Random((uint)System.DateTime.Now.Millisecond);
16.   }
17.  
18.   protected override void OnStopRunning() {
19.     base.OnStopRunning();
20.     if (ProcessedValues.IsCreated)
21.       ProcessedValues.Dispose();
22.   }
23.  
24.   [BurstCompile]
25.   struct ProcessJob : IJobParallelFor {
26.     [WriteOnly]
27.     public NativeArray<int> Data;
28.     public int Seed;
29.     public void Execute(int index) {
30.       var rnd = new Random((uint)(Seed + index));
31.       Data[index] = rnd.NextInt();
32.     }
33.   }
34.  
35.   protected override JobHandle OnUpdate(JobHandle inputDeps) {
36.     if (ProcessedValues.IsCreated)
37.       ProcessedValues.Dispose();
38.     ProcessedValues = new NativeArray<int>(1000, Allocator.TempJob);
39.     inputDeps = new ProcessJob {
40.       Data = ProcessedValues,
41.       Seed = m_random.NextInt()
42.     }.Schedule(ProcessedValues.Length, 64, inputDeps);
43.     InputDeps = inputDeps;
44.     return inputDeps;
45.   }
46. }

System B

Code (CSharp):

1. [UpdateAfter(typeof(SystemA))]
2. public class SystemB : JobComponentSystem {
3.  
4.   SystemA m_systemA;
5.  
6.   protected override void OnCreate() {
7.     base.OnCreate();
8.     m_systemA = World.GetOrCreateSystem<SystemA>();
9.   }
10.  
11.   [BurstCompile]
12.   struct ReadJob : IJobParallelFor {
13.     [ReadOnly]
14.     public NativeArray<int> Data;
15.     public void Execute(int index) {
16.       // Do something
17.     }
18.   }
19.  
20.   protected override JobHandle OnUpdate(JobHandle inputDeps) {
21.     inputDeps = JobHandle.CombineDependencies(inputDeps, m_systemA.InputDeps);
22.     inputDeps = new ReadJob {
23.       Data = m_systemA.ProcessedValues
24.     }.Schedule(m_systemA.ProcessedValues.Length, 64, inputDeps);
25.     // Calling complete here fixes the problem
26.     //inputDeps.Complete();
27.     return inputDeps;
28.   }
29. }

Thanks

 

It might not be relevant, but there seems to be some confusion between similar variable names and you keep overwriting the input param of the OnUpdate method

Try this on SystemB:

Code (CSharp):

1.  protected override JobHandle OnUpdate(JobHandle inputDeps) {
2.     //inputDeps.Complete(); // Why do you need this?
3.     if (ProcessedValues.IsCreated)
4.       ProcessedValues.Dispose();
5.     ProcessedValues = new NativeArray<int>(1000, Allocator.TempJob);
6.     InputDeps = new ProcessJob {
7.       Data = ProcessedValues,
8.       Seed = m_random.NextInt()
9.     }.Schedule(ProcessedValues.Length, 64, inputDeps);
10.     return InputDeps;
11.   }

Also in SystemA:

Code (CSharp):

1. protected override JobHandle OnUpdate(JobHandle inputDeps) {
2.     var combinedInputDeps = JobHandle.CombineDependencies(inputDeps, m_systemA.InputDeps);
3.     var jobHandle = new ReadJob {
4.       Data = m_systemA.ProcessedValues
5.     }.Schedule(m_systemA.ProcessedValues.Length, 64, combinedInputDeps);
6.     // Calling complete here fixes the problem
7.     //inputDeps.Complete();
8.     return jobHandle;
9.   }

 

System A needs the JobHandle of System B in OnStopRunning so that you don't destroy the array while System B is still reading it.

It may be worth developing a custom manager to keep track of inter-system dependencies like this.

 

Sorry.
You don't, it got hanging around from testing. Edited the system above.

 

https://forum.unity.com/threads/lock-free-native-queue.660286/#post-4423837

 

Thanks a lot. This is just what i was looking for.
The solution is to mark the read only array with [NativeDisableContainerSafetyRestriction]

Code (CSharp):

1.   [BurstCompile]
2.   struct ReadJob : IJobParallelFor {
3.     [ReadOnly]
4.     [NativeDisableContainerSafetyRestriction]
5.     public NativeArray<int> Data;
6.     public void Execute(int index) {
7.       // Do something
8.     }
9.   }

 

I made up a simple abstraction for managing shared dependencies. The native containers are declared and created/destroyed in the concrete implementation. With a dedicated ComponentSystem that creates/destroys those implementations and makes them available via public fields.

So as per @Joachim_Ante 's suggestion in another post, you call Combine before you schedule the job that uses a shared dependency, and after it call Set. Complete is for main thread access.

Code (csharp):

1.  
2. public abstract class SharedDependency
3.     {
4.         private JobHandle JobHandle;
5.  
6.         public void Complete()
7.         {
8.             if (!JobHandle.IsCompleted)
9.             {
10.                 JobHandle.Complete();
11.             }
12.         }
13.  
14.         public JobHandle Combine(JobHandle other)
15.         {
16.             return JobHandle.CombineDependencies(JobHandle, other);
17.         }
18.  
19.         public void Set(JobHandle other)
20.         {
21.             JobHandle = other;
22.         }
23.  
24.         public abstract void OnCreate();
25.         public abstract void OnDestroy();
26.     }
27.  

 

Another solution is just to schedule all your dependent jobs in one place, that way you can easily pass in whatever they need as you schedule them without passing your data all over the place. For my game I have a single "board" (NativeArray<Entity>) and I was originally trying to pass it between a bunch of systems along with the job handle. It never really felt right.

Eventually I realized (in my case at least) it was pointless to have all these separate systems operating on the same data. I broke out all the jobs from each system into their own file to keep things clean then I just schedule them all in order in a single BoardSystem:

Code (CSharp):

1.  
2.    protected override JobHandle OnUpdate(JobHandle inputDependencies)
3.     {
4.         // Clear the board
5.         for (int i = 0; i < board_.Length; ++i)
6.             board_[i] = Entity.Null;
7.         for (int i = 0; i < heightMap_.Length; ++i)
8.             heightMap_[i] = 0;
9.  
10.         var boardJob = inputDependencies;
11.  
12.         boardJob = new InitializeBoardJob
13.         {
14.             board = board_,
15.             childLookup = GetBufferFromEntity<Child>(true),
16.             tilesLookup = GetBufferFromEntity<PieceTiles>(true),
17.         }.Schedule(this, boardJob);
18.  
19.         boardJob = new BuildHeightmapJob
20.         {
21.             heightMap = heightMap_,
22.             tilesLookup = GetBufferFromEntity<PieceTiles>(true),
23.         }.Schedule(this, boardJob);
24.  
25.         And so on...
26.  

 

EntityManager.LockChunkOrder is for this purpose. It lets you build your array right into chunk iteration.