Search Unity

Jobified Networking - job dependency problems

Discussion in 'Entity Component System' started by Spy-Shifty, Apr 4, 2019.

  1. Spy-Shifty

    Spy-Shifty

    Joined:
    May 5, 2011
    Posts:
    546
    Well this problem is really specific (Advanced).

    I've the following error message:
    "You are trying to schedule a new job which writes to the same NativeArray"

    So basically I'm writing NetworkSerializationSystems code.
    Therefore I have created custom jobtypes and containers for NativeNetworkWriter.
    And the error happens when two jobs of different NetworkSerializationSystems try to write on the NativeNetworkWriter

    Here is an example of a SerializationSystem:
    Code (CSharp):
    1. public class SerializePositionSystem : BBSSerializeJobComponentSystem {
    2.     [BurstCompile]
    3.     struct SerializePosJob : IJobSerializeNetworkComponentData<Translation> {
    4.         public void Execute(NativeNetworkWriter nativeNetworkWriter, ref Translation translation) {
    5.             nativeNetworkWriter.Write(translation.Value);
    6.         }
    7.     }
    8.  
    9.     protected override JobHandle OnUpdateNetwork(JobHandle inputDeps) {
    10.         return new SerializePosJob().Schedule(this, inputDeps);
    11.     }
    12. }

    Scheduling a IJobSerializeNetworkComponentData<T> works similar to IJobProcessComponentData
    except that it is singlethreaded (base job is IJob) and it provides the NativeNetworkWriter as parameter.

    The baseclass BBSNetworkBaseJobComponentSystem executes the OnUpdate and is responsible for adding inputDeps to the EndFrameBBSNetworking system:
    Code (CSharp):
    1.  
    2.     public abstract class BBSNetworkBaseJobComponentSystem : JobComponentSystem {
    3.         //....
    4.         protected sealed override JobHandle OnUpdate(JobHandle inputDeps) {
    5.             inputDeps = OnUpdateNetwork(inputDeps);
    6.             EndFrameBBSNetworking.AddJobHandleForNetworkProducer(inputDeps);
    7.             return inputDeps;
    8.         }
    9.     }
    10.  
    EndFrameBBSNetworking works similar to a normal EntityCommandBufferSystem except that it provides methods to create NativeNetworkWriter and is also responsible for CombineDependencies to ensure everything is threadsafe and finished at the execution time of EndFrameBBSNetworking:
    Code (CSharp):
    1.  
    2.     public class EndFrameBBSNetworking : EntityCommandBufferSystem {
    3.         //....
    4.         internal void AddJobHandleForNetworkProducer(JobHandle producerJob) {
    5.             m_NetworkProducerHandle = JobHandle.CombineDependencies(m_NetworkProducerHandle, producerJob);
    6.         }
    7.  
    8.  
    9.         protected override void OnUpdate() {
    10.             base.OnUpdate();
    11.             FlushBuffers();
    12.         }
    13.  
    14.         private void FlushBuffers() {
    15.             m_NetworkProducerHandle.Complete();
    16.             m_NetworkProducerHandle = new JobHandle();
    17.  
    18.             for (int i = 0; i < m_PendingWriters.Count; i++) {
    19.                 NativeNetworkWriter writer = m_PendingWriters[i];
    20.                 //TODO Data transfer to network API
    21.                 writer.Dispose();
    22.             }
    23.  
    24.             for (int i = 0; i < m_PendingReaders.Count; i++) {
    25.                 NativeNetworkReader reader = m_PendingReaders[i];
    26.                 reader.Dispose();
    27.             }
    28.  
    29.             m_PendingWriters.Clear();
    30.             m_PendingReaders.Clear();
    31.         }
    32.     }
    Also the NativeNetworkWriter has the same implementation as the DataStreamWriter from multiplay project from github:
    https://github.com/Unity-Technologi...ter/com.unity.transport/Runtime/DataStream.cs



    Now my problem is that I don't know why the system throw me the dependency exception...

    It's a lot of code so I can't provide full source code...

    By the way for all of you who are interested in this work.
    I plan to publisch this.


    Edit:
    A more detailed view of the error and the system
    upload_2019-4-4_20-37-1.png
     
    Last edited: Apr 4, 2019
  2. Spy-Shifty

    Spy-Shifty

    Joined:
    May 5, 2011
    Posts:
    546
    Ok I made a test with just the NativeNetworkWriter and two jobs in the same system
    Code (CSharp):
    1.  
    2. public class TestSys : JobComponentSystem {
    3.  
    4.     struct Job1 : IJob {
    5.         public NativeNetworkWriter NativeNetworkWriter;
    6.         public void Execute() {
    7.             NativeNetworkWriter.Write(1);
    8.         }
    9.     }
    10.  
    11.     struct Job2 : IJob {
    12.         public NativeNetworkWriter NativeNetworkWriter;
    13.  
    14.         public void Execute() {
    15.             NativeNetworkWriter.Write(2);
    16.         }
    17.     }
    18.  
    19.     protected override JobHandle OnUpdate(JobHandle inputDeps) {
    20.         var networkStream = new NativeNetworkWriter(1024, Unity.Collections.Allocator.TempJob);
    21.         var job1 = new Job1() {
    22.             NativeNetworkWriter = networkStream,
    23.         };
    24.  
    25.         var job2 = new Job2() {
    26.             NativeNetworkWriter = networkStream,
    27.         };
    28.  
    29.         inputDeps = job1.Schedule(inputDeps);
    30.         inputDeps = job2.Schedule(inputDeps);
    31.  
    32.         return inputDeps;
    33.     }
    34. }
    this works without any problem (ignore dealocation)
     
  3. Spy-Shifty

    Spy-Shifty

    Joined:
    May 5, 2011
    Posts:
    546
    I've found my problem...

    I have to add "inputDeps = JobHandle.CombineDependencies(EndFrameBBSNetworking.NetworkJobHandle, inputDeps);" right bevor OnUpdateNetwork function.

    Code (CSharp):
    1.  
    2.     public abstract class BBSNetworkBaseJobComponentSystem : JobComponentSystem {
    3.         protected sealed override JobHandle OnUpdate(JobHandle inputDeps) {
    4.             inputDeps = JobHandle.CombineDependencies(EndFrameBBSNetworking.NetworkJobHandle, inputDeps);
    5.             inputDeps = OnUpdateNetwork(inputDeps);
    6.             EndFrameBBSNetworking.AddJobHandleForNetworkProducer(inputDeps);
    7.             return inputDeps;
    8.         }
    9. }

    @Joachim_Ante is this the intended way for handling cross-system native containers?