ECS and Networking

Far too early for me, to consider using networking as of yet. But is super cool that somebody is working on networking system.

Surely I will be watching thread.
All best.

 

No, I didn't tried that! Thank you for this post. I'll try to implement that!

At the moment I use reflection to get and set the values of the component members. So I have some generic methods. They create groups for that specific component type. Then I use cashed getter and setter (which I build dynamically in case of fields) to access the member data of that component.

Well, I'll take a look at the ArchetypeChunk API.

 

Whats the benefit of using the ArchetypeChunk instead of just creating the group?
Or is there something that I'm missing?


I didn't used the ArchetypeChunk API bevore. So I don't know if I'm right...
I've something like this (just an quick test for illustration)

Code (CSharp):

1.  private unsafe void Test<T>() where T: struct, IComponentData {  
2.      
3.         EntityArchetypeQuery query = new EntityArchetypeQuery {
4.             All = new ComponentType[] { typeof(T) },
5.             Any = Array.Empty<ComponentType>(),
6.             None = Array.Empty<ComponentType>()
7.         };
8.  
9.  
10.         NativeArray<ArchetypeChunk> chunks = EntityManager.CreateArchetypeChunkArray(query, Allocator.Temp);
11.         ArchetypeChunkComponentType<T> componentData = EntityManager.GetArchetypeChunkComponentType<T>(true);
12.         for (int i = 0; i < chunks.Length; i++) {
13.             ArchetypeChunk chunk = chunks[i];
14.             NativeSlice<T> chunkData = chunk.GetNativeSlice(componentData);
15.             for (int j = 0; j < chunkData.Length; j++) {
16.                 T data = chunkData[j];
17.                 int* dataPointer = null;
18.                 UnsafeUtility.CopyStructureToPtr(ref data, dataPointer);
19.                 int offset = UnsafeUtility.GetFieldOffset(fieldInfo);            
20.                 *(dataPointer + offset) = newValue;
21.                 UnsafeUtility.CopyPtrToStructure(dataPointer, out data);
22.                 chunkData[j] = data;
23.             }
24.         }
25.         chunks.Dispose();
26.     }

But I can archive the same with groups:

Code (CSharp):

1.  
2.     private unsafe void Test<T>() where T : struct, IComponentData {
3.         ComponentGroup group = GetComponentGroup(ComponentType.Create<T>());
4.         ComponentDataArray<T> dataArray = group.GetComponentDataArray<T>();
5.         for (int i = 0; i < dataArray.Length; i++) {
6.             T data = dataArray[i];
7.             int* dataPointer = null;
8.             UnsafeUtility.CopyStructureToPtr(ref data, dataPointer);
9.             int offset = UnsafeUtility.GetFieldOffset(fieldInfo);
10.             *(dataPointer + offset) = newValue;
11.             UnsafeUtility.CopyPtrToStructure(dataPointer, out data);
12.             dataArray[i] = data;
13.         }
14.     }

is this the correct way of how to use it?
I'm not sure

 

@Spy-Shifty awesome work! Not using it now but sure will soon. Looks very promising!

 

Thank you. Your answer is very helpful. May I ask, why did you decide to refuse that approach in your github repo?

 

Well, short answer. The first approach is more a low level api. My current networksystem is an high level api.

You don't have to worry about how to send your data. You define just the components and members you want to synchronize. All the data conversion, interpolation, optimization will be handled by the network api.
It's more like Unet vs legacy networking.

 

Thank you. For low level, Do you suggest to use the barrier or UpdateInGroup? Thank you.

 

With a barrier you have a fix point for updating your messages. Also you can use EntityComponentBuffer from that barrier within jobs.

Why do you don't use my framework?

 

I need more flexibility. I use Google.Protobuf with predefined messages instead of Protobuf-net runtime serialization. So you suggest to use barrier for low level, because we can access messages from jobs? I am sorry, I did not understand that part.

I appreciate your help.

 

If you're inside a job and you want to create, delete network messages you will need a EntityComandBuffer for that.
A jobsystem itself hasn't that. So you need a barrier for that.
To send all the data from that frame in the same frame, it requires that the SendNetworkMessageSystem runs after that barrier and all the other systems bevor that barrier.

For the ReceivingNetworkMessageSystem I would suggest to use a second barrier. Therfore ReceivingNetworkMessageSystem runs befor that barrier and all other Systems runs after that barrier.

Such a setup could be buggy in execution order so you have to keep an eye on it in the entity debugger.
Because the UpdateBefor/UpdateAfter System itself has some bugs. (Known by Unity)


So your system execution order could look like this(order is top first, bottom last)

• ReceiveNetworkMessageSystem
• ReceiveNetworkBarrier
• MySystem1 - Works with network messages
• MySystem2 - Works with network messages
• MySystem3 - Works with network messages
• ...
• SendNetworkBarrier
• SendNetworkMessageSystem
• EndFrameBarrier

 

Well I'm not sure but you can combine thouse two. Also you have the source

Anyway, share your experience with us, please

 

Thank you for the explanation. Now it makes sense to me. I will get back soon.

 

May I ask? Why we would use the second barrier for receiving system? In order to delete messages from a job?

 

Without the barrier it could look like that.

• MySystem1 - Works with network messages
• MySystem2 - Works with network messages
• ReceiveNetworkMessageSystem
  
• MySystem3 - Works with network messages
• ...
• SendNetworkBarrier
• SendNetworkMessageSystem
• EndFrameBarrier

So MySystem1 and MySystem2 won't see the receiving message in the same frame as MySystem3 does.

If this isn't a problem for you, then you won't need the barrier.

 

So the barrier will allow multiple systems to read the same message at the same frame? Without it, system 1 and 2 couldn't read the message because it was destroyed on previous frame?
Why don't just put [UpdateAfter(NetReceiveSystem)], why do we need to use [UpdateAfter(NetReceiveBarrier?]


Could you help me with an example? I would really appreciate that.
I have NetReceivePacketSystem and NetSendPacketSystem
But I don't listen to my network library messages in NetReceivePacketSystem as you do by NetworkManager.Service(); I am calling the method like this outside of the receive system.

So I have a system that polls events and when an internal message arrives calls receive method on NetReceiveSystem:

Code (CSharp):

1. public class NetServerSystem : ComponentSystem, INetEventListener
2. {
3.     private struct ServerGroup
4.     {
5.         [ReadOnly] public SharedComponentDataArray<NetLibrary> NetLibraries;
6.         [ReadOnly] public ComponentDataArray<Server> Servers;
7.  
8.         public readonly int Length;
9.     }
10.  
11.     [Inject] private ServerGroup servers;
12.     [Inject] private readonly NetReceivePacketSystem receivePacketSystem;
13.  
14.     protected override void OnUpdate()
15.     {
16.         if (servers.Length > 0)
17.         {
18.             servers.NetLibraries[0].Manager.PollEvents();
19.         }
20.     }
21.  
22.     public void OnNetworkReceive(NetPeer peer, NetDataReader reader)
23.     {
24.         receivePacketSystem.OnNetworkReceive(peer.ConnectId, reader.Data, true);
25.     }
26.  

And in OnNetworkReceive of packet system I am creating an Entity for message

Code (CSharp):

1. public class NetReceivePacketSystem : ComponentSystem
2. {
3.        public void OnNetworkReceive(long connectionId, byte[] data, bool dedicatedServer = false)
4.        {
5.             Entity receivedPacket = EntityManager.CreateEntity(receivedPacketArchetype);

So it means I have to put this system in order as well?
And then I have NetSendPacketBarrier and NetReceivePacketBarrier

lastly, test system that works with received messages in OnUpdate() : TestMessageHandler.

I would really appreciate if you would help me to sort it. Also, donation from me is coming.

 

If you jobify your system using JobComponentSystem you can't use the EntityManager nor PostCommandUpdate of the system. EntityManager just isn't possible and PostCommandUpdate doesn't exist in JobComponentSystem.

Thats why you should use a barrier. It's a sync point for creating, destroying and editting entities.

Well I don't know what else do you do in NetReceivePacketSystem. But in case of just holding OnNetworkReceive
it's not truely a system. It's more a delegator.

Yes, NetReceivePacketSystem isn't the executing instance here its NetServerSystem

So if you would have a system order like this

• NetReceivePacketSystem
• ReceiveNetworkBarrier
• MySystem1 - Works with network messages
• MySystem2 - Works with network messages
• NetServerSystem -> calls NetReceivePacketSystem.OnNetworkReceive
  
• MySystem3 - Works with network messages
• ...
• SendNetworkBarrier
• NetSendPacketSystem
• EndFrameBarrier

That means NetServerSystem creates messages somewhere in the frame. And in combination with the Barrier you would sync your messages but you would read them one frame later...

I don't understand why do you split up this into 3 systems. Maybe you can explane me why.

 

Because I have 2 listeners that split the logic. Client system and server system, but they both have OnNetworkReceive. I am using LiteNetLib. I still didn't get how to use reveive barrier. Thank you.

 

Ok. I've tested with debugging Time.frameCount

Everything is working fine. It creates and reads messages in the same time. I just need a suggestion how to jobify a system like "MySystem1" to read the message from barrier. Thank you.




NetClientSystem.cs

Code (CSharp):

1. public void OnNetworkReceive(NetPeer peer, NetDataReader reader)
2.         {
3.             Debug.Log("Received on frame: " + Time.frameCount);
4.             netReceiveSystem.OnNetworkReceive(peer.ConnectId, reader.Data);
5.         }

NetReceiveSystem.cs

Code (CSharp):

1.       public void OnNetworkReceive(long connectionId, byte[] data, bool dedicatedServer = false)
2.         {
3.             Debug.Log("Read on frame: " + Time.frameCount);        
4.             Entity receivedPacket = EntityManager.CreateEntity(receivedPacketArchetype);
5.            
6.             // snip
7.         }
8.  

TestMessageHandler.cs

Code (CSharp):

1. [UpdateInGroup(typeof(NetUpdate))]
2. public class TestMessageHandler : NetComponentSystem
3. {
4.     private ComponentGroup packetPool;
5.  
6.     protected override void OnCreateManager()
7.     {
8.         packetPool = GetComponentGroup(typeof(NetOpcodeFilter), typeof(NetReceivedPacket));
9.         packetPool.SetFilter(new NetOpcodeFilter { Value = Opcode.SOP_DATA });
10.     }
11.  
12.     protected override void OnUpdate()
13.     {
14.         int length = packetPool.CalculateLength();
15.         EntityArray entity = packetPool.GetEntityArray();
16.  
17.  
18.         Debug.Log("Injected on frame: " + Time.frameCount);
19. }

 

This is quite easy. You don't have to change much...


just use JobComponentSystem instead ComponentSystem and create and schedule your in there

Here is an example:

Code (CSharp):

1. [UpdateInGroup(typeof(NetUpdate))]
2. public class TestMessageHandler : JobComponentSystem
3. {
4.     private ComponentGroup packetPool;
5.    
6.    
7.     struct SomeJob : IJobParallelFor {
8.         [ReadOnly] EntityArray entities;
9.         [ReadOnly] ComponentDataArray<NetReceivedPacket> netReceivedPackets;
10.         public EntityCommandBuffer.Concurrent commandBuffer;
11.        
12.         public void Execute(int index) {
13.             // do some stuff
14.             commandBuffer.CreateEntity(index);
15.             commandBuffer.AddComponent(index, new SomeData { xy = netReceivedPackets[index].xy } );
16.            
17.         }
18.     }
19.    
20.    
21.     [Inject] EndFrameBarrier EndFrameBarrier;
22.     protected override void OnCreateManager()
23.     {
24.         packetPool = GetComponentGroup(typeof(NetOpcodeFilter), typeof(NetReceivedPacket));
25.         packetPool.SetFilter(new NetOpcodeFilter { Value = Opcode.SOP_DATA });
26.     }
27.     protected override JobHandle OnUpdate(JobHandle inputDeps)
28.     {
29.         int length = packetPool.CalculateLength();
30.         EntityArray entity = packetPool.GetEntityArray();
31.         ComponentDataArray<NetReceivedPacket> data = packetPool.GetComponentDataArray<NetReceivedPacket>()
32.         Debug.Log("Injected on frame: " + Time.frameCount);
33.        
34.        
35.         var someJob = new SomeJob() {
36.             entities = entity,
37.             netReceivedPackets = data,
38.             commandBuffer = EndFrameBarrier.CreateCommandBuffer().ToConcurrent(),
39.         };
40.         inputDeps = someJob.Schedule(length, 64, inputDeps);
41.        
42.         return inputDeps;
43.     }
44. }

Some references:
https://github.com/Unity-Technologi...temSamples/blob/master/Documentation/index.md
https://docs.unity3d.com/Manual/JobSystem.html

 

Thank you. But still is it necessary to have the receive barrier? There is no need to use it in such setup. And instead, pass a byte[] to a job or deserialized packet.

 

well no you won't need the barrier

Use DynamicBuffer!

https://github.com/Unity-Technologi...ster/Documentation/content/dynamic_buffers.md

Byte or Deserialized is kind of your strategy

 

Yes. I am using it. I've created ByteBuffer. So my strategy that I will read a packet and deserialize it in the main thread and then pass all of the needed data into a job. Thank you.

 

how is this going? last update was in 2018

 

Unfortunately this is not under active development anymore