ECS and Networking

EDIT:

I've developed an Networking Framework for Unity's Entity Component System
It's free to use. I would be happy for feedback. What do you want to improve? Some other features, performance...
BBSNetworkSystem


Simple shooter example:
BBSNetworkSystemExample


-----------------------------------------------------
Hi all,

I'm struggling on networking in an ecs context a little bit....

I've a simple shooter like game. So what I need to sync is:

• positions,
• instantiations,
• some player props (health and so on)

nothing special...

Now, I search for a good way to go of organizing my systems and components..

Spoiler: 1. Approach

I've two systems for sending and receiving network data.
I would take use of the following entity structure:

Message Entity:

• NetworkEventComponent [Tag],
• NetworkOwnerComponent { Entity sender/receiver}
• NetworkMessageComponent { eventId, datasteam }

I would also create systems to read and write from and into the stream.

Cons:

• well the problem would be the use of the NetworkMessageComponent in combination of a stream.
  (We can't use arrays, and I won't use SharedComponentData in combination with nativearrays)

Pros:

• each system has a well defined task
  ->adding new behaviour would be easy (only adding new systems)

Spoiler: 2. Approach

again two systems for sending and receiving network data.

Message Entity:

• NetworkEventComponent [Tag],
• NetworkOwnerComponent { Entity sender/receiver}
• NetworkSpecificMessageComponent { e.g. Vector3 position }

I would have to create a switch statement or some converter classes to transfere my specific components for networking

Cons:

• adding new behaviour would be complicated (adding new components, adding converter for each component, handling component for networkstream)
• not generic

Pros:

• it would be feasible

Spoiler: 2. Approach

I've one system for receiving network data (Networkmanager requires an update method)
The network update system provides an registration method for other systems with an specific interface.

The idea is that I can register a specific system (NetworkPositionSyncSystem) for an specific network message, so that I can handle sending and receiving directly in that system.

Cons:

• It will break job dependencies because of the delegate methods to other systems
• Some overhead for registration of systems (code writing)

Pros:

• it would be feasible
• each system has a well defined task
  ->adding new behaviour would be easy (only adding new systems)

Well the tasks are always the same, but the focus will change (where to do the task and what dependencies will I have between components and systems and so one)

Well what do you think would be the best approach or does someone has a better idea.

 

i see it this way:
Any entities that need to be synchronized over network are marked with a "Synchronized" tag (containing some options)
Networking system, on its internal frequency, magically synchronizes those entities, and no one knows or care how it does it.

 

the issue I'm seeing with this is that you'll lose countless opportunities for bandwidth optimization with such an automagical system. I think it would work for very simple projects, but the lack of explicit case-by-case low-level control over serialization will be an instant turn-off for others.

This was one of the biggest problems with HLAPI

 

options in Synchronized tag may contain optimization instructions ( like "send only if changed", "synchronize only in one direction", "synchronize only once", "synchronize every 5 min", "synchronize only partially", "don't care if it get lost" etc)

 

There are other cases where attributes probably would not give you enough control. For example:

• serializing an entire quaternion as just 4 bytes instead of 16 if you know it'll just rotate around one specific euler axis
• serializing a Vector3 as half-precision or even single-byte precision depending on some arbitrary conditions like distance from camera, occlusion, evaluated priority, etc....
• making serialization frequency depend on a specific condition
• preceding the serialization of any struct or data type with a single byte that could be used to describe frequent "presets" for that data, so that most of the time it'll be serialized to one byte, and the rest of the time it'll take one byte more than it normally would
• etc, etc

 

nothing prevents you from making a system that prepare data to be synchronized, it may extract and minify data to be sent, store it in separate entity, and after synchronization unpack it back

 

The problem I see here is: That you have one big system basically for all your components. As I understood this right.
Adding and removing features wouldn't be easy over time.

This is what I can archive with my first approach.

 

it should be a generic system that can send any IComponentData

benefit is that all networking is inside that system, other systems don't need to care about any network specific, you can replace "network driver system" and use p2p or lan or dove mail for example in a same manner

 

Well,
yeah I think you're right

worst case scenario is that you have to manually write a system that pre/post-processes data upon sending or recieving for message types that require custom serialization, but I think that would work

 

There could always be a code-generation approach that generates "networking systems" for each different type of entity (or each different type of message, depending on the architecture)

 

So you say the network system should sync an entire entity? E.g. serialize all components and then do a deltacompression...

Something like that?

 

yes, and you have to keep that entity clean to get better performance

this is my vision, probably not best approach

 

Realtime games generally have two classes of data. 90%+ is data you sent infrequently. Just use a good serializer like protobuf with a sane data model and send the entire thing. You are primarily optimizing for developer productivity here.

Then you have a small set of data that you send frequently. Movement/combat related data being the most common. For that you send only what is needed.

For both I generally standardize on convert floats to integers with the precision you need. I generally use those as the default server side, only using floats in the few cases where it's actually needed, like if you need to do area of interest queries. I use integers throughout for database id's, everything. You should never have to send a string over the wire outside of one time setup/initialization if designed well.

FYI protobuf packed/repeated fields fit nicely with component data design. Both are basically flattened structures and the way it works copying from components to your network messages will be extremely efficient. Not so much because of the copying actually but due to being able to fit many logical entities into a single network message.

To visualize this better this is my network message for sending position/rotation/input updates (both ways, some fields are optional based on context). I'm representing data as integers with a precision of 2. I'm not sending stuff I don't need to, like the Y vector I can deduce on the client. Rotations I send as a single float(packed to integer) heading.

I only do the following packed style for frequently sent data. Also note the singular entity type field. I send entity types in their own message to avoid having a list of entity types.

Code (csharp):

1.  
2. [Serializable]
3.     [ProtoContract]
4.     public class TrackDataPacked : IGameMessage
5.     {
6.         [ProtoMember(1, IsPacked = true, DataFormat = DataFormat.ZigZag)]
7.         public List<int> PositionX = new List<int>();
8.  
9.         [ProtoMember(2, IsPacked = true, DataFormat = DataFormat.ZigZag)]
10.         public List<int> PositionZ = new List<int>();
11.  
12.         [ProtoMember(3, IsPacked = true, DataFormat = DataFormat.ZigZag)]
13.         public List<int> Haxis = new List<int>();
14.  
15.         [ProtoMember(4, IsPacked = true, DataFormat = DataFormat.ZigZag)]
16.         public List<int> Vaxis = new List<int>();
17.  
18.         [ProtoMember(5, IsPacked = true)]
19.         public List<int> Heading = new List<int>();
20.  
21.         [ProtoMember(6, IsPacked = true)]
22.         public List<int> EntityPoolId = new List<int>();
23.  
24.         [ProtoMember(7, IsPacked = true, DataFormat = DataFormat.ZigZag)]
25.         public List<int> Speed = new List<int>();
26.  
27.         [ProtoMember(8)]
28.         public GameEntityType EntityType;
29.  
30.         [ProtoMember(9, IsPacked = true)]
31.         public List<int> Alive = new List<int>();
32.  
33.         public void Clear()
34.         {
35.             PositionX.Clear();
36.             PositionZ.Clear();
37.             Haxis.Clear();
38.             Vaxis.Clear();
39.             Heading.Clear();
40.             EntityPoolId.Clear();
41.             Speed.Clear();
42.             Alive.Clear();
43.         }
44.  
45.         public void Add(int entityId, int x, int z, int heading, int haxis, int vaxis, int speed)
46.         {
47.             PositionX.Add(x);
48.             PositionZ.Add(z);
49.             Haxis.Add(haxis);
50.             Vaxis.Add(vaxis);
51.             Heading.Add(heading);
52.             Speed.Add(speed);
53.             EntityPoolId.Add(entityId);
54.         }
55.  
56.         public void AddVehicle(int entityId, int x, int z, int heading)
57.         {
58.             PositionX.Add(x);
59.             PositionZ.Add(z);
60.             Heading.Add(heading);
61.             EntityPoolId.Add(entityId);
62.         }
63.  
64.         public void AddAgent(int entityId, int x, int z)
65.         {
66.             PositionX.Add(x);
67.             PositionZ.Add(z);
68.             EntityPoolId.Add(entityId);
69.         }
70.  
71.     }
72.  

 

Thank you for this detailed answer!
This fills a piece of puzzle in my mind!

 

I am interested in ECS & Networking as well.
Would you mind sharing information on what you managed to achieve until now ?
That would be awesome !

 

Well I almost testing around!

At the moment I use a static NetworkManager which contains a instance of Photon RealtimeApi class LoadBalanceClient, provides all necessary information to the network and publishes some network events.

I take use of a NetworkMessageBarrier for ordering executions. It's located after ScriptRunBehaviourFixedUpdate and before DirectorFixedUpdate

The NetworkManagerSystem just calls the update method of the NetworkManager which handles reading from the buffer and calling events (photon internal stuff). It also handles connection / disconnection to the photon cloud master though the NetworkManager. Its located after EndFrameBarrier and befor NetworkMessageBarrier. System can register for specific data receiving events by providing an OnReceiveData method and an eventId. So each system receives only that type of data that it has register for.

I also have GameManagerSystem which subscripe to event of the Networkmanager to handle connection events like OnPlayerJoined, OnPlayerLeft,... So the system handles e.g. creating player character.


As @SubPixelPerfect sad before, each entity which should be synchronized with the network will need a SychronizeComponent attached to it.

The NetworkSynchronizeSystem will handle each entity based on the SyncOptions of the SychronizeComponent.
At the moment I just handle the positioning and orientation.

I handle data as @snacktime said, using ProtoBuffer-net lib.

Code (CSharp):

1.  
2. [Flags]
3. public enum SyncOptions : int {
4.     Owner = 1 << 0,
5.     Position = 1 << 1,
6.     Heading = 1 << 2,
7. }
8.  
9. struct SychronizeComponent : ComponentData {
10.        public int NetworkId;
11.        Public SyncOptions Options;
12. }
13.  
14.  

As I sad I still testing around.

 

Thanks for the heads up !

 

Ok short update how the system looks for now!

I had some problems with reading and writing data to Components.
My network was updating after EndFrameBarrier, the result was that all componentarrays were deallocated at that moment. On the other side I wanted a fix point there all my components would be synchronized through the network and writing is save...

So I decided to change my layout.

Systems won't interact with the Networkmanager directly. They will do it through the ecs but how?

Well each incomming or outgoing event will be stored in form of an entity. So that my Networksystem and other systems can read / write from and to them.


Here is the source:

Components:
// This component stores the information of the eventId in it and can be used to filter for an specific event. This component is used for both, sending and reseiving data.

Code (CSharp):

1. public struct NetworkMessageEvent : ISharedComponentData {
2.     public byte id;
3. }

// This component indecates that the data should be sended and if it should be reliable or not

Code (CSharp):

1. public struct NetworkSendEvent : IComponentData {
2.     public bool1 reliable;
3. }

// this component indecates that the data was received from the network and contains the senderId

Code (CSharp):

1. public struct NetworkReceiveEvent : IComponentData {
2.     public int senderId;
3. }

The data of the message will be stored in the entity as well. This is done by use of an FixedArray with typeof(byte). So that the message is always serialized as a byte array.

Code (CSharp):

1. EntityManager.AddComponent(entity, ComponentType.FixedArray(typeof(byte), dataBuffer.Length));
2. NativeArray<byte> messageData = EntityManager.GetFixedArray<byte>(entity);

Systems:
This is the Networkbarrier and helps to sort the execution order of the network system and other systems.
It will be placed after ScriptRunBehaviourFixedUpdate and befor the DirectorFixedUpdate.

Code (CSharp):

1. [UpdateAfter(typeof(UnityEngine.Experimental.PlayerLoop.FixedUpdate.ScriptRunBehaviourFixedUpdate))]
2. [UpdateBefore(typeof(UnityEngine.Experimental.PlayerLoop.FixedUpdate.DirectorFixedUpdate))]
3. public class NetworkMessageBarrier : BarrierSystem { }

The following system handles all receiving data and calls the NetworkManager.Service() method, which calls the Photon.Service() method.

Well, if a message arrives this system from the NetworkManager, it will create an entity with an eventId, senderId, and the buffered data attached to it. Now all other systems can read from that entity. After a frame, the created entity will automatically destroyed by use of endFrameBarrier.CreateCommandBuffer().DestroyEntity(entity) (We call this system after the EndFrameBarrier so that the entity lifes exactly one frame)

Code (CSharp):

1.  
2. [UpdateAfter(typeof(EndFrameBarrier))]
3. [UpdateBefore(typeof(NetworkMessageBarrier))]
4. public class NetworkReceiveMessageSystem : ComponentSystem {
5.  
6.     private EntityArchetype networkReceiveMessageArchetype;
7.     [Inject] EndFrameBarrier endFrameBarrier;
8.  
9.     protected override void OnCreateManager(int capacity) {
10.         networkReceiveMessageArchetype = EntityManager.CreateArchetype(typeof(NetworkMessageEvent), typeof(NetworkReceiveEvent));
11.         NetworkManager.OnEventData += NetworkManager_OnEventData;
12.         NetworkManager.Connect();
13.     }
14.  
15.     private void NetworkManager_OnEventData(byte eventId, Player player, object data) {
16.         byte[] dataBuffer = (byte[])data;
17.         Entity entity = EntityManager.CreateEntity(networkReceiveMessageArchetype);
18.         EntityManager.SetSharedComponentData(entity, new NetworkMessageEvent { id = eventId });
19.         EntityManager.SetComponentData(entity, new NetworkReceiveEvent { senderId = player.ID });
20.  
21.         EntityManager.AddComponent(entity, ComponentType.FixedArray(typeof(byte), dataBuffer.Length));
22.         NativeArray<byte> messageData = EntityManager.GetFixedArray<byte>(entity);
23.         for (int i = 0; i < dataBuffer.Length; i++) {
24.             messageData[i] = dataBuffer[i];
25.         }
26.         endFrameBarrier.CreateCommandBuffer().DestroyEntity(entity);
27.     }
28.  
29.     protected override void OnDestroyManager() {
30.         NetworkManager.Disconnect();
31.     }
32.     protected override void OnUpdate() {
33.         NetworkManager.Service();
34.     }
35. }

This system will watch for enties with want to send data to the network. After handling an entiy it will destroy it, so that we won't handle them twice. The execution order of this system is not importend! But needs to be executed befor EndFrameBarrier, because we read from entites.

Code (CSharp):

1.  
2. public class NetworkSendMessageSystem : ComponentSystem {
3.     struct SendData {
4.         [ReadOnly] public SharedComponentDataArray<NetworkMessageEvent> networkEvent;
5.         public ComponentDataArray<NetworkSendEvent> sendEvent;
6.         public FixedArrayArray<byte> message;
7.         public EntityArray entity;
8.         public int Length;
9.     }
10.  
11.     [Inject] SendData sendData;
12.  
13.     protected override void OnUpdate() {
14.         for (int i = 0; i < sendData.Length; i++) {
15.  
16.             NativeArray<byte> messageData = sendData.message[i];
17.             byte[] dataBuffer = new byte[messageData.Length];
18.             for (int j = 0; j < dataBuffer.Length; j++) {
19.                 dataBuffer[j] = messageData[j];
20.             }
21.  
22.             NetworkManager.SendMessage(sendData.networkEvent[i].id, dataBuffer, sendData.sendEvent[i].reliable);
23.             PostUpdateCommands.DestroyEntity(sendData.entity[i]);
24.         }
25.     }
26. }

With less explanation... Here is the use case of reading and wirting data from network entities...

Code (CSharp):

1. using ExitGames.Client.Photon;
2. using ExitGames.Client.Photon.LoadBalancing;
3. using ProtoBuf;
4. using System;
5. using System.Collections;
6. using System.Collections.Generic;
7. using System.IO;
8. using Unity.Collections;
9. using Unity.Entities;
10. using Unity.Jobs;
11. using Unity.Mathematics;
12. using UnityEngine;
13.  
14. [ProtoContract]
15. public class TrackDataPacked {
16.  
17.     [ProtoMember(1, IsPacked = true, DataFormat = DataFormat.ZigZag)]
18.     public List<int> NetworkId = new List<int>();
19.  
20.     [ProtoMember(2, IsPacked = true, DataFormat = DataFormat.ZigZag)]
21.     public List<int> PositionX = new List<int>();
22.  
23.     [ProtoMember(3, IsPacked = true, DataFormat = DataFormat.ZigZag)]
24.     public List<int> PositionZ = new List<int>();
25.  
26.     [ProtoMember(4, IsPacked = true, DataFormat = DataFormat.ZigZag)]
27.     public List<int> Heading = new List<int>();
28.  
29.     [ProtoMember(5, IsPacked = true, DataFormat = DataFormat.ZigZag)]
30.     public List<int> VAxis = new List<int>();
31.  
32.     public void Clear() {
33.         NetworkId.Clear();
34.         PositionX.Clear();
35.         PositionZ.Clear();
36.         Heading.Clear();
37.         VAxis.Clear();
38.     }
39.  
40.     public void Add(int entityId, int x, int z, int hAxis, int vAxis) {
41.         NetworkId.Add(entityId);
42.         PositionX.Add(x);
43.         PositionZ.Add(z);
44.         Heading.Add(hAxis);
45.         VAxis.Add(vAxis);
46.     }
47. }
48.  
49. public class NetworkSynchronizeSystem : ComponentSystem {
50.  
51.     struct SyncronizedData {
52.         public ComponentDataArray<Syncronize> synchronized;
53.         public ComponentDataArray<NetworkPosition> networkPosition;
54.         public ComponentDataArray<NetworkHeading> networkHeading;
55.         public ComponentArray<Transform> transform;
56.         public int Length;
57.     }
58.  
59.     [Inject] SyncronizedData synchronizedData;
60.  
61.     private ComponentGroup receivedData;
62.  
63.     private int lastExectutionTime = Environment.TickCount & Int32.MaxValue;
64.     private TrackDataPacked trackDataPacked = new TrackDataPacked();
65.  
66.     protected override void OnCreateManager(int capacity) {
67.         receivedData = GetComponentGroup(typeof(NetworkMessageEvent), typeof(NetworkReceiveEvent));
68.         receivedData.SetFilter(new NetworkMessageEvent { id = NetworkEventTags.TrackedData });
69.     }
70.  
71.  
72.  
73.     protected override void OnUpdate() {
74.         ReadTrackedData();
75.         WriteTrackedData();
76.         UpdateEntities();
77.     }
78.  
79.     private void UpdateEntities() {
80.         float teleportDistance = Bootstrap.GameSettings.TeleportDistance;
81.         float motionInterpolationSpeed = Bootstrap.GameSettings.MotionInterpolationSpeed;
82.         float rotationInterpolationSpeed = Bootstrap.GameSettings.RotationInterpolationSpeed;
83.  
84.         for (int i = 0; i < synchronizedData.Length; i++) {
85.             Syncronize syncronize = synchronizedData.synchronized[i];
86.             if (syncronize.IsOwner) {
87.                 continue;
88.             }
89.             NetworkPosition networkPosition = synchronizedData.networkPosition[i];
90.             NetworkHeading networkHeading = synchronizedData.networkHeading[i];
91.             Transform transform = synchronizedData.transform[i];
92.  
93.             Vector3 newPosition = new Vector3(NetworkHelper.ConvertToFloat(networkPosition.x), transform.position.y, NetworkHelper.ConvertToFloat(networkPosition.z));
94.             if (Vector3.Distance(transform.position, newPosition) > teleportDistance) {
95.                 transform.position = newPosition;
96.             } else {
97.                 transform.position = Vector3.Lerp(transform.position, newPosition, Time.deltaTime * motionInterpolationSpeed);
98.             }
99.             transform.rotation = Quaternion.RotateTowards(transform.rotation, Quaternion.Euler(0, NetworkHelper.ConvertToFloat(networkHeading.value), 0), Time.deltaTime * rotationInterpolationSpeed);
100.         }
101.     }
102.  
103.     private void WriteTrackedData() {
104.         int sendMessageInterval = Bootstrap.GameSettings.SendMessageInterval;
105.         int currentTime = Environment.TickCount & Int32.MaxValue;
106.         if (math.abs(lastExectutionTime - currentTime) < sendMessageInterval) {
107.             return;
108.         }
109.  
110.         lastExectutionTime = Environment.TickCount & Int32.MaxValue;
111.  
112.         trackDataPacked.Clear();
113.         for (int i = 0; i < synchronizedData.Length; i++) {
114.             Syncronize syncronize = synchronizedData.synchronized[i];
115.             if (!syncronize.IsOwner) {
116.                 continue;
117.             }
118.  
119.             trackDataPacked.NetworkId.Add(syncronize.NetworkId);
120.             Transform transform = synchronizedData.transform[i];
121.             int posX = NetworkHelper.ConvertToInt(transform.position.x);
122.             int posZ = NetworkHelper.ConvertToInt(transform.position.z);
123.             trackDataPacked.PositionX.Add(posX);
124.             trackDataPacked.PositionZ.Add(posZ);
125.             synchronizedData.networkPosition[i] = new NetworkPosition { x = posX, z = posZ };
126.  
127.             int heading = NetworkHelper.ConvertToInt(transform.rotation.eulerAngles.y);
128.             trackDataPacked.Heading.Add(heading);
129.             synchronizedData.networkHeading[i] = new NetworkHeading { value = heading };
130.         }
131.  
132.         using (var stream = new MemoryStream()) {
133.             Serializer.Serialize(stream, trackDataPacked);
134.             Bootstrap.CreateNetworkMessage(NetworkEventTags.TrackedData, false, stream.ToArray());
135.         }
136.     }
137.  
138.  
139.  
140.     private void ReadTrackedData() {
141.         int length = receivedData.CalculateLength();
142.         EntityArray entity = receivedData.GetEntityArray();
143.         for (int i = 0; i < length; i++) {            
144.             byte[] dataBuffer = EntityManager.GetFixedArray<byte>(entity[i]).ToArray();
145.             using (var stream = new MemoryStream(dataBuffer)) {
146.                 var trackDataPacked = Serializer.Deserialize<TrackDataPacked>(stream);
147.  
148.                 NativeArray<int> networkIds = new NativeArray<int>(trackDataPacked.NetworkId.Count, Allocator.TempJob, NativeArrayOptions.UninitializedMemory);
149.                 for (int j = 0; j < trackDataPacked.NetworkId.Count; j++) {
150.                     networkIds[j] = trackDataPacked.NetworkId[j];
151.                 }
152.  
153.                 var networkIdToSynchMap = new NativeHashMap<int, int>(trackDataPacked.NetworkId.Count, Allocator.TempJob);
154.                 var networkIdToEntityMapperJob = new NetworkIdToSynchronizeMapperJob {
155.                     networkIds = networkIds,
156.                     syncronize = synchronizedData.synchronized,
157.                     networkIdToSynchMap = networkIdToSynchMap
158.                 };
159.  
160.                 JobHandle networkIdToEntityMapperJobFence = networkIdToEntityMapperJob.Schedule(networkIds.Length, 1); //JobHandle?
161.                 networkIdToEntityMapperJobFence.Complete();
162.  
163.                 for (int j = 0; j < trackDataPacked.NetworkId.Count; j++) {
164.                     int id = networkIds[j];
165.                     int index;
166.                     if (!networkIdToSynchMap.TryGetValue(id, out index)) {
167.                         continue;
168.                     }
169.  
170.                     if (synchronizedData.synchronized[index].IsOwner) {
171.                         continue;
172.                     }
173.  
174.                     synchronizedData.networkPosition[index] = new NetworkPosition {
175.                         x = trackDataPacked.PositionX[j],
176.                         z = trackDataPacked.PositionZ[j],
177.                     };
178.  
179.                     synchronizedData.networkHeading[index] = new NetworkHeading { value = trackDataPacked.Heading[j] };
180.                 }
181.                 networkIds.Dispose();
182.                 networkIdToSynchMap.Dispose();
183.             }
184.         }
185.     }
186.  
187.     #region Jobs
188.     private struct NetworkIdToSynchronizeMapperJob : IJobParallelFor {
189.         [ReadOnly] public ComponentDataArray<Syncronize> syncronize;
190.         [ReadOnly] public NativeArray<int> networkIds;
191.  
192.         [WriteOnly] public NativeHashMap<int, int>.Concurrent networkIdToSynchMap;
193.  
194.         public void Execute(int index) {
195.             int id = networkIds[index];
196.             for (int i = 0; i < syncronize.Length; i++) {
197.                 Syncronize synchronize = syncronize[i];
198.                 if(synchronize.NetworkId == id) {
199.                     networkIdToSynchMap.TryAdd(id, i);
200.                 }
201.             }
202.         }
203.     }
204.     #endregion
205. }
206.  

There is some potential for optimization as well but for the first tests I'm happy with the result!

 

Thank you so much for sharing that much, I think it will be really useful to many people !

One major issue I see with this is that it does not seem easy to handle rollback, for instance when doing client-side prediction. Do you plan on adding some client-side prediction ? If so, how would you go about it ?

Thanks again !

 

Well all depends on your scenario.
I've a really simple one, so I think this won't match all cases...

In my case I store the position send through the network in a NetworkPosition component.

Code (CSharp):

1. struct NetworkPosition : IComponentData {
2.      public int X;
3.      //public int Y;
4.      public int Z;
5. }

This position is a fixed point (integer value with accuracy of 2 digits). This value expect always the real world situation. which is send by the authority or the owning client...

Code (CSharp):

1.  
2. public static class NetworkHelper {
3.     public const int DEFAULT_ACCURACY = 100;
4.     public static int ConvertToInt(float value, int accuracy = DEFAULT_ACCURACY) {
5.         return (int)(value * accuracy);
6.     }
7.     public static float ConvertToFloat(int value, int accuracy = DEFAULT_ACCURACY) {
8.         return (float)value / accuracy;
9.     }
10. }

My NetworkSynchronizeSystem handles the writing of this NetworkPosition component and also handles the interpolation between object position and network position. So if I would implement a prediction of the next position, and the prediction won't fit, I can simply rollback to my old network position. Now I assigning the new network position and do the interpolation, extrapolation... stuff on the object.


Well if you have a more complex scenario...
Instead of assigning the NetworkPosition component directly to the target entity, you can e.g. create a proxy entity. (I don't know if proxy would be the right word for that...)
This proxy entity contains all relevant data of the target object state at time t. It also knows its target object.

With such a construct you can jumb back and forth in time as you want. Because you can create thousends of such proxy entity for just one target object.

This is just a stimulus and my first thinking about it...

 

If we make the assumption that serialization of entities and components will be added in the future, what do you think about serializing the whole game state (i.e. all entities and their components) and using delta compression to send the whole game state to the clients ?

You could then easily handle interpolation and client-side prediction by running some systems on the last received state, etc.

In my opinion it has the major advantage of being easy to implement (if we can easily compute the delta) and new features are automatically handled (you don't need to write some feature-specific network code).
One immediate drawback I see is that it could use a lot of bandwidth depending on the game, and also doesn't allow to have different synchronization rates per entity/client (thus, it is probably not suitable for mmo).

Any opinion on that ?

 

Hi well I'm working on a similar approach.

But instead of sync whole game state, I just sync specific components / member of entities.

My idea is to provide some attributes that indicates what and how an component / member will be synchronized.

The system will than create an phantom entity that only contains the network view of the sync entity and its components in an deterministic way (integer as fixedpoint).

The advantage is that I can interpolate between the networkstate and the real state easly.
But this system is still in an early state.

The main problem is that I need to use reflection to sync real state and network view...

 

Short update from mine:

I've the basics of network sync system done!

How does it look like from outside?
So if you would like to sync a component through the network, you only have to do 3 simple steps!

1. Create Components with an NetSync attribute attached to it.
2. Define what member should be synchronized through the network by attaching a NetSyncMember attribute to it.
3 Create an entity add a NetworktSync component to it, and a NetworkOwner component to it if you are the owner.

E.g.

Code (CSharp):

1. [NetSync]
2. public struct ComponentA : IComponentData {
3.     [NetSyncMember(lerpSpeed:1)]
4.     public float Data1;
5.     [NetSyncMember]
6.     public bool1 Data2;
7.     [NetSyncMember]
8.     public int Data3;
9. }
10.  

Everything else will be handled in the background for you!
Sync of values, instantiation of remote entities, adding and removing components.

Well and the best of it...

I'll release this on Github for yall!
Well ok it will be specialized for Photonnetwork for now.

It will take some time until I can release this... Because of finishing all features, bugfixes/ testing and performance test and so one.

But I think this will be a really nice feature!

I also use Protobuf-net for message size reduction and delta compression, network culling and some optimized message sending algorithms.

 

Nice work !
Does each entity has a unique Network ID ?
Did you think of some way to create local entities on client that will then be synchronized if the server also creates a corresponding entity or destroyed otherwise (this can occur when doing client-side prediction of spawned objects) ?

Can't wait to see how the code looks

 

The more I'm thinking about this problem the more it's worrying me though what the performance impact of such a generic solution would be. It's not necessary to sync ALL information across all clients and most games would require a custom solution in any ways. Having a little library that just allows for simpler wrapping/setup of a "network system" would probably suffice. This way any game state that requires synchronisation would be a little more involved, but it'll allow for being a little more in control of the different interpolation/extrapolation strategies in specific cases. It'll be interesting to see how this solution works though, I'm just not sure how much would be gained.

 

Yes, each entity will have a unique Network ID.

No, not at the moment. I don't have this usecase in my game.

Well my solution should be performant in sending number of messages and message size. (My target network platform is photon cloud)

I use a number of optimized algorithms to achive that.
1. Send only those informations which have changed since the last send.
2. Send all messages to the masterclient, collect them and send the result to all other clients -> this reduces the number of messages from N² to 2(N-1)*N
3. Network culling -> send messages to only those entities which are in range (changing update interval for specific groups). The idea is that entities who are far away from your own position won't require to be updated each 100ms because you don't need high precision there.
4. Using of Protobuf which cuts off unnecessary bits. Instead of sending 32 bit it will only send those who represent the number. There is also an optimzation for negative values.


Well performance is an issue, because of the use of reflection, which is as we know slow. I'll sure have to investigate more in the performance section there.

My solution won't sync all components. It will only sync those with the NetSyncAttribute attached to it and only those members with the NetSyncMember attribute as well.

My goal is to provide a greate framework with good performance an optimized messages. At the end it's up to the developer to get the best performance out of the system!

So pls stay tuned

 

Ok, it took longer as expected, but a early alpha version is done.
Hopefully I get it up to github this weekend with some documentation and the excample scene.

This version ins't production ready jet, but you can use it and extend it as you want.
I'll create a new thread for it and hope that you'll enjoy this framework. (I'll write a link to the new thread here)

 

Cool, I'm also working on some basic network stuff and it's good to share some ideas.

 

Ok it's done...

https://github.com/Spy-Shifty/BBSNetworkSystem

A feedback for the doc would be nice! Whether it is readable and easy to understand.
I'll add an example later on...

If you have any questions feel free to ask!

 

I'd suggest using google flatbuffers over protobuf. They were designed for realtime applications where as protobuf was not. Google also claims its much better on memory and the garbage collector.

 

I use protobuf-net it's just like googles protobuf but don't uses googles dll's
https://github.com/mgravell/protobuf-net

But I'll investigate more in flatbuffers

 

Is protobuf-net more efficient on the garbage collector? The official c# bindings for protobuf are pretty hard on it.

 

Ya protobuf-net is highly optimized internally. It also has a Merge method that takes an existing object instance to deserialize into. So if you are only sending value types and use Merge, it's just as good as flatbuffers generally for allocation.

Flatbuffers doesn't have varints which are heavily used for space optimization in games. Protocol buffers have a number of space saving features flatbuffers lack.

In a network context though you still have to pool the streams, no serialization library is going to handle that for you.

 

Flatbuffers was never designed with realtime games in mind. It's actually quite bad there.

 

Thank you for the clarification.

I also updated the readme one more and fixed an error. Mistakenly I've deleted the Serilization attribute from the NetSync component yesterday...

And I did some code cleaning and reordering off files and folders

 

Awesome! I will take a look at this tomorrow
Thank you so much for sharing such precious code

 

Thank you! You find the donate button at the bottom of the readme

I'll provide a demo in the next days

 

I will gladly consider it

Just one "silly" question: you say in the readme that your system handles bool attributes.
Since bool is not blittable and unity removed the bool1 attribute, how do you actually handle bools ?

Thanks !

 

I've created my own bool type

called 'boolean'
take a look at boolean.cs

 

https://google.github.io/flatbuffers/

The developers say otherwise.

 

Flatbuffers was not optimized for wire size. That's directly from the mouth of the author in response to a post I made back in 2015 on their google group.

Their primary design focus was mobile.

 

There are some other options:
https://github.com/neuecc/MessagePack-CSharp (Has some benchmarks that say it's the fastest, also compresses)
https://github.com/neuecc/ZeroFormatter/ (Simular to flatbuffers optimized for C#)

But maybe since we actually have all the memory we need in a nice blittable format probably the most optimized way is to just go unsafe and directly pack the bytes, then if we need some quantization or optimization we make a system that pre/post processes this information before passing down to the serializer?

 

So most don't seem to know what varints are. They are variable length encoded integers using algorithms that are basically the best known general approach for compressing small numbers. You can potentially do better if you are working with very small and very specific ranges, but at that point it's usually a case of diminishing returns.

So protobuf is packing the bytes, in a very efficient manner.

In practice, pooling network streams and pooling your network messages combined with using the Merge api results in 0 per message allocation with protobuf-net.

Plus, runtime messaging in PC games is always low enough so as not to be a cpu usage issue. It's just inherently the type of thing where if you are pushing enough data where serialization is an issue, you have bigger issues.

 

MessagePack by Neuecc has been fast and reliable for me. Comparisons in the readme

 

Hi all,

first of all I've update the BBSNetworkSystem.
I realized that I've forgot to add the protobuf lib. I also fixed some bugs and added a new feature to the SyncMemberAttribute -> InitOnly
This will only sync the data on adding the component. Changes won't be synchronized after initialization.
BBSNetworkSystem

If this isn't enough...
I've added an example project. It's a small shooter.
Just test it and have fun!
BBSNetworkSystemExample

If you have any questions feel free to ask me!

 

Been checking out protobuf.net the last few days you were right runs much better than the official c# port. Are you aware if it they are still compatible across languages? IE can you use protobuf.net on a client and the official protobuf on a c++ server?

 

I want to ask if someone has some feedback for me?

What can I improve or some inspirations maybe?

 

I still didn't have time to try it, but I will give you feedbacks then !
Just from looking at the docs :

- If we have hundreds of different types of network entities, do we have to declare the entityfactory method for each, or is it possible to have a function with the entity's id as a parameter (so that we can call another factory for instance) ?
- How easy is it to handle prediction and rollback with your system ?
- Is it easy to use Unity's networking instead of Photon's ?
- When a new entity is created and sent over the network, does it send all the networked components, or does it use the EntityFactory to only send the instance id and components that are not defaulted by the factory ?

That's all for now

 

Yes, you have to.

You have to do it by your self if needed.
Sync is similar to unet.

Yes, it should be. You only have to implement an interface.

The complete state will be send by the system on creation. The factory methods should be used to initialize non network components.

Sorry for the short answers. I only have my smartphone at the moment... I'm in Hollidays...

 

Update:

I've added some cool new features to the network system.

• You can now synchronize thirdparty components through the network. See ProxyNetSyncAttribute
• You can now set authority to the creating client, the master or the scene. Please take a look at NetworkAuthority and NetworkSync component. Hint: NetworkOwner was replaced by NetworkAuthority
• I also added the ability to synchronize entity references through the network. So if you have a component there a member referes to an entity, than the entity reference will be synchronized through the network, too. The refering entity must also be synchronized through the network. So it needs a NetworkSync component as well. See NetSyncMemberAttribute

Have fun!

And leave me some comments, please.