Resolved Some newbie questions regarding Unity Transport

In my current project I'm using NGO and custom messages for communication between server and client. I need to do away with this high level solution so I'm looking to use Unity Transport instead. I'm just feeling my away along right now and have a few questions regarding Unity Transport and the Jobs system.

I couldn't see instructions on how to stop a running server, is with driver.Dispose()? If clients are connected presumably I need to wait until their disconnect jobs are complete before disposing of the driver?

For serialised messages in NGO I've been using FastBufferReader/Writer and serialising the individual fields with Read/WriteValueSafe. I'm struggling to see what the best approach would be using Unity Transport and have the sending of messages be part of a job. Is it still possible to follow this approach or is there a better way to do this?

Looking in the example code for ServerUpdateConnectionsJob it has this code:

Code (CSharp):

1.         // CleanUpConnections
2.         for (int i = 0; i < connections.Length; i++)
3.         {
4.             if (!connections[i].IsCreated)
5.             {
6.                 connections.RemoveAtSwapBack(i);
7.                 --i;
8.             }
9.         }

This seems to swap the current position with the last, and if so does that mean the last connection is checked twice, or is the last element removed and if so is connections.Length reduced during the for loop?

That's all the questions for now.

 

That's pretty much what I was expecting, I'll use a flag or state to keep track or it.

The example is very similar to how it works in NGO, from what you've said and what I've seen it looks like I won't have to modify my code too much which is a bonus as the message system is fairly elaborate and I didn't want to have to rewrite it all.

What I have currently is a service that has a call SendMessage(ulong clientId, IMessage message) that kicks off the serialisation and sending of the message. So in theory I can create a job passing in something very similar and inside the job use the DataStreamWriter from BeginSend and pass that along in a similar fashion to FastBufferWriter for the serialisation. I'll have a play with it and see how I get on. This is quite a shift from what I'm currently used to, I've not had to use Update at all apart from running timers.

That makes sense, I wanted to check as that's the first time I've seen that behaviour.

Many thanks for the answers Simon, I think I've enough understanding to make a proper start.

 

Hmm okay, something else to think about. I didn't get a chance to look at it yet but I did look at stopping the server and start on some way of triggering events based upon network events. I hope you don't mind if I run some code passed you in case I'm missing something I can make use of or my ideas are flawed.

For stopping the server I'm calling disconnect for each connection then setting a state to track where I am. This is the scheduling code called in Update:

Code (CSharp):

1.     private void ScheduleByState()
2.     {
3.         switch (networkState)
4.         {
5.             case ServerNetworkState.Started:
6.                 serverJobHandle = networkDriver.ScheduleUpdate();
7.                 serverJobHandle = connectionsJob.Schedule(serverJobHandle);
8.                 serverJobHandle = updateJob.Schedule(networkConnections, 1, serverJobHandle);
9.                 break;
10.  
11.             case ServerNetworkState.ShuttingdownStart:
12.                 Debug.Log("ShuttingdownStart");
13.                 serverJobHandle = networkDriver.ScheduleUpdate();
14.                 networkState = ServerNetworkState.ShuttingdownEnd;
15.                 break;
16.  
17.             case ServerNetworkState.ShuttingdownEnd:
18.                 Debug.Log("ShuttingdownEnd");
19.                 networkDriver.Dispose();
20.                 networkState = ServerNetworkState.Shutdown;
21.                 break;
22.         }
23.     }

This appears to work fine, at least I'm not getting the usual Dispose related errors.

For network event handling I'm not sure on the best course of action. What I've done is have each NetworkConnection as part of a struct with an event type I can act on after a network event has occurred.

Code (CSharp):

1. public struct ServerNetworkConnection
2. {
3.     NetworkConnection connection;
4.     ServerEventType eventType;
5.  
6.     public ServerNetworkConnection(NetworkConnection connection, ServerEventType eventType = ServerEventType.None)
7.     {
8.         this.connection = connection;
9.         this.eventType = eventType;
10.     }
11.  
12.     public NetworkConnection Connection { get => connection; }
13.     public ServerEventType EventType { get => eventType; set => eventType = value; }
14. }

And the update jobs Execute code (this is just prelimary and only Disconnect does anything):

Code (CSharp):

1.     public void Execute(int index)
2.     {
3.         NetworkConnection connection = connections[index].Connection;
4.  
5.         if (connections[index].Connection.IsCreated)
6.         {
7.             NetworkEvent.Type eventType = driver.PopEventForConnection(connection, out DataStreamReader reader);
8.  
9.             switch (eventType)
10.             {
11.                 case NetworkEvent.Type.Empty:
12.                     // do nothing
13.                     break;
14.  
15.                 case NetworkEvent.Type.Data:
16.                     Debug.Log("ServerUpdateJob Execute Data: " + reader.Length);
17.                     break;
18.  
19.                 case NetworkEvent.Type.Disconnect:
20.                     Debug.Log("ServerUpdateJob Execute Disconnect: " + reader.Length);
21.                     connections[index] = new ServerNetworkConnection(connection, ServerEventType.Disconnect);
22.                     break;
23.  
24.                 default:
25.                     Debug.LogWarning("ServerUpdateJob unhandled eventType: " + eventType);
26.                     break;
27.  
28.             }
29.         }
30.     }

Thie idea is to use the eventType once outside the job to invoke an Action like OnClientDisconnected. The problem with this approach is when I want to set the eventType (and later clear it) I have to create a new struct each time which isn't ideal.

I had some other quick questions.

Is NetworkEvent.Type.Connect client side only?

In the example code they use a while loop to check for events:

while ((cmd = driver.PopEventForConnection(connections[index], out stream)) != NetworkEvent.Type.Empty)

is there a particular reason for this?

On a Data event is it possible to pass out the DataStreamReader to process the message outside the job, or should the message be read inside it?

Thanks for any guidance you can give.

 

I see, plenty more food for thought there.

Perhaps I won't try parcel things up into a single struct, I'm struggling to work out how to get a struct out of the job with information I need. I need a better grounding in the Jobs system for sure so I'll do some more research.

Thanks again Simon!

 

I followed your advice and corrected my code and implemented NativeQueue.AsParallelWriter and with a bit of trial and error and ChatGPT trying to help I was able to the data out of the job as I needed. Here's the revised struct:

Code (CSharp):

1. public struct ServerConnectionEvent
2. {
3.     int connectionId;
4.     ServerEventType eventType;
5.     DataStreamReader streamReader;
6.  
7.     public ServerConnectionEvent(int connectionId, ServerEventType eventType = ServerEventType.None,
8.         DataStreamReader streamReader = default)
9.     {
10.         this.connectionId = connectionId;
11.         this.eventType = eventType;
12.         this.streamReader = streamReader;
13.     }
14.  
15.     public int ConnectionId { get => connectionId; }
16.     public ServerEventType EventType { get => eventType; }
17.     public DataStreamReader StreamReader { get => streamReader; }
18. }

In all it's a better solution to what I was trying to make work and I should be able to make better progress with this part out of the way (not that it's properly tested mind you).

Thanks again for your help.

 

So I've been able to send a message from client to server. When the client gets a message to send it's added to a NativeList and in Update the messages are sent in parallel. The only sticky part is tracking when to know I can Clear that list after the messages are sent, I'm currently doing with a separate State field.

One difference I've noticed between NGO's FastBufferWriter and DataStreamWriter is that FastBufferWriter can be passed by value and retains its original writer buffer, with DataStreamWriter it has to be sent by reference otherwise it ends up with a new write buffer. Incidentally through testing with this if an empty message is sent to the server it triggers the Data event but there appears to be have no read buffer so I'm checking reader.IsCreated to guard against this.

One major issue I have is I can't use my current message structure. What I have in NGO is interface IMessage which is implemented by an abstract class ClientMessage/ServerMessage which is inherited by other message classes with have their own interface IMessageDetail field for other classes that make up the various message body types. This won't work at all and I'm not sure what's a good approach to go with, if you have any suggestions I'm all ears.

 

Where I have NativeList<ClientMessage> clientMessages I'm calling clientMessages.Clear() once those messages have been sent. Originally I wasn't using a list and was just scheduling the job immediately, but thought a list might be the better option?

That sounds good to me. I just wasn't aware there wouldn't be a read buffer so didn't expect the exceptions being thrown, with reader.Length for example as I thought it would return zero.

I created a simple message:

Code (CSharp):

1. public struct ClientMessage
2. {
3.     int value;
4.  
5.     public ClientMessage(int value)
6.     {
7.         this.value = value;
8.     }
9.  
10.     public void DeSerialize(DataStreamReader reader)
11.     {
12.         value = reader.ReadInt();
13.     }
14.  
15.     public void Serialize(DataStreamWriter writer)
16.     {
17.         bool result = writer.WriteInt(value);
18.     }
19.  
20.     public override string ToString()
21.     {
22.         StringBuilder stringBuilder = new StringBuilder("ClientMessage");
23.         stringBuilder.Append(" value ").Append(value);
24.  
25.         return stringBuilder.ToString();
26.     }
27. }

In Serialize it looks like the writer is using its own write buffer as when Serialize returns and I call EndSend with the original writer returned by BeginSend the EndSend returns zero. If I have the Serialize call return its writer and call EndSend with that the data is sent. I've gone with sending in the original writer by reference to Serialize instead as the better option.

It is the problem the managed types, or rather it's a problem of coming up with the best alternative understanding the limitations. I could serialise the messages before adding them for sending within a job, but then I'd lose half the benefit of handling them within a job?

 

The simplest approach does have a certain appeal . I'm not expecting to handle a huge amount of network traffic and Jobs may be overkill for what I'm doing at the moment but it's interesting to learn and makes things more scalable.

Thanks for the example, I'll give it some thought and see what I can come up with.

 

I've been playing with this and decided for now to handle the serialisation and deserialisation outside of jobs. There's too many messages currently to handle individually so I'll look at amalgamating some of them together.

For the serialisation I'm using a new DataStreamWriter for each message then storing that writer in NativeQueue<DataStreamWriter> which in Update is scheduled in a job. This appears to work but inside the job the writers appear empty which I assume is due to the same problem you've outlined previously. Is there an alternative way of doing this for now? I think I'm missing a trick as everything I've tried so far results in an error.

Incidentally sending DataStreamReader out of the job looks to be working fine.

 

As a workaround for now I'm calling writer.AsNativeArray().ToArray() to get the serialised message (I can't use AsNativeArray directly due to its allocation type) and adding it to a job with the list of jobs scheduled in Update.

I've not been able to work out how to get an array of the serialised message arrays in a job, or an array of structs with each containing an array although as an array of DataStreamWriters does work there must be a trick to it.

 

I did wonder if using DataStreamWriters this way was a good idea. I'll look into keeping it simple and as you suggest serialise and send in the main thread.

My code is not far removed from your test code and I see now I'm getting the same result as you. I was using a different constructor DataStreamWriter(256, Allocator.Temp) which wasn't allowing me to call writer.Length as it was WriteOnly.

Actually now I can check the length I can see the problem, I'm passing the dequeued writer into the out of BeginSend and the writer length is now zero. Here's the job code:

Code (CSharp):

1. public struct ClientSendJob : IJob
2. {
3.     public NetworkDriver networkDriver;
4.     public NativeArray<NetworkConnection> connection;
5.     public NativeQueue<DataStreamWriter> writers;
6.  
7.     public ClientSendJob(NetworkDriver networkDriver, NativeArray<NetworkConnection> connection, NativeQueue<DataStreamWriter> writers)
8.     {
9.         this.networkDriver = networkDriver;
10.         this.connection = connection;
11.         this.writers = writers;
12.     }
13.  
14.     public void Execute()
15.     {
16.         while (writers.TryDequeue(out var writer))
17.         {
18.              Debug.Log("ClientSendJob Execute length: " + writer.Length);
19.             if (networkDriver.BeginSend(connection[0], out writer) == 0)
20.             {
21.                 Debug.Log("ClientSendJob Execute BeginSend length: " + writer.Length);
22.                 int result = networkDriver.EndSend(writer);
23.  
24.                 Debug.Log("ClientSendJob Execute result: " + result);
25.             }
26.         }
27.     }
28. }

I guess passing in a writer rather than using the one provided by BeginSend is a bad idea?

 

Got you. I'll look into going the conventional route and reduce the number of messages and serialise with the provided writer.

I'd looked into this previously but due to the setup of the project, where it's potentially an asset used by other developers I was passing out the message directly and didn't want to have messages with lots of fields where only some are relevant to a particular message type. I'll have to put in an extra layer to simplify the output.

Thanks for letting me know.