Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Resolved Innate latency of several dozens of milliseconds when communicating on localhost. Is it normal ?

Discussion in 'Unity Transport' started by Drayanlia, Aug 3, 2023.

  1. Drayanlia

    Drayanlia

    Joined:
    Jan 14, 2021
    Posts:
    11
    I implemented a very basic ping interaction using the Entities package and the Transport package which does the following :
    1. ClientTransportSystem (updates once per frame - framerate set to 60 per second) sends timestamped ping
    2. ServerTransportSystem (updates once per frame - framerate set to 60 per second) receives the ping and send it back
    3. ClientTransportSystem receives the response and computes the round time trip.

    When testing this on the localhost, the resulting time is averaging 66ms (~4 frametimes) with a few 50ms (~3 frametimes) and 81ms(~5 frametimes).

    From my naive understanding, this could be due to :
    • the ConnectionDriver actually taking 1 frame to sends the UDP packet and another 1 frame to acknowledge the packet on the receiving end.
    • adding 1 whole frametime when receiving on each end because of the synchronous nature of my ClientTransportSystem/ServerTransportSystem.

    I'd really appreciate if someone could help me understand what is actually going on or have any lead on how to improve this !
     
  2. simon-lemay-unity

    simon-lemay-unity

    Unity Technologies

    Joined:
    Jul 19, 2021
    Posts:
    441
    I don't know how your code is set up, but one gotcha with the transport package is that it only sends in the jobs scheduled by either
    ScheduleUpdate
    or
    ScheduleFlushSend
    . That is, calling
    EndSend
    doesn't actually get anything on the wire. It basically just queues the packet for sending, and the actual socket operations will be performed in the job.

    So assuming your code basically does something like this every frame:
    1. Schedule an update of the driver.
    2. Process events and send messages.
    Once you send your ping it would only be actually sent on the next frame (so up to one frame of delay). Then the server will receive it at the beginning of its next frame, so again up to another frame of delay. It then receives the ping and sends its response, but again the actual send will only occur on the next frame. So up to another frame of delay. The client will then receive the response at the beginning of its next frame, incurring up to another frame of delay. That's (if you're unlucky) up to 4 frames between the initial send (call to
    EndSend
    ) and receiving the response.

    To improve this, you could schedule a send job with
    ScheduleFlushSend
    after you've processed events and sent messages. This should get your messages on the wire faster and improve latency. For example, Netcode for Entities will only schedule a single update job per frame, but will schedule send jobs at multiple points during a frame. The send job has been written to be relatively lightweight to allow these kinds of uses.
     
    Drayanlia likes this.
  3. Drayanlia

    Drayanlia

    Joined:
    Jan 14, 2021
    Posts:
    11
    Thanks for the response Simon. It is quite insightful.

    I'm already scheduling an update of the driver at the end of my systems OnUpdate method (I don't use job since the OnUpdate already use the [BurstCompile] attribute). I'm also using a NativeQueue to schedule the outgoing messages. It looks like this for my client :

    Code (CSharp):
    1.     [BurstCompile]
    2.     public void OnUpdate(ref SystemState state)
    3.     {
    4.         ref var driver = ref networkDriver.Data;
    5.         ref var connection = ref clientConnection.Data;
    6.  
    7.         if (!driver.IsCreated || !connection.IsCreated)
    8.         {
    9.             return;
    10.         }
    11.  
    12.         // Ping request
    13.         if (Time.realtimeSinceStartup - lastPingRequestTime > PingInterval)
    14.         {
    15.             if (TrySendPingRequest(ref state))
    16.             {
    17.                 lastPingRequestTime = Time.realtimeSinceStartup;
    18.             }
    19.         }
    20.  
    21.         // Send messages
    22.         while (outgoingMessages.Data.TryDequeue(out OutgoingMessage msg))
    23.         {
    24.             switch (msg.Channel)
    25.             {
    26.                 case NetworkChannel.Unreliable:
    27.                     {
    28.                         driver.BeginSend(connection, out var writer);
    29.                         writer.WriteBytes(msg.Payload);
    30.                         driver.EndSend(writer);
    31.                     }
    32.                     break;
    33.                 case NetworkChannel.ReliableSequenced:
    34.                     {
    35.                         driver.BeginSend(reliablePipeline, connection, out var writer);
    36.                         writer.WriteBytes(msg.Payload);
    37.                         driver.EndSend(writer);
    38.                     }
    39.                     break;
    40.                 default:
    41.                     break;
    42.             }
    43.  
    44.             msg.Dispose();
    45.         }
    46.  
    47.         // Read messages
    48.  
    49.         clientJobHandle = driver.ScheduleUpdate();
    50.         clientJobHandle.Complete();
    51.     }
    The server code is quite similar. Just handling multiple NetworkConnections.

    From what you said all the messages should be flushed by the time the clientJobHandle completes. Knowing this I would not expect the kind of latency I'm experimenting. It should be at max 2 frames if I am unlucky as you said.

    I didn't take time to really dig up the NetCode for Entities samples though. Maybe I should start looking how things are set up there.

    Edit : As I read my reply I noticed that I'm actually sending the outgoing messages before handling the received ones, which is not how I designed it. I checked my server code and the sending is done after. This does not impact the ping time since my client is not responding to anything for this specific interaction but it does affect the reactiveness of my client for other actions.
     
    Last edited: Aug 3, 2023
  4. simon-lemay-unity

    simon-lemay-unity

    Unity Technologies

    Joined:
    Jul 19, 2021
    Posts:
    441
    When is the processing of incoming packets occurring here? If it's done right before the
    ScheduleUpdate
    call, then there's going to be at least a full frame of delay between a packet being received and it being processed.

    The reason for this is that receives are similar to sends: we only touch the socket in a job. For receives that only happens in the
    ScheduleUpdate
    job however. The job basically pulls from the socket and puts the packets in a queue, and the data events are then drawn from that queue. So assuming that the processing of received packets happens right before the
    ScheduleUpdate
    call, here's what's going to happen:
    1. A packet is received while (say) checking the outgoing message queue.
    2. The code that processes new events will not see that packet since we haven't pulled from the socket yet.
    3. The
      ScheduleUpdate
      job is executed, pulls the received packet from the socket, and puts it in some queue.
    4. In the next frame, the data event is popped from the driver and the packet is processed.
    So basically there's an extra full frame of delay added to the receive direction.

    Ideally, data events would be processed immediately after the
    ScheduleUpdate
    job completes to reduce latency. And then once events are processed, which could have caused new packets to be sent, a send job would be scheduled to immediately send the responses. And if the processing of events is jobified, the whole thing can be scheduled as a chain of jobs, moving the entire network processing off the main thread.
     
    Drayanlia likes this.
  5. Drayanlia

    Drayanlia

    Joined:
    Jan 14, 2021
    Posts:
    11
    Somehow I completely missed that part ! That should solve the problem.

    Yes, that's something I'll have to do. I'm still figuring out how things work using the Entities package.

    Thank you again for your help. It is greatly appreciated !