Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

NetworkTransport a way to clear queue?

Discussion in 'Multiplayer' started by munkbusiness, Sep 4, 2017.

  1. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    Hi

    I have some bare bones networking fucntionality running, where I use one client to send positions and the the server just shows it. It usually works fine, but sometimes the servers starts lagging behind. By showing the queue with NetworkTransport.GetIncomingMessageQueueSize() I can see that the lag starts when the queue goes from the 0-2 that it usually resides in, to 9+. I am using the unreliable channel as I don't really care about order or if some packets are lost, I just want the newest position as fast as possible.

    Is there a way to ignore or empty the queue, I don't care for old positions, only the newest one. I can't really find anything. I am using NetworkTransport.
     
  2. Driiades

    Driiades

    Joined:
    Oct 27, 2015
    Posts:
    151
    If you clear the queue... you will not have the lastest message sent. You will have the lastest message received. It's different.

    Send a timeStamp with the message and store only the lastest based on this timestamp :)
     
  3. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    For MessageUpdates. You probably want to use StateUpdate as your QOS. In addition. You might want to call Transport.Recieve more than once per frame. I call it until the queue is empty but I also have a max amount of messages to process per frame so we don't freeze on one frame for too long.
     
  4. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    Makes sense that with unreliable I don't get ordering correctly, thanks.

    For your solution how do I iterate through the queue? In NetworkTransport I cannot find any methods for manipulating the queue. I think what you mean is to iterate through the queue and only act on the one with the most recent timestamp.

    Will take a look at stateUpdate, from the description it seems like that is exactly what I want, but when I test it my queueSize just rapidly increases? Shouldn't I deal with that?

    For calling more than once per frame, are you using a seperate thread? I thought about that, but since my visuals will only update on frame, in my mind there was no reason to read more that one message per frame. Or do you just loop until the the queue has a size of 1 and use that. From Driiades's post it seems like that won't necessarily get me the latest one, and how this this work with stateUpdate to empty the queue?
     
  5. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Well, if you try to send a ton of messages with the State Update QOS. It will still fill the queue. But once it's time to send them away. It only takes the last one. So I guess only send once every so often.

    And personally I iterate though all messages in each update. In the Main thread.Usually you will have game logic to be ran in the Update method. So if you can process multiple messages between each logic thats great. Usually you want to empty the queue.
     
  6. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    What I mean is that the server who is receiving the messages, gets growing incoming queue, which confuses me as the only thing sending to it is a single client using state update. Is it because state update only receive the most recent, but still leaves everything else in the incoming queue?

    Can you give a code example of how to process multiple messages in a single update.

    Is it like this?

    Code (csharp):
    1.  
    2. do {
    3.    NetworkEventType recData = NetworkTransport.Receive(out recHostId, out connectionId, out channelId, recBuffer, bufferSize, out dataSize, out error);
    4. } while { NetworkTransport.GetIncomingMessageQueueSize(recHostId, out error) > 1 }
    5.  
    And how would that go together with state update?
     
  7. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Code (CSharp):
    1. int messagesProcessed = 0;
    2. do
    3.         {
    4.             networkEvent = NetworkTransport.Receive(out recHostId, out connectionId, out channelId, recBuffer, ServerSettings.singleton.MaxMessageSize, out recSize, out error);
    5.             switch (networkEvent)
    6.             {
    7.                 case NetworkEventType.Nothing:
    8.                     break;
    9.                 case NetworkEventType.ConnectEvent:
    10.                     throw new NotImplementedException();
    11.                     break;
    12.                 case NetworkEventType.DataEvent:
    13.                     throw new NotImplementedException();
    14.                     break;
    15.                 case NetworkEventType.DisconnectEvent:
    16.                     throw new NotImplementedException();
    17.                     break;
    18.             }
    19.             messagesProcessed++;
    20.         } while (networkEvent != NetworkEventType.Nothing &&
    21.                     ServerSettings.singleton.ProcessMultipleRequestsPerTick &&
    22.                     (messagesProcessed == -1 || messagesProcessed < ServerSettings.singleton.MaxMessagesToProcessPerTick));
    In your Update loop. As for how the QOS is handled on the Recieving end, I am actually unsure of. All the documentation states is that we only recieve the latest one. Not sure how that would work tho. @aabramychev is probably the one to answer that one.

    The maxMessageProcessed is just there if the queue is totally full the game doesnt just freeze.

    My above code simply Polls the queue until we get "Nothing" OR until our limit is reached (If enabled)
     
    Last edited: Sep 4, 2017
  8. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    for state update:
    every time when you send state update message you just exchange current state update message waiting to be sent:
    Code (CSharp):
    1. Send(msg):
    2. if( m_CurMessage != null )
    3.   remove(m_CurMessage);
    4. m_CurMessage = msg;
    the same picture on receiving part, channel has one member variable receiving message and every new message which come in will change this variable.
     
  9. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Aah. So if I add a state update message to the queue every frame. The queue should still just have a size of 1?
     
  10. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    Sorry I am a bit confused. Are you saying that is how state update work, or something I should add to my code when using state update? In your code snippet, how do I get those variables, current message and remove. I cannot find anywhere where I can manipulate the sending queue from networkTransport.

    If you mean that i how it just works, that means I should just ignore the growing incommingqueue? with 600+ message in queue after a minute.
     
  11. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Thats how state update works internally. Aabramachev is the LLAPI dev.
     
  12. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    So the conclusion is that I just ignore the massively building incoming queue? Doesn't it block my other channels, if I want a reliable message also for example?
     
  13. TwoTen

    TwoTen

    Joined:
    May 25, 2016
    Posts:
    1,168
    Can't answer that for you. From my understanding of alex's statement the queue should not grow.
     
  14. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    No it won't block any reliable messages.
    > Aah. So if I add a state update message to the queue every frame. The queue should still just have a size of 1?
    yes, should be 1
     
  15. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    Are you referring to outgoing or Incoming queue?

    On my client which is sending the stateupdate i get a constant queueSize of 2 with this code. (VDebug just writes to Debug.log and prints on my screen)

    Code (csharp):
    1. int queueSize = NetworkTransport.GetOutgoingMessageQueueSize(hostConnId, out error);
    2. VDebug("Queue size: "+queueSize+", Sending: " + message, false);
    On my server my queue from only one client and only stateUpdates being sent I have an increasing queue growing with approx 15 every second. In my dataEvent i have this:

    Code (csharp):
    1. int queueSize = NetworkTransport.GetIncomingMessageQueueSize(recHostId, out error);
    2. VDebug("Queuesize: "+queueSize+", Receiving: " + msg + " from: " + connectionId, false);
    In my client I am sending a stateupdate once per. frame/update. and On my server I am checking for message once per frame/update.
     
    Last edited: Sep 7, 2017
  16. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    @munkbusiness, Difficult to say, need to see the code :(, report the bug please. I think that it is not very urgent task? Let's our engineers check it first.

    Sorry, that I cannot jump on this immediately :(
     
  17. HiddenMonk

    HiddenMonk

    Joined:
    Dec 19, 2014
    Posts:
    987
    Would it be possible in the future that we get the ability to clear the Outgoing and Incoming NetworkTransport queue, as well as the Ack buffer?
    My use case is, if I detect that a connection doesnt seem to be receiving any of my reliable messages and the ack buffer is almost full, I basically want to put that connection into an "unready" state. I would clear all incoming and outgoing messages for that connection as if I disconnected from them, I would also clear the ack buffer since I no longer care about those reliable messages reaching them, and will then send a "ResetConnection" reliable message and wait for their reply.
    This way I dont have to go through the whole disconnect connection and reconnect process, destroying all their objects and what not.
    I might have a long DisconnectTimeout, such as 20 seconds (unity sets default to 2 seconds), so it is possible that I might have sent to many reliable messages during those 20 seconds, which would cause problems as reliable packets start to not send since the ack buffer is full.
    If you think 20 second DisconnectTimeout is to long, I think Team Fortress 2 has a 20 or 30 second DisconnectTimeout, and there are plenty of times where I lose connection for 5, 10, 15 seconds or so before finally reconnecting (I can see on my windows wireless network connection icon that my internet is having issues, so it seems to be my connection and not the game). So having it not disconnect me from the game during those 20 seconds is really nice.


    From memory, I think I also saw my OutgoingQueueSize being a size of 2 when using NetworkTransport.QueueMessageForSending. When using regular NetworkTransport.Send, it seems, at least on localhost, that StateUpdate gets sent instantly regardless of your connectionconfig update rate, and ReliableStateUpdate just keeps increasing the OutgoingQueueSize. However, with NetworkTransport.QueueMessageForSending, both seem to work fine, but they do queue up 2 and not 1 for some reason.
     
  18. munkbusiness

    munkbusiness

    Joined:
    Aug 22, 2017
    Posts:
    55
    Hmm I am using just the send method. I can take a look at the QueueMessageForSending, but I don't see why I would do that for a stateupdate.

    @aabramychev It might be a bug, but it could very much be me doing something wrong somewhere else in my code. How do I report a bug?
     
  19. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    @munkbusiness "On my client which is sending the stateupdate i get a constant queueSize of 2 with this code. (VDebug just writes to Debug.log and prints on my screen)" it is correct
    But for server, not sure, it should be ~2 too if you send only stateupdate messages. So, yes it is probably bug, report them. As a problem is not urgent i guess, the bug reporting will be better here :)

    @HiddenMonk, not sure. As You have already said, you can always call connect/disconnect then, +I will open (which was hidden before) send callback, where library will notify you when you can call send again. I'm not say not, but now I can not foresee any side effects of this feature:( If you already have a proper design in your head you are welcome PM me :) Make sense?