Search Unity

Resolved Mutiple ErrorCode: -5 related warnings

Discussion in 'NetCode for ECS' started by optimise, Apr 23, 2021.

  1. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    When I hit multiple warnings at Android build, my game will become super lag like 10+ fps. I'm not sure the exact step to reproduce the bug but it's quite frequent to trigger this bug randonly. Will this bug be fixed in next version? Below are the logged warnings

    1) An error occured during EndSend. ErrorCode: -5
    2) ProcessPiplineSend failed with the following error code -5.
    3) CompleteSend failed with the following error code: -5
     
    Last edited: Apr 23, 2021
    Lukas_Kastern likes this.
  2. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    I think -5 is send queue full, what's the callstack to the calls that fail? I have not seen that issue before so i don't think we have a fix for it, can you please report a bug for this so we can investigate?
    A potential workaround until we fix it would be to increase the send queue size, you can pass a `BaselibNetworkParameter` to specify the queue sizes when creating the network driver (you can override driver creation by assigning `NetworkStreamReceiveSystem.s_DriverConstructor` in the bootstrap)
     
  3. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    @timjohansson, @CMarastoni. You can refer to case 1333309 repro project for now. Let me know if u need latest repro project. For my latest update of the project seems like ErrorCode error didn't come out anymore but still have suddenly become extremely low fps issue.
     
  4. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Hi @CMarastoni @timjohansson. Have u found root cause of this issue? If yes, which unity transport version fix this issue? It seems like it's much easier reproduce this issue when you have higher number of ghosts. If until now you still not able to reproduce it I will try to submit new repro project.
     
  5. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Hi @timjohansson @CMarastoni. I tried to directly modify the dots netcode package to set sendQueueCapacity to 96 and rest is default for both client and server system but I still get error code -5 spamming.
    var baselibNetworkParams = new BaselibNetworkParameter {receiveQueueCapacity = 64, sendQueueCapacity = 96, maximumPayloadSize = NetworkParameterConstants.MTU};
    I believe it's caused by transport package didn't implemented properly and really needs more attentions and improvements. Although this issue can be fixed by player just quit and reconnect the game but it gives bad experience to player.
     
    Lukas_Kastern likes this.
  6. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    The receive and send queue capacity is how many packets you can receive and send in single frame. If you try to send more than that number of packets you'll get the -5 error, if you receive more packets will be dropped. It is up to the user of transport to make sure those queues are large enough.

    The current default values in netcode are not large enough to handle a lot of clients connected to the same server and needs to be increased significantly on the server side (but client side should be ok).

    If you are seeing this error on the server when you only have a single or a handful of clients - or seeing it on the client - I'm pretty sure it is caused by sending more RPCs than we can handle. You get the same error if the reliable send window (which cannot be larger than 32) is full and that seems like the most likely cause if you are sending a lot of RPCs or have really high ping.

    I would suggest looking at how many RPCs you send and how frequently. I have filed a task to expose more statistics on that to make RPCs easier to analyze and track, but I am not sure when that will land.
     
  7. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    @timjohansson I get -5 error at client. It's not 100% reproducible every time. I guess server dun have -5 error but I will verify after that.

    I believe I only super few RPCs just for initialize players. Then after that only use ghost for the whole game. Or it's actually caused by ghost that has component with too many data marked as GhostField?
     
  8. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    Since the error is happening on the client and the client is not sending any ghost data - it's only sending commands and RPCs - it should not be related to GhostFields. In general, the server will split snapshots into multiple packets if it cannot fit a single entity into a packet of the default size - but you need a lot of ghost fields for that to happen. The client commands does not support fragmentation so the commands will only be one packet per frame. So from the client the only thing we can send multiple of is RPC packets.

    How long it takes the OS to actually send the packet and return the packet to the "free" send queue depends on the platform though, that could take more than one frame.
    Do you know if the errors from the CommandSendSystem or the RPCSendSystem? Have you seen it on other platforms or only android? Are you using wifi or not when it is happening?
     
  9. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    Not sure. The error doesn't tell me which system coming from. Most probably it's directly from transport I dunno. Maybe because it's not debug build? I will try to reproduce the bug again. Currently I only see it at Android but for iOS I dun have device to test it. I'm using wifi.

    Edit: So, I think can conclude that there's something wrong at Transport for Android platform?
     
    Last edited: Dec 13, 2021
  10. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    2,129
    @timjohansson Btw can u make sure the next release of dots netcode in dots 0.5 release will have latest version of transport as dependency? Currently at the time of writing the transport version is 1.0.0.pre-10.

    For my testing yesterday I found that even error code -5 is not spamming, my game will also have unstable issue. But of course when there's error code -5 it will become super unstable. After restart and reconnect the game back to the same game server. The game becomes super smooth again.
     
    desertGhost_ and Lukas_Kastern like this.
  11. Lukas_Kastern

    Lukas_Kastern

    Joined:
    Aug 31, 2018
    Posts:
    97
    On that note, I would like to hijack this thread to ask how NetCode will work together with the new relay service that the transport package seems to support now. Will host migration be handled automatically for us or is this something we need to build ourselves?
     
  12. timjohansson

    timjohansson

    Unity Technologies

    Joined:
    Jul 13, 2016
    Posts:
    473
    It is certainly a possibility and sounds likely, but it could also be some system causing more RPCs than expected to be sent.

    I can't guarantee that the dependency for 0.50 will be the absolute latest at the time of release - but we are aiming to keep it as up to date as we can.

    We are not planning to do any automatic host migration in 0.5 or 1.0.
     
    Lukas_Kastern and optimise like this.