Search Unity

Multi-thread server

Discussion in 'Multiplayer' started by iShadowfax, Jun 10, 2015.

  1. iShadowfax

    iShadowfax

    Joined:
    Mar 18, 2015
    Posts:
    38
    Please explain how does server works if all Unity runs in one thread?
    At a high number of connections in case of using one thread requests will be delayed by processor?
     
    iShadowfax, Jun 10, 2015
    #1

  2. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    network part is multi-threaded to reduce time to handle sending messages when you have a lot of connections.
     
    aabramychev, Jun 11, 2015
    #2

  3. seanr

    seanr

    Unity Technologies

    Joined:
    Sep 22, 2014
    Posts:
    669
    Script Callbacks for network events all happen in the Unity main thread.
     
    Last edited: Jun 11, 2015
    seanr, Jun 11, 2015
    #3

  4. iShadowfax

    iShadowfax

    Joined:
    Mar 18, 2015
    Posts:
    38
    So Unity networking is single threaded? Then how can it handle 100+ connections?
     
    iShadowfax, Jun 11, 2015
    #4

  5. AndreyPakhomov

    AndreyPakhomov

    Joined:
    Mar 27, 2014
    Posts:
    6
    I join to a question. How to create high perfomance server to handle several thousands of connections? Or it impossible and UNET designed to only work with Multiplayer Service without server logic?
     
    AndreyPakhomov, Jun 12, 2015
    #5

  6. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Unity Networking is being rolled out in phases, this is the first phase which handles common game requirements. I would imagine it's ideal for peer to peer games where you're having 2-32 players. Future milestones (check blog) will add more functionality, scaling all the way up to MMO someday.
     
    hippocoder, Jun 12, 2015
    #6

  7. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    unet library contains two different part; first part is working in main unity thread. the second part working other thread(s). When you send message you actually put message in the send queue which will utilized by network thread(s). When you check is message received, you actually check the received queue. Both operation is o(1) complexity for user.

    For client library this network thread which utilized all network work is one. For server library it will be pool.
     
    aabramychev, Jun 12, 2015
    #7
    akuno likes this.

  8. iShadowfax

    iShadowfax

    Joined:
    Mar 18, 2015
    Posts:
    38
    So then what is about thread safety? DO I need to lock concurrent collections on server?
     
    iShadowfax, Jun 12, 2015
    #8
    AlwaysBeCoding247 likes this.

  9. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    Sorry do not understand you question. UNET expects that there is only one user thread, so function Send() and PopData() are not thread safe. If you use more than one thread - you should create your lock policy I guess :)
     
    aabramychev, Jun 13, 2015
    #9

  10. iShadowfax

    iShadowfax

    Joined:
    Mar 18, 2015
    Posts:
    38
    I mean, if a peer connects to the server does server create a new thread for each peer? If not than it means that in 500+ peers connected and sending 20+ requests each it would be laggy... Such servers as photon and smartfox, use one thread for each peer.

    PS photon use phibers for each peer, but but whats about UNET
     
    iShadowfax, Jun 13, 2015
    #10

  11. seanr

    seanr

    Unity Technologies

    Joined:
    Sep 22, 2014
    Posts:
    669
    No. In 5.1 in UNet there is one network thread, then script callbacks happen in the Unity main thread. This may change in future versions.
     
    seanr, Jun 13, 2015
    #11

  12. iShadowfax

    iShadowfax

    Joined:
    Mar 18, 2015
    Posts:
    38
    Is there any formula to count how many connections + number of requests it is comfortable for 1 thread?
     
    iShadowfax, Jun 13, 2015
    #12

  13. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    >I mean, if a peer connects to the server does server create a new thread for each peer? If not than it means that in 500+ peers connected and sending 20+ requests each it would be laggy... Such servers as photon and smartfox, use one thread for each peer.

    Gothca, Sean has been already answered.
    Performance: Difficult to say, but I have some digits in my pocket. I did stresses following configuration:
    I created echo server just as unity game (but without gui and any rendering/physics stuff inside)
    0. Size of the packet whole packet ~ 50bytes, all messages are reliable
    1. Echo server working on one machine, and multiply echo clients working on another, connection WiFi 11g
    2.Echo Server on Win, Android (Samsung S4) iPad 3rd generation.

    For Server on win combination it was ~2000 pkt per sec ~800kBit (it is ~15% of full allowable bandwidth) per sec and server fps rate was stable 60fps... so I can conclude that maximum value should be significantly greater.
    For Server on android it was ~1600 pkt per sec.

    Take into account
    - that for computer itself there is not significant difference to handling 50 or 500 bytes packets (all computation are the same + two memcpy call per packet)
    -it was a goal to compare client library performance on mobile devices vs pc standalone client
    -all above mentioned digits described full stack - user create message call send, library delivers message to other peer, echo server receives this message an replay them back...

    it is what I have for this library
     
    aabramychev, Jun 13, 2015
    #13

  14. iShadowfax

    iShadowfax

    Joined:
    Mar 18, 2015
    Posts:
    38
    The problem is that you test it sending packets from 1 client. If you try 100 clients and 1 thread you'll notice that some clients experience a heavy delay cause one thread is waiting for whole 2000 packets to be processed
     
    iShadowfax, Jun 14, 2015
    #14

  15. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    >The problem is that you test it sending packets from 1 client. If you try 100 clients and 1 thread you'll notice that some clients experience a heavy delay cause one thread is waiting for whole 2000 packets to be processed


    With fps = 60 and one message per frame it is impossible to reach 2000 packet per sec... For sure it was about 50 clients...
     
    aabramychev, Jun 16, 2015
    #15

  16. g_a_p

    g_a_p

    Joined:
    Mar 16, 2015
    Posts:
    281
    No, SmartFoxServer does not run one thread-per-peer.
    Typically you can run tens of 1000s of users with a 30-40 threads max. Btw, that's the only way in which a server can scale in the tens or hundreds of thousands.

    I know it, because I wrote the server's engine :)

    cheers
    Lapo
     
    g_a_p, Jun 24, 2015
    #16

  17. g_a_p

    g_a_p

    Joined:
    Mar 16, 2015
    Posts:
    281
    Running a stress test on WiFi is not a good idea, the network will generate a lovely bottleneck.
     
    g_a_p, Jun 24, 2015
    #17

  18. aabramychev

    aabramychev

    Unity Technologies

    Joined:
    Jul 17, 2012
    Posts:
    574
    >Running a stress test on WiFi is not a good idea, the network will generate a lovely bottleneck.
    I know :) The main task of my stress testing was to compare performance between mobile devices and PC. UNET so far is client library. However, as the above-mentioned digits can give an idea about performance in general, I presented them :)

    >Typically you can run tens of 1000s of users with a 30-40 threads max.
    Hm it depends... How many cores do you have, how many io operation are you going to perform, are you io operation can be done async or not? ... :)
     
    aabramychev, Jun 26, 2015
    #18

  19. g_a_p

    g_a_p

    Joined:
    Mar 16, 2015
    Posts:
    281
    I was referring to non-blocking I/O, of course. Blocking I/O is too hard to scale.
     
    g_a_p, Jul 1, 2015
    #19

  20. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    Blocking io, one thread per connection scales better then people think. Most of what you hear is from years ago, and people just keep repeating it until it's something 'everyone' knows.

    10-15 years ago non blocking started to get popular primarily because linux threads sucked so bad. But those days are gone. So basically where threads and context switching used to be much more expensive, now the non blocking selectors are what is expensive (comparatively). Idle threads use basically 0 cpu, and not only is context switching just not a big factor, the difference in context switching between a handful and thousands of threads is minimal. Connection per thread also lets you use blocking data structures which are faster.

    My own tests using Netty (popular java NIO framework) vs connection per thread weren't even close. NIO used more cpu, created more garbage, and was slower all the way around. Some of that was Netty not NIO, but it was mostly NIO itself. It's pretty obvious what's using what when you spin up visualvm and poke around.
     
    snacktime, Jul 2, 2015
    #20

  21. g_a_p

    g_a_p

    Joined:
    Mar 16, 2015
    Posts:
    281
    yeah, well I have no idea how you could scale up to 50-100K CCU with that model...
     
    g_a_p, Jul 4, 2015
    #21

  22. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    I wouldn't. And anyone doing that is just flat doing it really really wrong. There is only one valid reason to cram that much into a single server, and it's when you need locality. And I don't know of any current commercial games where they need more then around 1000 per server (Eve online has gone a bit higher now and then).
     
    snacktime, Jul 6, 2015
    #22

  23. g_a_p

    g_a_p

    Joined:
    Mar 16, 2015
    Posts:
    281
    Well, I am sure everyone thinks they know the right way to do things, but I can assure you that it's a little more complex than a black or white proposition.

    While cramming 100K on a single machine may have some side-effects (namely if the server dies it's a total black-out), being forced to use 100 machines because the server technology is limited by blocking I/O isn't looking very good, given the costs and maintenance headaches that come with it.

    cheers
     
    g_a_p, Jul 6, 2015
    #23