Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only.

    Please, do not make any changes to your username or email addresses at id.unity.com during this transition time.

    On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live.


    Read our full announcement for more information and let us know if you have any questions.

NetworkMessageInfo.timestamp is BROKEN

Discussion in 'Multiplayer' started by Aubrey-Falconer, Sep 5, 2009.

  1. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    When acting as a server, Network.time - NetworkMessageInfo.timestamp gives me the correct time that a client networkview packet took to reach me.

    When acting as a client, Network.time - NetworkMessageInfo.timestamp returns the same value for the server's NetworkView and all the other client's NetworkViews.

    This is obviously impossible, and it leads me to believe that the server is "resetting" the timestamps of NetworkView packets that it forwards between clients.

    This is a major issue, because it is playing havoc with my player position interpolation code. The docs clearly say that "Timestamps can be used to implement interpolation or extrapolation of continous streams of packets", but it looks to me like this statement is false unless your game is designed with an authoritative server.

    Hopefully someone can point out something that I am doing wrong, or Unity will fix this asap. I have already run into several other significant flaws in Unity's networking. It's disappointing that Unity - so incredible in other areas - is this bugridden when it comes to creating multiplayer games.

    Mike Mac suggested the ingenious system of sending my own timestamp with each NetworkView sync and determining lag from the delta of the previous timestamp, but I wish I could just use Unity's built in components like they say that they are meant to be used.
     
  2. larus

    larus

    Unity Technologies

    Joined:
    Oct 12, 2007
    Posts:
    281
    Right now the timestamp is only the time spent in transit from the sender to the receiver. Which for a client is always the server, as you correctly assumed. The way it works is that the server gets bombarded with updates from all the clients, possibly at various frequencies depending on each clients bandwidth, temporary stutters, jitter etc. Then at regular intervals it sends forward updates on all the network views to the connected clients. It doesn't send the exact packets he did receive so far, the network views themselves get updated with that information. He sends one update for all network views to all clients. The thing is that any transit time from clients to the server is lost this way.

    However, it might be possible to add this as extra information in the network message info structure. There are definetly scenarios where you just want the actual transit time from server to client and nothing else so we can't just remove or pollute that information. We can't assume anything about the type of game being made or how networking is implemented there. It might be possible retain extra timestamp information so that when a client gets a network view update to a view which belongs to another client he could also determine the ago of the last update the server received for that network view.

    Could you file a bug report so we have this in our system? It has a bigger chance of being implemented that way. If you have some example project which demonstrates your scenario where this is needed you could attach that as well.
     
  3. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    Thanks for your reply!
    I have filed Case # 284387 on this issue.

    Very interesting. I had always thought that clients sent their packets to the server with a frequency of the network sendrate, and that the server relayed those packets to the other clients as rapidly as possible.

    Has anyone ever created a reliable multiplayer vehicle physics based game in Unity - and used Unity's networking system? The Unity documentation led me to believe that Unity's built in networking would be perfect for my game, but now I am beginning to wonder if I have made the wrong choice.

    The NetworkMessageInfo.timestamp documentation explictly states that "Network.time - messageInfo.timeStamp is the time the packet spent in transit", and that it "can be used to implement interpolation or extrapolation". This statement is very hard to interpret correctly. How could the average Indy developer like me - trying to learn by reading the documentation - realize that all packets "originate" from the server, even if they are based upon information received from a client in a different packet in the past? It would be great if the "real" timestamp information could be added to the network message info structure - as I don't see what possible good a timestamp on a packet sent by the server could possibly serve when the data that timestamp corresponds to didn't even originate on the server.


    My game involves small numbers (currently 10 max) of players driving vehicles and firing lasers at each other. One of the essential components of my game is that any player can host their own server.

    I constructed it using a non authoritative server model, and it works pretty decently except for these four issues:

    1) I have experimented with Unity's main master server, many versions of the beta master servers, hosting my own release and beta master servers, and finally DimeRocker's master server - and so far, DimeRocker's server is the only one that is even close to an acceptably functional state. It still occasionally can't be connected to - especially from web player games - but it isn't nearly as unreliable as any of the other master servers I have tried - which result in an OnFailedToConnectToMasterServer call so often that it is often impossible to find servers to join.

    2) Connecting to running game servers is extremely unreliable. Attempting to connect to servers often results in "The connection request to XXXXX failed. Are you sure the server can be connected to?" being printed to the console. Interestingly enough, reconnecting several times occasionally results in a successful connection - demonstrating that these cases aren't all NAT issues or something similar.

    3) Player's NetworkViews will often stop sending their sync packets, leaving the player "not responding" to all other clients. Their network connection still works perfectly, as they can fire lasers and chat using RPCs just fine. Nothing I have found short of exiting and rejoining the server (reinitializing everything in the process) resolve this.

    4) I have spent a lot of time trying to build a perfect interpolation system, but each player's movements are still pathetically irregular on other client's games. As I now find out, this was (at least in part) due to Unity's documentation leading me astray on timestamps :)

    Are these issues that I have a chance of resolving?
    Can Unity's networking power a simple game like I am trying to build?
    Was it a mistake for me to build my game with a non-authoritative server model - and is Unity's networking just more tuned for authoritative game servers?

    Thank you very much,

    -Aubrey
     
  4. larus

    larus

    Unity Technologies

    Joined:
    Oct 12, 2007
    Posts:
    281
    The tricky thing is that the connection problem you are describing is hard to reproduce, I have not been able to do so and thus I have only been able to attempt to fix it through guesswork. IIRC I did send a modified beta version to you a couple of times since you experienced the problem frequently, so the last version also exhibited the problem? We can actually pick that up in the email thread we already had going.

    I haven't heard of any such problems with the production master server though, if you or anyone else does have a problem there, please send in bug reports. That is the best way to bring attention to a problem, even if it is a random one.

    Most often if not always, such connection problems come down to NAT implementations in routers. The connection tester was supposed to help with detecting such situations where NAT is a problem. If you think this is a bug please file a bug report. Unfortunately with these kinds of network problems it is very hard to reproduce since it often comes down to equipment used by players, but the report wouldn't hurt.

    Have you filed a bug report on this? Can it be reproduced (since you mention it happens often to you)? I can take a look at it if you file a report, with a project which reproduces the issue if possible.

    I'm sorry about the problem you had with that. I'll see if the documentation can't be improved there.

    This kind of game shouldn't be a problem although the amount of players will of course depend on how much data you end up needing to synch and the target bandwidth requirements. Regarding your problems, they won't get fixed if there aren't any bug reports on them. That's a good start at resolving any issues you are having.
     
  5. perlohmann

    perlohmann

    Joined:
    Feb 12, 2009
    Posts:
    221
    generally the documentation on the network should be improved.

    You should write what is really going on.
    The server does not relay the data the client has sent to the server. The server just acts as both a receiving and sending "socket".

    what this really means is that unless you are aware of it and interpolate/extrapolate the data on the server (received from the client) then the data other clients receives are actually from this interpolate/extrapolated set.
    What the player see's is then an interpolation of an interpolation (assuming you interpolate on the client too).

    http://forum.unity3d.com/viewtopic.php?t=24554 explains a bit better how it works.

    //perlohmann
     
  6. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    Thanks Perlohmann!

    Your description clears up several misconceptions I had formed, and it should be easy for me to embed the necessary timestamp myself on the server side before the data is resent to other clients.


    I understand how hard issues 1 and 2 can be to track down and eliminate Larus - and I will continue to work with you on this. I just really wish that I could use the systems like they are meant to be used and have them work reliably out of the box :)

    I was recently discussing issue 3 @ This Thread (Especially see Zumwalt's post on the next page) - and I will look into this again and file a bug report once I get my interpolation perfected and have a bit more time for new projects.

    I do have 8 open bug reports right now - several of which span multiple Unity version updates - but thanks for encouraging me to continue reporting things. I am sure the Unity team is very busy building great new stuff! I will let everyone know how my game works once I add proper timestamps to it's networking logic.

    All the best,

    -Aubrey
     
  7. dimeRocker

    dimeRocker

    Joined:
    Jun 25, 2009
    Posts:
    79
    Robor Thanks for shout out!

    A few weeks back we did a major upgrade to the dRock Masterserver, the biggest of which included moving it to a better, faster provider.

    We are nearing completion on our dimeRocker prototype, that consists of a centralized management platform enriched with standardized API to allow Unity develops to deploy and monetize their games cross the social web.

    For those of you who would like more info on the dimeRocker Masterserver:http://bit.ly/B9LYu
    Also make sure to sign up to our devlist for upcoming announcements: http://dimerocker.com/devsignup/

    And most importantly check-out our newest initiative www.UnityUsersGroup.com to network and meetup up with local Unity devs in your area. Since launching it a month back it has been building nicely. Need your support. Two more big events planned. Stay tuned.

    Cheers,
    J.Joly
     
  8. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    After quite a bit of tweaking, I now have an interpolation system that works great! I will clean it up a bit more, then add it to the wiki.

    Unfortunately, I am now running into serious issues with packet latency.

    The latency between a webplayer build and the Unity editor, both running on my computer with excellent framerates and no networking emulation, is averaging around 70 ms for me. Playing BZflag (an open source multiplayer game), I often average around 60 ms to a server somewhere else in the world. Playing my game over the internet, I average 150 ms with other players. If I interpolate .3 seconds into the past, my game is noticeably delayed. If I don't, it is noticeably jittery. Does anyone have suggestions for this, or have any idea why two games running on the same computer would have such incredibly high latency?

    I have also observed another somewhat annoying phenomenon: the server duplicating packets as it relays them between clients. It seems that when networking delays prevent a new authoritative client update from reaching the server between the server's send intervals, the server resends the last information it received - pretending to non-authoritative clients that it has new information even though it doesn't. This was crashing my interpolation code until I figured out what was happening and added a sanity check for it...

    I really don't understand why the server doesn't relay packets as soon as it receives them - it seems like it would make more sense, and would automatically resolve these kinds of issues. Is it "normal" for game servers to act like this?
     
  9. perlohmann

    perlohmann

    Joined:
    Feb 12, 2009
    Posts:
    221
    I cant really say anything about the lag. In the multiplayer game we made we also had a pretty high latency despite sitting on a LAN. If you are having issues with players being jerky it could posibly be handled by using extrapolation (dont know if you already do) but only if it already gets it correct 90% of the time or so in the exsisting solution.

    yes its pretty normal to do it that way. Normally a game server has to verify the data in the packets to e.g. prevent cheating. Before updateing other players.

    I can recomend reading some of the articles on Valve's website reguarding their network architecture.
    http://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking
     
  10. adisonpol

    adisonpol

    Guest

    I was suffering from the same problem. But this thread proved to be a boon for me. It is a really nice thread. A nice work began by the writer of this thread and furnished by the commenter. Really cool
     
  11. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    @perlohmann:

    Valve's articles are a wonderful resource! Thanks for sharing.

    I already have an extrapolation routine, and it does help - but as Valve said, extrapolation is only really useful when players follow ballistic trajectories :) I just implemented some new latency aware weapon firing and hit detection code in my game, and it seems to be working nicely.


    @adisonpol:

    Glad you found this useful. Once we can get all the bugs mentioned in this thread worked out, Unity's networking will be something to truly get excited about!


    @All:

    The latest, cutting edge beta of my game can be played @ http://dat.marsxplr.com/199/play.

    I would really appreciate everyone's feedback on how smooth and accurate network movement and firing feels, and information on whether or not you encounter any of the bugs that are plaguing me such as an unresponsive list server, NetworkViews that refuse to sync (the game will show "No Connection" next to the names of any players who's networkview has not yet received a packet, etc). You can set the networking interpolation time and strategy using the appropriate elements in the "Server Settings" panel while you are playing my game.

    During several hours of testing today, I ran into major issues with players being unable to join the game that I was hosting. Most players who did join stated that they had to retry connecting to my game 10 or more times before it would work. This is obviously not a NAT configuration issue, as they were eventually able to connect to me without any alterations to either mine or their internet connections. I also experienced several players who's networkviews would sync perfectly with the server (me), but which did not get relayed to other clients. While this was happening, a couple players who's networkviews were working perfectly exited and rejoined - and they then had the problem as well. It fixed itself after 20 minutes or so, and everyone's packets started relaying correctly again.
     
  12. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
  13. Der Dude

    Der Dude

    Joined:
    Aug 7, 2006
    Posts:
    213
    When I played I was alone, so I can't give you feedback on that.

    But what I can say ist that you've got one fine, polished Lobby-GUI! Very nice!
     
  14. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    Thanks!

    I just noticed that what was posted as the latest version in my previous thread is no longer the latest - this is:
    http://dat.marsxplr.com/2/play

    There's generally 10 or so people playing on this version, and there will be quite a bit more when I officially "release" this version and point everyone's autoupdater to it.
     
  15. DrHotbunz

    DrHotbunz

    Joined:
    Feb 14, 2009
    Posts:
    315
    I notice that the remote players are not smooth. Looking at the code you posted on the Wiki I wonder if this is not because the interpolation is only from network frame to network frame. For it to be really smooth it also needs to be updated every update or frame. Say you network update is 15 ups. It looks like your screen is running 15 fps which is not good.

    Somehow we need to update the position every update between network updates.

    I started writing a smooth script to do just that but have not finished it. Thoughts?
     
  16. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    I think you need to examine the code a little more closely :)

    It already updates the controlled object's position inside the FixedUpdate loop, based upon an interpolation of values in the States array (which is updated by the NetworkView 15 times per second by default). This should be most apparent in the "rigidbody.position = Vector3.Lerp(lhs.p, rhs.p, t);" line, which is where the script determines the position where it's controlled object should be based upon the two nearest appropriately timed states in the states array.

    Perhaps the lack of smooth movement you are experiencing is due to an improperly implemented object physics setup, or the need to average your object's calculated velocity over more than two States? (Current code is "rigidbody.velocity = ((rhs.p - lhs.p) / (rhs.t - lhs.t));"). Please try setting simulatePhysics to false, and let me know if that solves the issue.
     
  17. DrHotbunz

    DrHotbunz

    Joined:
    Feb 14, 2009
    Posts:
    315
    When I set simulatePhysics to false it worked.

    I can only imagine there was a lot of blood sweat and tears go into this script.

    I lost about a month fretting about NetworkInterpolatedTransform.cs. I still do not believe it works right. But your scripts really do work.

    Did you correct the script on the Wiki for the number 4 issue in this thread ?
     
  18. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    Great!

    In regards to #4, I don't have this script working as perfectly as I'd like it too - but it is much better that anything else I am aware of :)

    As I learn new techniques for further improvements, I will keep this script updated. I will definitely experiment with the velocity smoothing I mentioned in my previous post.

    Happy coding!
     
  19. diese440

    diese440

    Joined:
    May 25, 2007
    Posts:
    105
    I have a syntax error with the 'PlayerRemote.js' script on WIKI @ line 13 :

    "private var states = new State[15];"
    --> Assets/PLayerRemote.js(13,26): BCE0018: The name 'State' does not denote a valid type.

    I don't know how to fix the problem; any idea ? Thanks!

    Regards. :wink:
     
  20. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    Ah! thanks for pointing this out.

    I just updated the script to fix the syntax error, and also calculate velocities a little more smoothly.

    Let me know if you run into any other issues!

    Happy Coding,

    -Aubrey
     
  21. diese440

    diese440

    Joined:
    May 25, 2007
    Posts:
    105
    Hello Robur,
    Thx for the previous correction.

    I have another issue here : "PlayerLocal.Js" @lines 9 10 :
    -> Assets/PlayerLocal.js(10,9): BCE0005: Unknown identifier: 'vehicle'.

    I don't see any vehicule definition (no var)

    Thanks for your help...:D

    Claude M.
     
  22. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
  23. Jordos

    Jordos

    Joined:
    Sep 16, 2009
    Posts:
    84
    Is there a solution for the described timestamp behaviour? I mean, can we find out what the actual transit time was, from client to client?
     
  24. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
  25. Jordos

    Jordos

    Joined:
    Sep 16, 2009
    Posts:
    84
    Hi robur, thanks for your reply. I looked at your script, but can't say I totally understand it. Especially this part;

    Code (csharp):
    1. function OnSerializeNetworkView(stream : BitStream, info : NetworkMessageInfo) {
    2.     //We are the server, and have to keep track of relaying messages between connected clients
    3.     if(stream.isWriting) {
    4.         if(stateCount == 0) return;
    5.         p = states[0].p;
    6.         r = states[0].r;
    7.         m = (Network.time - states[0].t) * 1000;    //m is the number of milliseconds that transpire between the packet's original send time and the time it is resent from the server to all the other clients
    8.         stream.Serialize(p);
    9.         stream.Serialize(r);
    10.         stream.Serialize(m);
    11.     }
    12. ...
    13.  
    Where is the check whether this is the server? OnSerializeNetworkView() is called both on clients and server.
     
  26. Aubrey-Falconer

    Aubrey-Falconer

    Joined:
    Feb 13, 2008
    Posts:
    438
    A packet traveling between two clients has two "legs" of it's journey. The first leg is between the authoritative client and the server, which is where stream.isWriting == false and the server's NetworkView is updated. The second leg is between the server and other clients, which is where stream.isWriting == true and the most recent state (states[0]) is forward to other clients.
     
  27. Jordos

    Jordos

    Joined:
    Sep 16, 2009
    Posts:
    84
    I cant look at the script now, as the unifycommunity seems down. But I think you have stated this scipt should only be attached to an opponents object. It makes sense then :)

    Thanks alot Robur