[RELEASED] Smooth Sync - Smoothly network rigidbodies and transforms while reducing bandwidth

Hello, I bought your asset back when I was using UNET and it was better than any other solution available. I have now switched to Steam networking and was wondering if we could work together to expand the capabilities of Smooth Sync. Would you be interested in helping me make Smooth Sync work outside of UNet or is Smooth Sync strictly limited to use with UNet? Thank you.

 

@chadfranklin47
It should be able to be ported to another networking system fairly easily.

I'd be happy to answer specific questions if you have them, but I'm not looking to add another networking system to Smooth Sync at this time.

You may want to start with the Photon version of SmoothSync though. There's a method that Photon uses called something like Send On Serialize() method, and it's where I convert the SmoothSync data into a byte array to send over the network. It then receives it in that same method and you can see how to convert it from the byte array back into SmoothSync useable data. I would think Steam would have something that handles byte arrays sent over the network.

Then you'll have to convert any RPCs and CMDs over to whatever steam uses.

 

Does SmoothSync store the gameobject's position internally (assume this object using SmoothSync)? Especially when I switch it on and off and then on again in a specific scenario? Just wonder and thanks.

 

Yeah, Smooth Sync on the non-owner will store a list of positions that the owner sends over. It will keep this list when you turn Smooth Sync off and on.

You can use StateBufer[0].position to access the latest received position as well. EDIT: On non-owners that is.

But by the time you turn it on it might not use an old position if it is determined to be too old and might try to extrapolate from it. I don't think there's any wonky behavior from this but let me know if you see any.

EDIT: You were talking about authority changes earlier though and Smooth Sync will wipe the latest received positions on an owner change if you check SmoothAuthorityChanges.

 

Thanks @Fuestrine
Finally I used SmoothSync.clearBuffer() to resolve my problem and it works like a charm!

 

Thanks for your reply, I have another question though: What are the benefits of running SendState() in the FixedUpdate method?

 

@chadfranklin47
Actually trying to send out the states wouldn't matter but it does matter when you set the variables.

I wrote this in a PM to someone recently, let me know if you have any questions:

What I've learned about sending in FixedUpdate() vs Update().

Rigidbodies only get updated after every FixedUpdate(). This means position and velocity are the same for every Update() in between FixedUpdate() but the time is different. So basically you have one position with what could be many different times. So I believe sending in FixedUpdate() is best for rigidbodies.

If you are only updating position by setting position directly in Update() (and probably not using a rigidbody at all) then sending in Update() may be better since it will happen more often so your sending may happen more exactly to how you set your variables. In real scenarios though I can't really see this causing any issues because it'll just send out the next FixedUpdate() which is never far behind unless you set your physics timestep to really low in Unity, in which case your physics rigidbody objects will start looking strange anyway since they don't move at all between FixedUpdate()s.

 

Alright, I see. Thanks for the in-depth explanation.

 

@Fuestrine

I've finally given up and UNET and switched to PUN2. I'm using your new supported package for it so thanks for that.

I'm getting silky smooth syncing for rigidbodies that are not player controlled (they tend to move in one steady direction). However, I'm getting seriously horrible choppy syncing with my players using SmoothSyncPUN2. Players are Rigidbodies being moved almost entirely by rigidbody.MovePosition() with some occasional forces here and there for jumping, hits, etc. The movement though looks terrible and I don't know how to fix it.

Things I've tried:
1) Made network players kinematic rigidbodies to test if it was local physic simulation screwing it up.
2) Used with and without velocity syncing. A MovePosition rigidbody has no velocity so it made no difference.
3) Manually set hasRigidbody to false for your script to try and emulate a transform sync instead.

Any other ideas?

 

@dadamsj1
My guess is your InterpolationBackTime is lower than your SendRate / 100.

I think the default send rate in PUN is 10. So at a send rate of 10 if you put your InterpolationBackTime at >.1, it will interpolate only and should be smooth. If InterpolationBackTime is lower than .1 then you will be extrapolating pretty consistently and changes in direction may look a bit strange so that it can catch up with where it's supposed to be. You can either raise your send rate or your InterpolationBackTime to solve this issue.

If this is not the issue, check out the SmoothSyncExampleScenePUN2 for rigidbodies moving around without being choppy. See if this scene works for you. Use the arrow keys to AddForce() to move them around. Take the rigidbodies off and it will move around without AddForce().

I would also double check that only SmoothSync is changing the position of non-owned objects if it's not an InterpolationBackTime issue and the example scene works for you.

Let me know what you find.

 

Thanks for the quick reply, @Fuestrine .

I did A LOT of testing with the numbers. I set my Send Rate to 30 ticks/sec, Serialization Rate to 20 ticks/sec and, funny enough, I actually got poor results with Rate/100 for interpolationbacktime as you said and have documented in the code. However, I tried to do the inverse 1/20 ticks for 0.05 and got extremely nice and silky smooth syncing.

Any idea why this could be? I'm scratching my head trying to figure it out. SendRate/100 would actually INCREASE interpolation time as tick rate goes up which didn't make much sense to me as I read through your code. Wouldn't you want the interpolation time to be lower as tick rate increases?

 

Hey @Fuestrine. We have been using SmoothSync for a long time now and we love it! Do you plan on supporting it after the deprectation of UNET?

 

@dadamsj1
I think I explained it weird and misspoke. I believe in PUN2 it's PhotonNetwork.SerializationRate.

So you need your InterpolationBackTime to be greater than your PhotonNetwork.SerializationRate / 100.

So if your InterpolationBackTime is .1, your PhotonNetwork.SerializationRate should be 10 or higher.
If your PhotonNetwork.SerializationRate is 10, your InterpolationBackTime should be .1 or higher.

20 SerializationRate pairs with .05 InterpolationBackTime in this manner too.

Yep, that's the general idea. The higher the send rate, the lower you can be allowed to have your InterpolationBackTime and have it be silky smooth interpolation only.

Let me know if I'm still confusing in any of this or if you have more questions.

 

@TCROC
Yep, I plan on supporting Smooth Sync for awhile after the deprecation of UNet.

 

@Fuestrine I'm glad to hear that!

 

I understand you now. Thanks!

You might want to consider changing the documentation to not read "SendRate/100 or higher". The average person will likely read that as "20/100" = 0.2 or higher and get terrible results. Perhaps change it to "1/SendRate or higher".

 

@dadamsj1
Oh yeah, I was thinking about it completely wrong. Thanks for the correction! I'll change the comments for the next update.

 

Spoiler: Error

InvalidCastException: Cannot cast from source type to destination type.
Smooth.SmoothSyncPUN.OnPhotonSerializeView (.PhotonStream stream, PhotonMessageInfo info) (at Assets/Smooth Sync/Other Networking APIs/PUN/Smooth Sync Asset/SmoothSyncPUN.cs:1972)
PhotonView.ExecuteComponentOnSerialize (UnityEngine.Component component, .PhotonStream stream, PhotonMessageInfo info) (at Assets/Photon Unity Networking/Plugins/PhotonNetwork/PhotonView.cs:532)
PhotonView.DeserializeComponent (UnityEngine.Component component, .PhotonStream stream, PhotonMessageInfo info) (at Assets/Photon Unity Networking/Plugins/PhotonNetwork/PhotonView.cs:378)
PhotonView.DeserializeView (.PhotonStream stream, PhotonMessageInfo info) (at Assets/Photon Unity Networking/Plugins/PhotonNetwork/PhotonView.cs:363)
NetworkingPeer.OnSerializeRead (System.Object[] data, .PhotonPlayer sender, Int32 networkTime, Int16 correctPrefix) (at Assets/Photon Unity Networking/Plugins/PhotonNetwork/NetworkingPeer.cs:4380)
NetworkingPeer.OnEvent (ExitGames.Client.Photon.EventData photonEvent) (at Assets/Photon Unity Networking/Plugins/PhotonNetwork/NetworkingPeer.cs:2621)
ExitGames.Client.Photon.PeerBase.DeserializeMessageAndCallback (System.Byte[] inBuff)
ExitGames.Client.Photon.EnetPeer.DispatchIncomingCommands ()
ExitGames.Client.Photon.PhotonPeer.DispatchIncomingCommands ()
PhotonHandler.Update () (at Assets/Photon Unity Networking/Plugins/PhotonNetwork/PhotonHandler.cs:158)

Sync Scale and Sync Angular = None and Compress Position on, Everything else is default.

....

 

@killer1171090
I don't see any issues in the SmoothSyncExamplePUN scene with variables reset to default with Sync Scale is NONE, Sync Angular Velocity is NONE, and Compress Position on.

Are you on the latest version of SmoothSync? (3.10)
Does this happen in the SmoothSyncExamplePUN scene for you?

Do you have any other tips on how I can recreate this issue?

 

Yes I am on the latest, Doesnt seem to happen in the scene... And not really, It only happened once for me so its not a big deal. Ill get more information next time it happens.

 

Does anyone have any tips for the most optimal settings for a third person shooter game?

 

public override float GetNetworkSendInterval()
{
return 1 / sendRate;
}

How does this not create a DivideByZeroException when sendRate is 0? Should I add an if statement to return 0 when sendRate is 0?

 

@chadfranklin47
Yeah, it couldn't hurt to do that. Definitely an oversight by me. I'll update it for the next update.

For your optimal settings question, I'm not sure about optimal for third person shooter, but in order to get the most reactive and accurate setup I would do some of the following.

InterpolationBackTime is setting your objects in the past on purpose so you'll want this as low as possible if you are OK with it being wonkier with higher ping variances and missed message and such.
Don't put InterpolationBackTime lower than 1/sendRate or you'll be extrapolating consistently. If your InterpolationBackTime is exactly 1/sendRate then it will be attempting to move immediately to the latest received position as soon as it gets it. Increasing your send rate will allow you to put your InterpolationBackTime lower.

Increase your easing speeds. I think 1 would work but you'll have to see if it starts jittering for you.

The higher your TimeCorrectionSpeed is, the quicker it'll react to the time differences between messages, but it'll start getting jittery at too high. .1 is default but I would think about .3 might work without being too high.

Threshold settings should be default. Maybe turn off compression if your game doesn't stay close to the origin.

Let me know if you have any questions.

 

Alright, thank you very much for the explanation. I will make these changes and see how good I can make it. Just as a side question, I have a child object that I am syncing, but have no need to sync the root. Am I correct in just setting the root SmoothSync to SendRate: 0 or is there a better way of going about that?

 

Also, do you have any tips for extrapolation settings?

 

@chadfranklin47
I did have another issue with setting send rate to 0. I also introduced a child syncing problem in the last update. It'll be way smoother now.

Sorry about the issues and thanks a ton for bringing them to my attention. I've PMed you a copy of the new version. You'll be able to set SendRate to 0 with the new version.

For extrapolation, the defaults shouldn't give you any problems. You'll want some extrapolation, especially if you want the InterpolationBackTime as low as possible. Extrapolation will happen whenever someone has a network hiccup and doesn't send any transform messages for longer than your InterpolationBackTime.

 

Version 3.11 has been submitted to the Asset Store.

Change log:
-Teleport fix for dedicated servers.
-Added ability to set SendRate to 0.
-Made child syncing smoother.
-Fixed incorrect InterpolationBackTime comment advice.

 

Alright, thanks for all the hard work. My child objects were a bit jittery but I figured that was the best I could get them. What difference did moving that line cause?

 

@chadfranklin47
Oh no, that was awful looking, haha. Child objects are way smoother now.
It was telling the child to not use TimeCorrectionSpeed basically. So it was instantly adjusting to every new time received.

 

@Fuestrine Do you happen to know if SmoothSync will work with Mirror? I wouldn't be surprised if it does because Mirror's mostly a drop-in replacement for UNet but I don't really want to spend the money if no one's tried doing that yet.

 

@Clonkex
I don't think I've heard back from anyone who converted to Mirror. Not sure if that's a good thing or a bad thing. I don't see why it wouldn't be possible but I'm not sure how difficult it would be.

If you buy Smooth Sync and have specific questions while trying to convert it, I'd be happy to answer them.

 

@Fuestrine
Well as it turns out I bought it before you replied anyway and I'm working on converting it now. I managed to fix all the compile errors (mostly caused by small changes Mirror made vs UNet, such as reader.ReadNetworkID() now being reader.ReadNetworkIdentity() and then having to do netID.netId in FindLocalObject.... I'm no Mirror expert, I'm just guessing my way through haha) but now I'm hitting a KeyNotFoundException. I only get it on the server (or host/server+client/whatever you want to call it) with a client connected, and the transforms are not synced. I feel like it's something to do with state not being what it expects, but I'm a little stuck trying to figure out how to even find the problem.

I'm sure I will struggle my way through eventually but if you know off the top of your head what might cause this that'd be super helpful



The code is very nicely written and well commented, btw. Makes it very easy to do these sorts of modifications.

 

@Fuestrine
Hell yeah! I figured it out and now it works perfectly! The problem was this line:

NetworkConnection conn = NetworkServer.connections[i];

Apparently in Mirror the keys to the connections dictionary don't start from 0. I think I'll make another post later that shows the exact changes required to get SmoothSync working with Mirror but right now I have to go through and clean out all my logging...



I'm so excited that my money was well spent!

 

Ok, here's what you have to do to get SmoothSync working with Mirror, being very careful not to give away the source (and bearing in mind I'm no expert and this isn't thoroughly tested; I only know that this worked for me):

Common\Required Use Internal Scripts\HalfHelper.cs
- Comment out line 3; I don't think this is related to using Mirror, but it threw an error and it doesn't appear to be necessary

UNet\Smooth Sync Asset\SmoothSync.cs
- Change line 3 from using UnityEngine.Networking; to using Mirror;
- Change SendByChannel at lines 817, 821 and 2076 to Send
- Change line 2066 from for (int i = 0; i < NetworkServer.connections.Count; i++) to foreach (int key in NetworkServer.connections.Keys)
- Change NetworkServer.connections[i]; at line 2068 to NetworkServer.connections[key];
- Change for (int i = 0; i < netID.observers.Count; i++) at line 2089 to foreach (int key in netID.observers.Keys)
- Change netID.observers[i] at line 2091 to netID.observers[key]
- Comment out lines 2179 to 2182; not really sure what that's for but it seems it's no longer an overridable method in Mirror

UNet\Smooth Sync Asset\State.cs
- Change line 3 from using UnityEngine.Networking; to using Mirror;
- Change reader.ReadNetworkIdentity(); at line 440 to reader.ReadNetworkId();
- Change netID at lines 448 and 452 to netID.netId

It seems the reason for the dictionary error I was getting before is that Mirror doesn't use 0-based (and possibly not sequential) keys for NetworkServer.connections. I later discovered the same thing applies to netID.observers. Easy fix, just use foreach instead. As I mentioned earlier I haven't thoroughly tested this but with the very limited testing I've done so far it seems to work very nicely. I'll test more as I continue to build my game.

 

Hi @Fuestrine

Is me again and merry Xmas. Every time I run below codes:

object.GetComponent<SmoothSync>().clearBuffer();
object.GetComponent<SmoothSync>().enabled = true; // re-enable SmoothSync

I got below error:

NullReferenceException: Object reference not set to an instance of an object
Smooth.SmoothSync.addTeleportState (Smooth.State teleportState) (at Assets/Smooth Sync/UNet/Smooth Sync Asset/SmoothSync.cs:1535)
Smooth.SmoothSync.RpcTeleport (Vector3 position, Vector3 rotation, Vector3 scale, Single tempOwnerTime) (at Assets/Smooth Sync/UNet/Smooth Sync Asset/SmoothSync.cs:1509)
Smooth.SmoothSync.InvokeRpcRpcTeleport (UnityEngine.Networking.NetworkBehaviour obj, UnityEngine.Networking.NetworkReader reader)
UnityEngine.Networking.NetworkIdentity.HandleRPC (Int32 cmdHash, UnityEngine.Networking.NetworkReader reader) (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/NetworkIdentity.cs:650)
UnityEngine.Networking.ClientScene.OnRPCMessage (UnityEngine.Networking.NetworkMessage netMsg) (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/ClientScene.cs:747)
UnityEngine.Networking.NetworkConnection.InvokeHandler (UnityEngine.Networking.NetworkMessage netMsg) (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/NetworkConnection.cs:231)
UnityEngine.Networking.LocalClient.ProcessInternalMessages () (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/LocalClient.cs:138)
UnityEngine.Networking.LocalClient.Update () (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/LocalClient.cs:66)
UnityEngine.Networking.NetworkClient.UpdateClients () (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/NetworkClient.cs:966)
UnityEngine.Networking.NetworkIdentity.UNetStaticUpdate () (at /Users/builduser/buildslave/unity/build/Extensions/Networking/Runtime/NetworkIdentity.cs:1086)

Please advise if you need further info, thanks

 

Hello and Happy Holidays!

I was wondering what the purpose of the checkIfOwnerHasChanged() and the AssignAuthorityCallback() are. This is the last bit of SmoothSync that I haven't been able to make work with Steam Networking as I am sending positions from client to client instead of client to server to client. When I change the authority of my zombies, they jump to another position very quickly before getting back to the correct position. I am guessing these methods were designed to fix that, but I don't see how.

When a client gets a position from a new owner, shouldn't it just start interpolating to that one? What causes the jump? Why add a "fake state"?

 

@Clonkex
Good to hear you got it working with Mirror networking. Thanks for the write up. I'll probably link to your post whenever someone asks about Mirror now. Let me know if you have any questions.

@hmpublic2000
I'm not seeing this issue in the example scene when I turn off the SmoothSync scripts, then clear the buffer and enable the SmoothSync script like you are. In my test I got rid of the child syncing SmoothSync scripts on the example player objects. I tested from client and server side.
My script to test is as follows:

Code (CSharp):

1. public Smooth.SmoothSync smooth1;
2.     public Smooth.SmoothSync smooth2;
3.  
4.     void Update ()
5.     {
6.         if (Input.GetKeyDown(KeyCode.K))
7.         {
8.             smooth1.clearBuffer();
9.             smooth1.enabled = true;
10.             smooth2.clearBuffer();
11.             smooth2.enabled = true;
12.         }
13.     }

All variations I tried didn't get the error.
Try updating to the latest version of SmoothSync and see if you still have the issue.
If that doesn't solve it, do you have any other tips on how I can recreate the issue on my end?
If you don't need it, you can disable the code in OnEnable() that is trying to teleport.

@chadfranklin47
Ah yeah. If Steam Networking sends from client->client directly without going through the server, I could see the automatic authority change handling working incorrectly.
A manual way, and less smooth, would be to clear the buffer yourself (more details in SmoothSyncExamplePlayerController.cs) but you'll just need to call clearBuffer() on all instances of the SmoothSync object when you change authority (except the owner I guess, but it wouldn't hurt anything).
The reason for having to clear the buffer is because it gets the time from the owner to determine the time between States and if it receives States out of order.
CheckIfOwnerHasChanged() adds a "fake" State because you may be changing ownership between States so it just adds a State at the current position and time so it can interpolate between that and the first received State from the new owner.
AssignAuthorityCallback() is a way to know if the owner has changed. The server determines this so it keeps everything the same, but you might be able to RPC this out to the owner instead if Steam Networking doesn't go from client->server->client for messages.
Let me know if you have further questions or if I didn't answer anything fully.

 

Yes, thank you for explaining things. Since I'm not using UNET, can I just change AssignAuthorityCallback from an internal method to a public one and call it on the new assigned owner? Would this work as expected or no since it is supposed to be called on only one machine?

 

So it would be best to call AssignAuthorityCallback() on all players? If that is the case, should I not send ownerChangeIndicator over the network? I thought that would change it for all players.

 

@chadfranklin47
I deleted that fast, I'm surprised you read it. I think I was wrong in that last one. Give me a second to take a look at the code.

 

@chadfranklin47
Hmm, so I'm not really sure. The system was made to be used with one system controlling the ownership changes.

I would check out previousReceivedOwnerInt and ownerChangeIndicator, those are the important extra bits that might be strange.

I think you are right though that you may only need to change ownerChangeIndicator on the owner because it would then get sent to all other systems and get set in Deserialize() when they receive the States. Then previousRecievedOwnerInt would get set in Update() and all might be good.

Try only changing it on the new owner and then try changing it on everyone if that doesn't work would be my best advice.

 

Alright, no problem.

 

Alright, I'll give that a shot and see how things come out. Thanks for the help.

 

@chadfranklin47
Correction: It looks like previousReceivedOwnerInt gets set on received States (checkIfOwnerHasChanged()) not Update().

 

All good.

 

@Fuestrine

What is the drawback of "disable the code in OnEnable() that is trying to teleport"? Thanks.

 

@hmpublic2000
It won't automatically teleport when OnEnable() is triggered. It will lerp from where it currently is to the new received position.

 

Could you elaborate on authorityChangeUpdate()? I guess this is supposed to help keep the object smooth on the new owner. I believe the method has a problem because hasRigidbody is only true when there is no child object and therefore does not work for child syncing. Also, rb is always set to the root object regardless of whether there is a child or not. This may affect much more than just that method though.

 

@chadfranklin47
Yeah, I was ignoring the rigidbodies of children it looks like. Thanks for bringing it to my attention. I think I used to be under the impression that child rigidbodies didn't make sense in Unity land, but they definitely do.

It looks like you can move that hasRigidbody setting and rb component setting section down below where it sets realObjectToSync (in Awake()) and have it get the information from realObjectToSync instead and it should all work out. I haven't tested it at all but I should get some more time in the next couple days to take a good hard look at it if that doesn't work out.

 

Alright, thanks, I'll try that. Yeah, I believe child rigidbodies don't work in the case that a parent has a rigidbody as well.