Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We are updating our Terms of Service for all Unity subscription plans, effective October 13, 2022, to create a more streamlined, user-friendly set of terms. Please review them here: unity.com/legal/terms-of-service.
    Dismiss Notice
  3. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice
  4. Join us on Thursday, September 29, for a day with Unity's SRP teams here on the forum or on Reddit, and discuss topics around URP, HDRP, and the Scriptable Render Pipeline in general.
    Dismiss Notice

Question How do I make the player's mouth move whenever they speak so other players can see it?

Discussion in 'Netcode for GameObjects' started by EliasIoannouDev, Sep 21, 2022.

  1. EliasIoannouDev

    EliasIoannouDev

    Joined:
    Oct 20, 2021
    Posts:
    4
    I have already created the functionality for detecting when the player speaks and move the mouth with it. But how do I sync it over the network with netcode? I tried ServerRpc/ClientRpc and this which was the closest to working but it only works on Host because ConnectedClients can be only used on Server.
    Code (CSharp):
    1. private void UpdateMouth()
    2.     {
    3.         foreach (var networkClient in NetworkManager.Singleton.ConnectedClients)
    4.         {
    5.  
    6.             if (networkClient.Key == 0)
    7.             {
    8.                 if (MicLoudness*100 <= 1f && endTalking)
    9.                 {
    10.            
    11.                     //FixMouthServerRpc();
    12.                     return;
    13.                 }
    14.  
    15.                 endTalking = false;
    16.                 mouth1.Value = networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>()
    17.                     .GetBlendShapeWeight(1);
    18.                
    19.                 networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1,
    20.                     _skinnedMeshRenderer.GetBlendShapeWeight(1) + Time.deltaTime * reverseMouth);
    21.                 if (networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().GetBlendShapeWeight(1) <= 0f)
    22.                 {
    23.                     reverseMouth = -1f * reverseMouth;
    24.                     networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1, 0.01f);
    25.                 }
    26.        
    27.                 if (networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().GetBlendShapeWeight(1) >= 100f)
    28.                 {
    29.                     reverseMouth = -1f * reverseMouth;
    30.                     networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1, 99f);
    31.                 }
    32.                 StartCoroutine(StopTalking());
    33.             }
    34.             else if (networkClient.Key == 1)
    35.             {
    36.                 if (MicLoudness*100 <= 1f && endTalking)
    37.                 {
    38.            
    39.                     //FixMouthServerRpc();
    40.                     return;
    41.                 }
    42.  
    43.                 endTalking = false;
    44.  
    45.                 mouth2.Value = networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>()
    46.                     .GetBlendShapeWeight(1);
    47.                 networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1,
    48.                     _skinnedMeshRenderer.GetBlendShapeWeight(1) + Time.deltaTime * reverseMouth);
    49.                 if (networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().GetBlendShapeWeight(1) <= 0f)
    50.                 {
    51.                     reverseMouth = -1f * reverseMouth;
    52.                     networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1, 0.01f);
    53.                 }
    54.        
    55.                 if (networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().GetBlendShapeWeight(1) >= 100f)
    56.                 {
    57.                     reverseMouth = -1f * reverseMouth;
    58.                     networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1, 99f);
    59.                 }
    60.                 StartCoroutine(StopTalking());
    61.             }
    62.             else if (networkClient.Key == 2)
    63.             {
    64.                 if (MicLoudness*100 <= 1f && endTalking)
    65.                 {
    66.            
    67.                     //FixMouthServerRpc();
    68.                     return;
    69.                 }
    70.  
    71.                 endTalking = false;
    72.                
    73.                 mouth3.Value = networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>()
    74.                     .GetBlendShapeWeight(1);
    75.                 networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1,
    76.                     _skinnedMeshRenderer.GetBlendShapeWeight(1) + Time.deltaTime * reverseMouth);
    77.                 if (networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().GetBlendShapeWeight(1) <= 0f)
    78.                 {
    79.                     reverseMouth = -1f * reverseMouth;
    80.                     networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1, 0.01f);
    81.                 }
    82.        
    83.                 if (networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().GetBlendShapeWeight(1) >= 100f)
    84.                 {
    85.                     reverseMouth = -1f * reverseMouth;
    86.                     networkClient.Value.PlayerObject.GetComponentInChildren<SkinnedMeshRenderer>().SetBlendShapeWeight(1, 99f);
    87.                 }
    88.                 StartCoroutine(StopTalking());
    89.             }
    90.         }
    91.     }
     
  2. CosmoM

    CosmoM

    Joined:
    Oct 31, 2015
    Posts:
    184
    First off, doing multiple GetComponents per frame is horribly inefficient – much better to only do that once per player object and cache those. Second, you could drive the mouth movements through animations, in which case all you need to do is add a NetworkAnimator component (and make sure the audio is played at the same time).

    But thirdly, if you're going the route of sending specific operations over the network, don't use a loop over connected clients, but instead use ServerRpcs and ClientRpcs, possibly with ClientRpcParams. The latter is an optional parameter to ClientRpcs that can make it target only specific clients.

    Your code then becomes much simpler. On any client with a player that's speaking, call a ServerRpc on the player object that calls a ClientRpc on the player object that replicates the mouth movement. This means you just need to pass the blend shape weights and anything else you might need to the ServerRpc, which then passes it on to the ClientRpc. Since you are calling these on the *same* player object (i.e. the player that is speaking is the same NetworkObject that other players should see speaking), there is no need to grab other player objects from clients.

    Some uncompiled example code:

    Code (CSharp):
    1. private void Awake() {
    2.   mySkinnedMeshRenderer=gameObject.GetComponentInChildren<SkinnedMeshRenderer>();
    3. }
    4.  
    5. public void UpdateMouth(bool endTalking,float mouthParam1,float mouthParam2) {
    6.   if (endTalking) {
    7.     //end talking here
    8.    return;
    9.   }
    10.   mySkinnedMeshRenderer.SetBlendWeight(mouthParam1,mouthParam2);
    11.   if (IsLocalPlayer) {
    12.     if (!IsServer) {
    13.       UpdateMouthServerRpc(endTalking,mouthParam1,mouthParam2);
    14.     }
    15.     else {
    16.       UpdateMouthClientRpc(endTalking,mouthParam1,mouthParam2);
    17.     }
    18.   }
    19. }
    20.  
    21. [ServerRpc]
    22. private void UpdateMouthServerRpc(bool endTalking,float mouthParam1,float mouthParam2) {
    23.   UpdateMouthClientRpc(endTalking,mouthParam1,mouthParam2);
    24. }
    25.  
    26. [ClientRpc]
    27. private void UpdateMouthClientRpc(bool endTalking,float mouthParam1,float mouthParam2) {
    28.   if (IsLocalPlayer || IsServer) {
    29.     return;
    30.   }
    31.   UpdateMouth(endTalking,mouthParam1,mouthParam2);
    32. }
     
  3. EliasIoannouDev

    EliasIoannouDev

    Joined:
    Oct 20, 2021
    Posts:
    4
    Thanks for the quick reply! I tried your code and its the closest I got it to working. Basically now the Host sees their own mouth move on both him and other players but the client sees the correct result. Did I do something wrong? I'm relatively new to netcode so this is all a bit confusing for me :)
    Code (CSharp):
    1. private void LateUpdate()
    2.     {
    3.         if (MicLoudness*100 >= 1f && !endTalking)
    4.         {
    5.             UpdateMouth();
    6.            
    7.         }
    8.     }
    Code (CSharp):
    1.  public void UpdateMouth() {
    2.         _skinnedMeshRenderer.SetBlendShapeWeight(1,_skinnedMeshRenderer.GetBlendShapeWeight(1)+Time.deltaTime * reverseMouth);
    3.         if (_skinnedMeshRenderer.GetBlendShapeWeight(1) <= 0f)
    4.         {
    5.             reverseMouth = -1f * reverseMouth;
    6.             _skinnedMeshRenderer.SetBlendShapeWeight(1, 0.01f);
    7.         }
    8.        
    9.         if (_skinnedMeshRenderer.GetBlendShapeWeight(1) >= 100f)
    10.         {
    11.             reverseMouth = -1f * reverseMouth;
    12.             _skinnedMeshRenderer.SetBlendShapeWeight(1, 99f);
    13.         }
    14.         if (IsLocalPlayer) {
    15.             if (!IsServer) {
    16.                 UpdateMouthServerRpc();
    17.             }
    18.             else {
    19.                 UpdateMouthClientRpc();
    20.             }
    21.         }
    22.     }
    23.     [ServerRpc]
    24.     private void UpdateMouthServerRpc() {
    25.         UpdateMouthClientRpc();
    26.     }
    27.     [ClientRpc]
    28.     private void UpdateMouthClientRpc() {
    29.         if (IsLocalPlayer || IsServer) {
    30.             return;
    31.         }
    32.         UpdateMouth();
    33.     }
     
  4. CosmoM

    CosmoM

    Joined:
    Oct 31, 2015
    Posts:
    184
    I think the only thing missing is that UpdateMouth in LateUpdate should only be called on the local player (or, if you prefer, only on the server). The Rpcs then ensure the result is replicated for other clients. So do a check for IsLocalPlayer or IsServer depending in LateUpdate.
     
  5. EliasIoannouDev

    EliasIoannouDev

    Joined:
    Oct 20, 2021
    Posts:
    4
    Still the same issue occurs. The client can see the correct movement but the host doesnt see movement in the clients mouth. I tried it with both IsServer and IsLocalPlayer both didnt work. IsServer makes the host see his mouth on both him and the clients and IsLocalPlayer does not show any movement on the client when you are host.
     
  6. CosmoM

    CosmoM

    Joined:
    Oct 31, 2015
    Posts:
    184
    Ah, now I understand what you mean, my bad! Yes, if LateUpdate runs on a local client then the Server never calls UpdateMouth() itself. To fix this, change the ServerRpc:

    Code (CSharp):
    1.     [ServerRpc]
    2.     private void UpdateMouthServerRpc() {
    3.         if (!IsLocalPlayer) UpdateMouth();
    4.         UpdateMouthClientRpc();
    5.     }
     
    EliasIoannouDev likes this.
  7. goodnewsjimdotcom

    goodnewsjimdotcom

    Joined:
    May 24, 2017
    Posts:
    253
    You'd send one packet, it would be the ability for them to trigger the animation of the mouth, and the time stamp so they can project the animation Xmillisconds from when it started
     
    EliasIoannouDev likes this.
  8. EliasIoannouDev

    EliasIoannouDev

    Joined:
    Oct 20, 2021
    Posts:
    4
    Thank you sir! This works perfectly.
     
unityunity