Multiplayer with SteamVR

Hey guys,

I am working on VR tennis game, which should support 2 players with SteamVR.
Every player in the game has a object attached to the head and the controllers which he is able to control(for example a tennis bat).
The connecting and spawning part works fine, but my problem is, the host can control the objects of the client,too.
What I'm trying to do is to spawn the objects from the server side to the controllers of the clients.
I'm using the script from this site to spawn the objects:
https://gamedev.stackexchange.com/questions/132192/vr-play-postion-sync-over-the-network

Every spawned Object has a network identity with local player authority activatet and a network transform component attached to it.

I tried to spawn the Objects with NetworkServer.SpawnWithClientAuthority but they are still under the hosts control.

I hope you guys can help me

Thanks in advance

 

After a while of testing now I noticed the problem is,that the host can control the controllers of the client. The controllers don't have their own network identity, because they are attached to the SteamVR camera rig which has its own network identity.
My problem is, that I don't know how to prevent the host from moving the clients controllers.

 

First, please make sure you refresh your memory of this: Multiplayer Tutorial

You need a setup with three transforms for each network player (HMD, left, right hand). The local player is only is connected to the rig. DO NOT MAKE THE RIG A NETWORK OBJECT. Use the rig only to control (provide location updates for) the local player, and transmit that to the other computer so they can update their remote player info. Separate the network player and rig.

Remember that each scene has only one 'real' local rig. That rig controls only the network player that is attached to the local player. Do that by specifically moving the local player instance's transforms (HMD, hands) to the controller only if (isLocalPlayer), then copy the values for x,y,z, rotation to the network transforms to be transferred to the other computer
If the object isn't local player, pick up the x,y,z for head etc from the network, and copy them to the representation of that player in the scene.

 

The multi player tutorial link is just taking me to https://learn.unity.com/ not a specific tutorial. Anyone know how to get VR set up for multiplayer specifically with SteamVR and preferably using Mirror for a LAN only setup? @JoBo123 did you get Steam VR to work with two players?

 

Looking at the link from JoBo123 ( https://gamedev.stackexchange.com/questions/132192/vr-play-postion-sync-over-the-network) The issue I seem to be having is that the assigned local player head and hands are deleted from the prefab on load when the SteamVR Player is moved to DontDestroyOnLoad. This then gives errors about the object not being assigned. How can I hard code the local player SteamVR LeftHand, RightHand, and FollowHead where needed?

 

Don't hard-code them. Soft-code them by lazy-loading via GameObject.Find each time they are null. It'll only happen once per scene load, but unfortunately not at the immediate start of the scene. It's annoying, agreed.

Note also that I make the rig not a persistent object, but have it load fresh every scene.

 

I must be missing something?

I tried using a GameObject.Find rather then just loading the game object in the inspector however the instantiated network player still has no link to the Steam VR local player.



Here is my version of the script based on my understanding of the first link in this thread.

Code (CSharp):

1.  
2.  
3. using UnityEngine;
4. using Mirror;
5.  
6. public class VRNetPlayerCtrl : NetworkBehaviour
7. {
8.  
9.     //source gameobjects head, left and right controller object of local/host/first player
10.     [SerializeField] private GameObject localHead;
11.     [SerializeField] private GameObject localLeftHand;
12.     [SerializeField] private GameObject localRightHand;
13.  
14.     //prefabs to assign head, left, right controller for Network visbile LAN/client/second player
15.     [SerializeField] private GameObject networkedHead;
16.     [SerializeField] private GameObject networkedLeftHand;
17.     [SerializeField] private GameObject networkedRightHand;
18.  
19.     private GameObject vrHeadObj;
20.     private GameObject vrLeftCtrl;
21.     private GameObject vrRightCtrl;
22.  
23.     void Start()
24.     {
25.  
26.         Debug.Log("Start of the vr player");
27.  
28.         if (isLocalPlayer)
29.         {
30.             //instantiate prefabs
31.             CmdInstantiteHeadAndController();
32.             //disabled conroller meshes at VR player side so it cannont be viewed by local player
33.             vrLeftCtrl.GetComponent<MeshRenderer>().enabled = false;
34.             vrRightCtrl.GetComponent<MeshRenderer>().enabled = false;
35.         }
36.     }
37.  
38.     //Instantiate on start head and vr controller object so that it can be viewed by other players
39.     void CmdInstantiteHeadAndController()
40.     {
41.         Debug.Log("instantiateing the controller and head object");
42.         vrHeadObj = (GameObject)Instantiate(networkedHead);
43.         vrLeftCtrl = (GameObject)Instantiate(networkedLeftHand);
44.         vrRightCtrl = (GameObject)Instantiate(networkedRightHand);
45.  
46.         // spawn the player parts on the clients
47.         NetworkServer.Spawn(vrHeadObj);
48.         NetworkServer.Spawn(vrLeftCtrl);
49.         NetworkServer.Spawn(vrRightCtrl);
50.     }
51.  
52.     void Update()
53.     {
54.  
55.         if (localHead = null)
56.         {
57.             localHead = GameObject.Find("FollowHead");
58.         }
59.  
60.         if (localLeftHand = null)
61.         {
62.             localLeftHand = GameObject.Find("LeftHand");
63.         }
64.  
65.         if (localRightHand = null)
66.         {
67.             localRightHand = GameObject.Find("RightHand");
68.         }
69.  
70.         if (!isLocalPlayer)
71.         {
72.             return;
73.         }
74.  
75.         //sync pos on network
76.         CmdControllerPositionSync();
77.     }
78.     //sync position on VR controller objects so that VR player movemnts/action can be viewd by normal user
79.     [Command]
80.     public void CmdControllerPositionSync()
81.     {
82.  
83.         vrHeadObj.transform.localRotation = localHead.transform.localRotation;
84.         vrHeadObj.transform.position = localHead.transform.position;
85.  
86.         vrLeftCtrl.transform.localRotation = localLeftHand.transform.localRotation;
87.         vrLeftCtrl.transform.localPosition = localLeftHand.transform.position;
88.  
89.         vrRightCtrl.transform.localRotation = localRightHand.transform.localRotation;
90.         vrRightCtrl.transform.localPosition = localRightHand.transform.position;
91.     }
92.  
93. }

What am I missing? The head and hands show up in the scene when I Launch the host but they don't move with the HMD and controllers.

 

I don't know if and what version of SteamVR you are using. In my version (which is LTS), the scene's VR rig is unfortunately re-initialized by Steam, and the various items are moved around, especially the HMD. Make sure to pause the game after it has started and look at the three VR objects to see their state and parenting.

In my player script, I init the connections to the rig in OnStartLocalPlayer. first I access the rig object itself, I then get the Camera (which I know is the object that is the HMD), and finally use transform.Find to find the named objects "Controller (left)" and "Controller (right)" inside the rig - apologies for incorrectly stating I used GameObject:

Code (CSharp):

1. public override void OnStartLocalPlayer() {
2.         // this is ONLY called on local player
3.         // connect to rig
4.  
5.         Debug.Log(gameObject.name + "Entered local start player, locating rig objects");
6.         isLinkedToVR = true;
7.  
8.         // find the gaming rig in the scene and link to it
9.         if (theLocalPlayer == null) {
10.             theLocalPlayer = GameObject.Find("Local VR Rig");// find the rig in the scene
11.         }
12.  
13.  
14.         // now link localHMD, localHands to the Rig so that they are
15.         // automatically filled when the rig moves
16.         localHMD = Camera.main.gameObject; // get HMD
17.         localLeftHand = theLocalPlayer.transform.Find("Controller (left)").gameObject;
18.         localRightHand = theLocalPlayer.transform.Find("Controller (right)").gameObject;
19.  
20.         trackedObjRight = localRightHand.GetComponent<SteamVR_TrackedObject>();
21.         trackedObjLeft = localLeftHand.GetComponent<SteamVR_TrackedObject>();
22.  
23.     }

Hope this helps

 

Oh, and just so we are on the same page (It's not entirely clear to me if the screenshot you showed was from a running scene or not) - the networked player must NOT be part of the scene, it will have to be spawned by the network manager.

In a similar vein, I think you should re-think your command CmdControllerPositionSync - I'm not sure it will work that way, and in any way introduce way too much overhead. Use ChildNetworkTransforms inside the networked player to store the locations of the hmd and controllers, and they will automatically be synched via mirror.

 

Thanks I will look into this.
The screen shot is from a running game. You can see the Player is moved under a Do not destroy on load and the parts are LeftHand RightHand and VRCamera. The clone parts such as Net_L_hand are all spawned. The SteamVR plugin is V2.2.0.
Are you adding the OnStartLocalPlayer to the Steam VR Player Script or your own player script? My Script is on a NetPlayer Prefab that gets a clone spawned when a host or client is started.

Where are you getting isLinkedToVR = true; and theLocalPlayer? Are those your variables or something from the LTS steamVR version?

 

That's part of the network player script on the player prefab. In my script, I named the game object that represents the local player (i.e. the VR Rig) to 'theLocalPlayer' as it is that object that contains many of my relevant VR stuff (left and right controller. I found out that SteamVR removes the HMD, so I'm using Camera.main to find that).

Likewise isLinkedToVR is a bool I set for my script to internally know if it's the version of the multiple running instances that is linked to the Rig and that OnStartLocalPlayer has run, otherwise it is identical to isLocalPlayer. So this variable is only useful between the moment the script awakens and OnStartPlayer is called. I use it mainly for debugging.

 

OK. Thank you very much. So instead of:

1. Make one Gameobject for Head (simple cube or custom mesh) with Network Identity and network transform
2. Make one Gameobject for(right) Controller (simple cube, hand or controller mesh) with Network Identity and network transform
3. Make one Gameobject for(left) other Controller with network Identity and network transform
4. Make prefab of all above Gameobjects.
5. Add all three prefabs into Network Manager (registered Spawnable prefabs list)

Note: I also tested placing my script on the "rig" (SteamVR Player Prefab root) and another version with an empty game object holding my script and placed in the Network Managers Player Prefab without any content in the Spawnable prefabs list.

You did something more like:

1. Make one empty GeameObject for the player object that is separate from the VR rig.
2. Make 3 child GameObjects on it for your head & one for each hand.
3. Put a network transform component on the root object from step 1 and network transform child component on each of your game objects from step 3.
4. Prefab the one game object with its children and ad the script to that one object.
5. Put this prefab in the "Player Prefab" of the network manager.

Either set up should work but the Network transforms on each object (HMD + 2 hands) effectively makes three players on the server while the Network Transform Child method is just making one player for each user, is easier to set up and, more performant without the CmdControllerPositionSync section of code trying to duplicate what mirror is already handling.

The script then is put on the Network Player either way, never the VR rig, and because the rig moves locations in the hierarchy when the game starts do to steamVR scripts loading the camera and controllers based on attached hardware, we lose connections set in the inspector and have to rebind those connections when the network player is spawned.

Does that all sound right? If I have anything wrong please point it out as I hope this information will not only help me but any future noob who comes along and finds to little information on this topic available. (Dapper Dino's Mirror videos are great but they haven't helped me with the VR issues.)

I'm going to test all this and hopefully have some working scripts to share with the community soon.

Update: How do you test this. When I try to run a build and play from unity for host and client side the build closes or the Unity player stops. It wont let me run both. Do I need to send the build to another PC as both instances are trying to access the same HMD? That would make sense and would be one more way in which network building for VR is not the same as building standard multiplayer however I don't yet know if this is true. Something like uEcho would have been nice for testing but for now I am testing with two laptops and headsets which is a pain. WireWiz suggests making a non VR build to run as the server but that could also lead to issues in testing the interaction of synced events across to VR players and he doesn't really show the whole process or code while sharing the solutions on the blog making it hard to follow for an artist trying to teach themselves code.

 

Watched all of First Gear Games and Dapper Dino Videos on networking. Dug into mirrors docs as well as mirrors forum thread and I am stuck with the following code that is still failing to track the head or hands?

Code (CSharp):

1.  
2. using UnityEngine;
3. using Mirror;
4. using Valve.VR;
5.  
6. public class VRNetPlayerCtrl : NetworkBehaviour
7. {
8.  
9.     //source gameobjects head, left and right controller object of local player
10.     private GameObject theLocalPlayer;
11.     [SerializeField] private GameObject localHead;
12.     [SerializeField] private GameObject localLeftHand;
13.     [SerializeField] private GameObject localRightHand;
14.  
15.     //Player parts viewable to others and hidden from the local player
16.     [SerializeField] private GameObject netHeadObj;
17.     [SerializeField] private GameObject netLeftCtrl;
18.     [SerializeField] private GameObject netRightCtrl;
19.  
20.     //Objects tracked for server synchronization
21.     private SteamVR_TrackedObject trackedObjHead;
22.     private SteamVR_TrackedObject trackedObjRight;
23.     private SteamVR_TrackedObject trackedObjLeft;
24.  
25.     void Start()
26.     {
27.  
28.         Debug.Log("Start of the vr player");
29.  
30.         if (isLocalPlayer)
31.         {
32.             //disabled conroller meshes at VR player side so it cannont be viewed by local player
33.             //netHeadObj.GetComponent<MeshRenderer>().enabled = false; //commented out for testing
34.             netLeftCtrl.GetComponent<MeshRenderer>().enabled = false;
35.             netRightCtrl.GetComponent<MeshRenderer>().enabled = false;
36.         }
37.     }
38.  
39.     void Update()
40.     {
41.  
42.         if (!isLocalPlayer)
43.         {
44.             return;
45.         }
46.  
47.         //sync pos on network
48.        // OnStartLocalPlayer();
49.     }
50.  
51.     //instantiate networkPlayer prefab and connect to Local Player Rig
52.     public override void OnStartLocalPlayer()
53.     {
54.         // this is ONLY called on local player
55.  
56.         //Debug.Log(gameObject.name + "Entered local start player, locating rig objects");
57.         //isLinkedToVR = true;
58.  
59.         // find the gaming rig in the scene and link to it
60.         if (theLocalPlayer == null)
61.         {
62.             theLocalPlayer = GameObject.Find("SteamVRObjects");// find the rig in the scene
63.         }
64.  
65.         // now link localHMD, localHands to the Rig so that they are
66.         // automatically filled when the rig moves
67.         //localHead = Camera.main.gameObject; // get HMD with Camera.main wasnt working for me so a lazy load of the game object name was used.
68.         localHead = theLocalPlayer.transform.Find("VRCamera").gameObject;
69.         localLeftHand = theLocalPlayer.transform.Find("LeftHand").gameObject;
70.         localRightHand = theLocalPlayer.transform.Find("RightHand").gameObject;
71.  
72.         trackedObjRight = localRightHand.GetComponent<SteamVR_TrackedObject>();
73.         trackedObjLeft = localLeftHand.GetComponent<SteamVR_TrackedObject>();
74.  
75.     }
76. }
77.  

A note from vis2k on the mirror forom says "Update: NetworkTransform now uses .localPosition and .localRotation for proper VR support" ...Good to know.

 

It looks like pretty much got it.

The game object created in step 1 is the player prefab. It has the network transform (which will sync to the Rig), and three child network transforms, one each for 'netHMD', 'netLeftHand', and 'netRightHand' child objects of this player prefab.

A task each frame only if local player is to simply read the local hmd/controller's position and rotations, and write them to the network child objects. The child network transforms will make sure that they are updated on all other clients for their remote copies. This works really well, and is quite simple to accomplish.
Here's the relevant code in my local player script

Code (CSharp):

1.     void updateHeadAndHands() {
2.  
3.         if (!isLocalPlayer) {
4.             // do nothing, net transform does all the work for us
5.         } else  {
6.             // we are the local player.
7.             // we copy the values from the Rig's HMD
8.             // and hand positions so they can be
9.             // used for local positioning
10.  
11.             // prevent headless version of app from crashing
12.             // depends on SteamVR version if HMD is null or simply won't move
13.             if (localHMD == null) {
14.                 localHMD = defaultHMD;// when running as headless, provide default non-moving objects instead
15.                 localLeftHand = defaultLeftHand;
16.                 localRightHand = defaultRightHand;
17.                 Debug.Log("HEADLESS detected");
18.             }
19.  
20.             // now, we may be "stunned". In this case, we do NOT copy the values across
21.             if (!stunned) {
22.                 netHMD.transform.position = localHMD.transform.position;
23.                 netHMD.transform.rotation = localHMD.transform.rotation;
24.  
25.                 if (localLeftHand) {
26.                     // we need to check in case player left the hand unconnected
27.                     netLeftHand.transform.position = localLeftHand.transform.position;
28.                     netLeftHand.transform.rotation = localLeftHand.transform.rotation;
29.                 }
30.  
31.                 if (localRightHand) {
32.                     // only if right hand is connected
33.                     netRightHand.transform.position = localRightHand.transform.position;
34.                     netRightHand.transform.rotation = localRightHand.transform.rotation;
35.                 }
36.  
37.             }
38.         }
39.  
40.     }

Note:
"stunned" is a bool, and allows me to freeze the local player. It used to be for debugging, but was a great addition to the game later as the result of enemy action.

After that script runs on all instances, each player object's copy (for all players) has up-to-date location and orientation of their netHMD, netLeftHand and netRightHand. Usually you have some mesh/model that needs to sync to these three points. I use a separate 'sync model' script that syncs the playerObject's netXXX to the model's reference transforms, but you can also do this directly. I use indirection because the player can choose different avatar representations and it makes it easier for me to run IK.

the ugly truth is that you can't test the remote part easily without a second computer. But you can test the local logic (allow scene creation with minimal player count of 1), and test as far as you can go. This means that you^prbably also will have to get all your lobby code finished before you really can test the the remote code. Make sure you can play your game witout VR (i.e. it does not crash) so you don't necessarily need two VR sets, and then start testing with two PCs, no simply way around that.

 

OK. The network Child transform makes more sense to use the way you have it. Obvious mistake on my part.

I'm unsure of the script you shared. Is that on the Player Rig or part of the Network player prefab created in step one? I'm thinking the latter is true and will move forward with that in mind, I guess I was thinking the OnStartLocalPlayer function was already performing the work done in your updateHeadAndHands function. It seems the key part I was missing is the if !stunned section of your code replaces the CmdControllerPositionSync function of my old script which failed because I couldn't get the local players HMD and hand positions.

This seems close as I now launch the game and I can move around, pick things up etc and if I click launch host from the desktop the head appears where the HMD is located. (Not hidden in local for testing) but if I teleport the head stays where it is rather then moving with the HMD. (This may be something in the local player only and it is moving on the server. I need to build and move to another machine to test.) One very broken thing however is that if I turn off the host I end up with two hands for each controller. One set in front of the HMD Camera (The left one is inside out) and a set of disembodied steamVR hands that can still teleport and pick things up while The HMD is no longer effected by teleportation. Also any object moved when in the network is duplicated when the network is stopped and restarted. One where the moved object is and one where the original object was placed in the scene. This part will probably be as simple as adding a network identity to the intractable objects and starting and stopping the host probably shouldn't happen although starting and stopping a client may so testing with another machine will be needed there as well.

Currently I have all the code in one OnStartLocalPlayer() function but I suppose I may want to break that up for legibility and debugging.
In the mean time do you see anything wrong with this:

Code (CSharp):

1.  
2. //instantiate networkPlayer prefab and connect to Local Player Rig
3.     public override void OnStartLocalPlayer()
4.     {
5.         // this is ONLY called on local player
6.  
7.         //Debug.Log(gameObject.name + "Entered local start player, locating rig objects");
8.         //isLinkedToVR = true;
9.  
10.         // find the gaming rig in the scene and link to it
11.         if (theLocalPlayer == null)
12.         {
13.             theLocalPlayer = GameObject.Find("SteamVRObjects");// find the rig in the scene
14.         }
15.  
16.         // now link localHMD, localHands to the Rig so that they are
17.         // automatically filled when the rig moves
18.         //localHead = Camera.main.gameObject; // get HMD with Camera.main wasnt working for me so a lazy load of the game object name was used.
19.         localHead = theLocalPlayer.transform.Find("VRCamera").gameObject;
20.         localLeftHand = theLocalPlayer.transform.Find("LeftHand").gameObject;
21.         localRightHand = theLocalPlayer.transform.Find("RightHand").gameObject;
22.  
23.         trackedObjRight = localRightHand.GetComponent<SteamVR_TrackedObject>();
24.         trackedObjLeft = localLeftHand.GetComponent<SteamVR_TrackedObject>();
25.  
26.  
27.         if (!isLocalPlayer)
28.         {
29.             // do nothing, net transform does all the work for us
30.         }
31.         else
32.         {
33.             // we are the local player.
34.             // Copy the values from the Rig's parts so they can be used for positioning the online presence
35.  
36.             // prevent headless version of app from crashing
37.             // depends on SteamVR version if HMD is null or simply won't move
38.             if (localHead == null)
39.             {
40.                 headlessPlayer = GameObject.Find("NoSteamVRFallbackObjects");
41.  
42.                 // when running as headless, provide default non-moving objects instead
43.                 localHead = headlessPlayer.transform.Find("FallbackObjects").gameObject;
44.                 localLeftHand = headlessPlayer.transform.Find("FallbackHand").gameObject;
45.                 localRightHand = headlessPlayer.transform.Find("FallbackHand").gameObject;
46.                 Debug.Log("HEADLESS detected");
47.             }
48.             netHeadObj.transform.position = localHead.transform.position;
49.             netHeadObj.transform.rotation = localHead.transform.rotation;
50.  
51.             if (localLeftHand)
52.             {
53.                 // we need to check in case player left the hand unconnected
54.                 netLeftCtrl.transform.position = localLeftHand.transform.position;
55.                 netLeftCtrl.transform.rotation = localLeftHand.transform.rotation;
56.             }
57.  
58.             if (localRightHand)
59.             {
60.                 // only if right hand is connected
61.                 netRightCtrl.transform.position = localRightHand.transform.position;
62.                 netRightCtrl.transform.rotation = localRightHand.transform.rotation;
63.             }
64.         }
65.     }

 

You can easily tell (once you get more comfortable with Unity's wonky network coding that was saved by mirror ) that the code is from the network player. The dead giveaway is access to isLocalPlayer, because that requires the class to be a network class, and therfore requires a network transform and network id. Such objects must be spawned by the server, and the VR rig in my scenes is part of the pre-set (non-spawned) scene so I can also use it non-networked.

So that code is part of the network player. The part that I failed to mention is that updateHeadAndHands() is invoked every pass through Update() on the player prefab. Remember that if you have two or more players in the scene, you have two or more instances of that script running. Only the local player's copy of the script executes the update that reads the rig's hmd and controls, and copies them to the network transforms, which causes them automatically to update on all remote copies. That's pretty much the entire trick of how to make multiplayer VR rigs work across the network Graphically, the player models need track the network player's HMD/Control transforms, not the VR rig's. (initially, use three differently colord spheres to track their position remotely).

But they need to be updated each update (I preface all methods that are called by Update() with the word "update" so I know where they will be invoked). I put the null checks in (even though they are not very performant) to enable lazy and late initing.

 

The bit about isLocalPlayer makes total sense. it also makes sense to have the updateHeadAndHands() be called from Update(). I borrowed a laptop and another headset for some testing so hopefully I can get the code fixed up and see how it handles on a LAN soon. Didn't have time to test today but When I do I will let you know. I really appreciate all your help as I work to understand this!

 

The code seems to be working now as I can test locally and have the head and hands follow properly (tested with meshes viable) and I can at least pick things up with the mouse and see them move in both a build and the Unity play window. (tested with the HMD turned off). However now I m stuck with the NetworkManagerHUD reverting to local host even if I put in an IP address in the inspector or just typed into the game windows field prior to launching the host; alternately I tried the NetworkDiscoveryHUD and it is failing to find the second laptop connected with a net gear switch. I have never done a LAN party so setting up IPs and default gateways with standard 192.168.0.x and subnet mask of 255.*3 .0 seems like it should have worked but this is driving me nuts. It also doesn't help that I have been sick for three days so far; so learning this while my head is spinning has been a nightmare.

 

OK I got the LAN working with network discovery by setting the DNS preferred server to 1's and the Alternate DNS Server to 2's The rest of the settings stayed the same with the gateway as .1 and each laptops IP consecutive .2 and .3 IP addresses. Amazing that non of the websites I checked or videos i watched mentioned the DNS but I guess they all assumed I wanted to connect to the internet and not have an offline LAN-only connection.

Now I can launch I can only launch the client server from the second machine 192.168.0.3 if I launch from the first machine 192.168.0.2 the second machine can't find the server and when I launch the "Start Host" from machine two and "Find Server" from machine one I can see the head and hands of the host move around in the client players view. If the host push a ball it rolls but if they pick it up it disappears and the host can't see the movement of the client players head or hands nor can they see the clients interactions with objects. I'm sure 99% of this is just tweaking I need to do in Mirror with network transforms and network identities, maybe something to do with network authority or the sync mode. However the player transforms don't make sense to me and I suspect its still something in the code that is making the host visible to the client but hiding the clients movement from the host. (A head and hands do appear at a default location they just don't sync in the clients movements to the hosts view?) Also, Why can't I launch the host from either machine?

current script:

Code (CSharp):

1.  
2. using UnityEngine;
3. using Mirror;
4. using Valve.VR;
5.  
6. public class VRNetPlayerCtrl : NetworkBehaviour
7. {
8.  
9.     //source gameobjects head, left and right controller object of local player
10.     private GameObject theLocalPlayer;
11.     private GameObject headlessPlayer;
12.     [SerializeField] private GameObject localHead;
13.     [SerializeField] private GameObject localLeftHand;
14.     [SerializeField] private GameObject localRightHand;
15.  
16.     //Player parts viewable to others and hidden from the local player
17.     [SerializeField] private GameObject netHeadObj;
18.     [SerializeField] private GameObject netLeftCtrl;
19.     [SerializeField] private GameObject netRightCtrl;
20.  
21.     //Objects tracked for server synchronization
22.     private SteamVR_TrackedObject trackedObjHead;
23.     private SteamVR_TrackedObject trackedObjRight;
24.     private SteamVR_TrackedObject trackedObjLeft;
25.  
26.     void Start()
27.     {
28.  
29.         Debug.Log("Start of the vr player");
30.  
31.         if (isLocalPlayer)
32.         {
33.             //disabled conroller meshes at VR player side so it cannont be viewed by local player
34.             //netHeadObj.GetComponent<MeshRenderer>().enabled = false; //commented out for testing
35.             netLeftCtrl.GetComponent<MeshRenderer>().enabled = false;
36.             netRightCtrl.GetComponent<MeshRenderer>().enabled = false;
37.         }
38.     }
39.  
40.     void Update()
41.     {
42.  
43.         if (!isLocalPlayer)
44.         {
45.             return;
46.         }
47.  
48.         //sync pos on network
49.        OnStartLocalPlayer();
50.        updateHeadAndHands();
51.     }
52.  
53.     //instantiate networkPlayer prefab and connect to Local Player Rig
54.     public override void OnStartLocalPlayer()
55.     {
56.         // this is ONLY called on local player
57.  
58.         //Debug.Log(gameObject.name + "Entered local start player, locating rig objects");
59.         //isLinkedToVR = true;
60.  
61.         // find the gaming rig in the scene and link to it
62.         if (theLocalPlayer == null)
63.         {
64.             theLocalPlayer = GameObject.Find("SteamVRObjects");// find the rig in the scene
65.         }
66.  
67.         // now link localHMD, localHands to the Rig so that they are
68.         // automatically filled when the rig moves
69.         //localHead = Camera.main.gameObject; // get HMD with Camera.main wasnt working for me so a lazy load of the game object name was used.
70.         localHead = theLocalPlayer.transform.Find("VRCamera").gameObject;
71.         localLeftHand = theLocalPlayer.transform.Find("LeftHand").gameObject;
72.         localRightHand = theLocalPlayer.transform.Find("RightHand").gameObject;
73.  
74.         trackedObjRight = localRightHand.GetComponent<SteamVR_TrackedObject>();
75.         trackedObjLeft = localLeftHand.GetComponent<SteamVR_TrackedObject>();
76.     }
77.  
78.     void updateHeadAndHands()
79.     {
80.         if (!isLocalPlayer)
81.         {
82.             // do nothing, net transform does all the work for us
83.         }
84.         else
85.         {
86.             // we are the local player.
87.             // Copy the values from the Rig's parts so they can be used for positioning the online presence
88.  
89.             // prevent headless version of app from crashing
90.             // depends on SteamVR version if HMD is null or simply won't move
91.             if (localHead == null)
92.             {
93.                 headlessPlayer = GameObject.Find("NoSteamVRFallbackObjects");
94.  
95.                 // when running as headless, provide default non-moving objects instead
96.                 localHead = headlessPlayer.transform.Find("FallbackObjects").gameObject;
97.                 localLeftHand = headlessPlayer.transform.Find("FallbackHand").gameObject;
98.                 localRightHand = headlessPlayer.transform.Find("FallbackHand").gameObject;
99.                 Debug.Log("HEADLESS detected");
100.             }
101.             netHeadObj.transform.position = localHead.transform.position;
102.             netHeadObj.transform.rotation = localHead.transform.rotation;
103.  
104.             if (localLeftHand)
105.             {
106.                 // we need to check in case player left the hand unconnected
107.                 netLeftCtrl.transform.position = localLeftHand.transform.position;
108.                 netLeftCtrl.transform.rotation = localLeftHand.transform.rotation;
109.             }
110.  
111.             if (localRightHand)
112.             {
113.                 // only if right hand is connected
114.                 netRightCtrl.transform.position = localRightHand.transform.position;
115.                 netRightCtrl.transform.rotation = localRightHand.transform.rotation;
116.             }
117.         }
118.     }
119. }

 

A few posts ago in an aside I commented:

"This means that you probably also will have to get all your lobby code finished before you really can test the the remote code"

That's still true. Unfortunately, like me, you got the order wrong when building the prototype. You first need a reliable way to discover and connect. Depending upon the architecture model you choose (p2p or server/client) you need different network managers. Only after you can connect any two (or more) machines you choose, you should proceed to synching up the VR rigs.

I have no idea why you have difficulties discovering local clients, but it looks like some hard-coded IP addresses somewhere in your prefabs.

When an object disappears or does not behave as you expected, 99% of the time it is because something is wrong with the way you synchronize with the server (or rather do not synchronize) via commands, or because the object itself was spawned incorrectly (did you add it to the scene's registered spawnable obejcts? That one got me a few times). Remember that you cannot move any object that was not spawned via the server if you want that object to be synched across all clients. This means that you will usually first issue a command so the sever version of your script spawns it - 9 times out of ten I forgot that part and blithely only spawned on the client that had local authority.

 

I am wanting one player to be able to be active in the scene and then have another player join. So no. I didn't add them to the spawnable objects as I am using networked Scene Game Objects. This design choice also led me to skip the lobby but I may need to rethink that after all the system still has to know who the host is. Whats odd is the machines do connect with network discovery but only one machine is able to run the host making me wonder if its a physical cabling issue as I removed all hard coded address information from previous testing. Then once connected it seems the server authority is not allowing any client but the host to interact with objects. Its as if the client player is simply not connected as a player. wait... I don't think I turned on client authority for the NetworkTransformChild scripts controlling the head and hands!

minutes later:
Yep! that did it. I still have the issue of only one machine running the host and objects disappearing from the others players view when picked up but now I can see both players head and hands move about the scene and both can push objects around the scene with both players seeing the synchronized motion. I think what I need to do now is make a script that gives client authority to objects when they are in hand and returns authority to the server when the object is dropped, hopefully that will keep the objects visible to the other player while they are being held.

 

Is there a way to set and reset the port? Telepathy sets the port at 7777 however if I open cmd on windows run an ipconfig I can see my IPV for is set as I have specified I can run a ping between machines to insure they are connected and sending/receiving data from one another however when I run netstat -a The port of my set IP is not 7777 and I'm wondering if this may be why I am getting an error stating the Client Recv: failed to connect to port=7777 reason = System.netSockets.SocketExeption ... could not be made because the target machine actively refused it. ? Its not a firewall blocking the connection so I'm not sure whats happening. It worked for a few hours and then just stopped. I tested an auto connecting lobby similar to what Jared Brandjes did but I couldn't even get it to launch the host properly so I went back to using network discovery which will actually launch the host and doesn't show any errors in the console but now the client can't find the server on either machine! The only thing that changed from when it was working to when it stopped was a reboot of both laptops and the switch; then the working build just refused to connect again.

The thing is, none of the port forwarding or NAT punch through should mater as I am making an offline LAN Only network with no firewall.

 

I turned all my private, domain, and public firewalls on and off again for both PC's and now I can only run the host from PC 1 where as before I could only run the host from PC 2! This makes no sense. I should be able to run the host from either machine and connect from the other or indeed multiple other machines with the 5 port switch.

 

Rebooting the PC's previously turned all firewalls on for both machines and I had only turned some of them back off.

With firewalls configured properly I can connect either machine as the host or client.

 

For the syncing of objects being picked up I am looking into remote procedure calls (RPC) and have the following links for anyone else who may be working on a similar project.
https://forum.unity.com/threads/solved-how-to-set-transform-parent-via-rpc.31336/ general transform with RPC
https://forum.unity.com/threads/parenting-over-a-network.513229/ Using VRTK users suggest an RPC
https://answers.unity.com/questions/1634886/steamvr-photon-disappearing-objects-issue.html my issue is also seen in photon networking with SteamVR.

Statements like "A simple RPC to child it and dechild it upon grab and release fixes everything.", make this seem easy but I'm still having difficulty understanding the needed code and Mirrors documentation on
public class ClientRpcAttribute : Attribute doesn't tell me anything.

 

Well the RPC was the wrong rabbit hole. I learned some interesting things but nothing that fixes the disappearing object issue. As csofranz said

So time to look over Mirrors documentation, demos, and tutorials again.

 

I just noticed that the object being picked up isn't disappearing its being pushed below the ground plain and then spinning about as gravity pulls it down. so this is not just an issue with authority or lack of synchronizing its the way the rigid body is interacting. A ClientRpc or a [command] to the server may be useful but I no longer think its due to a lack of ownership on the part of the client or server at any given time (Could be wrong this stuff still confuses me) but maybe I could get the object to follow the hand once garbed using a SyncVar instead of a remote action. Or maybe simply buying smooth sync will clear up the issues with the rigidbody?

 

Hello Giantbean, I have been following this thread eagerly, how are you progressing?

 

So I got stuck worked on some other things and came back to this. I joined First Gear Games patreon and have had some help from the great people there and I purchased Smooth Sync. What I have working is Smooth Sync with some code inspired by and learned from Punfish of First Gear Games. I'm also working with them to see if I can use the Flex Network Transform (FNT) code he wrote to get smother motions then Smooth Sync is currently giving on jittery held VR objects.

For now what I have working is the previously posted VRNetPlayerCtrl.cs with the line " localHead = theLocalPlayer.transform.Find("VRCamera").gameObject; " currently commented out.
My network player prefab has the code on a game object which holds the Head and hand objects however the local head variable currently has the parent object (Itself) load in the inspector rather then its child head object. If that's confusing, ask and I can add an image to clarify.

The network player base object has a sphere collider that I am using to check the proximity of tagged objects. Tagged objects have a network id and either a network transform or a FNT script. These Items go to world origin 0,0,0 for non owners when picked up. Something wrong in my code but I'm not sure what yet. I think the item is losing its synchronization between the client and server when the SteamVR hand parents the object and moves its position in the hierarchy causing the server and client to look for the object in different locations and leading to the object being serialized at 0,0,0. But I'm not sure.

Then there are the objects with Smooth Sync. These objects can be picked up and used by the local client and the transform and rotation is synchronized to the server and populated for other clients to see. My terminology may be wrong but the point is, it works. So now for the key to getting it working! The code below looks for near by objects checks for a "networked" tag and registers the use of the grip on the VR controller to then ask for authority over an object.

Code (CSharp):

1.  
2. using UnityEngine;
3. using Mirror;
4. using Valve.VR;
5.  
6. public class AssignAuthority : NetworkBehaviour
7. {
8.     private SteamVR_Input_Sources inputSource;
9.  
10.     private void Start()
11.     {
12.         //so I can turn off the script in the inspector.
13.     }
14.  
15.     //Request authority
16.     private void OnTriggerEnter(Collider other)
17.     {
18.         //Make sure this object has authority when the scene is loaded
19.         //commands cant be sent without authority so...
20.         if (!base.hasAuthority)
21.             return;
22.  
23.         //Get the network ID of the object to grab when it is near
24.         if (other.gameObject.tag == "Networked")
25.         {
26.             //check to see if the SteamVR grip is active.
27.             if (SteamVR_Actions._default.GrabGrip.GetStateDown(inputSource) == true)
28.             {
29.                 //Does the collided object have a network ID?
30.                 NetworkIdentity id = other.GetComponent<NetworkIdentity>();
31.                 //If the client doesn't own the object...
32.                 if (id != null && !id.hasAuthority)
33.                 {
34.                     //request Authority
35.                     Debug.Log("requesting authority for " + other.gameObject.name);
36.                     CmdRequestAuthority(id);
37.                 }
38.             }
39.         }
40.         else { return; }
41.      
42.     }
43.  
44.     //assign the NetworkIdentity of the requested object to the requesting player
45.     [Command]
46.     private void CmdRequestAuthority(NetworkIdentity targetId)
47.     {
48.         //Take control of the target from another player or server.
49.         targetId.AssignClientAuthority(base.connectionToClient);
50.     }
51. }
52.  
53.  

I'm still working out kinks and when I moved the code to a new scene and tried to duplicate the success I strangely ended up with clients being able to pick up and move objects but others cant see the player (head or hands) who is moving the object (unless its the host). So it looks like items are just moving about on their own. Very odd. It could be the lack of one of my other scripts in the scene causing the issue but I'm still working that out.

P.S. I am a also testing with a non VR scene to speed up iteration using a .bat file technique shown here. However this is snot as accurate as testing with two machines and headsets so I need to move the build to another machine and run some more tests to make sure things are really working or not.

 

This script has also been helpful in debugging issues.

Code (CSharp):

1.  
2. using UnityEngine;
3. using Mirror;
4.  
5. public class AuthorityColorDebugger : NetworkBehaviour
6. {
7.     Renderer Object;
8.     //public Material main;
9.     public Material active;
10.  
11.     public void Start()
12.     {
13.         if(GetComponent<Renderer>() != null)
14.         {
15.             Object = GetComponentInChildren<Renderer>();
16.         }
17.         else Object = GetComponentInChildren<Renderer>();
18.     }
19.  
20.     //change the color on a client when they have authority of an object
21.     public override void OnStartAuthority()
22.     {
23.         base.OnStartAuthority();
24.             this.Object.material = active;
25.     }
26.  
27.     //This is not set up properly to show when an object is dropped but
28.     //may be usefull in the future or
29.     //for testing on two machines when another player takes authority.
30.     //public override void OnStopAuthority()
31.     //{
32.     //    base.OnStopAuthority();
33.     //        this.Object.material = main;
34.     //}
35. }

 

Quick update. I got rid of the jitter that I was seeing in smooth sync by using the Flex Network Transform Child Script from the Patreon listed two posts ago. I unchecked "Use Local Space" and made sure "Client Authoritative" was checked, all other settings where default from the current FNT release and it was supper smooth! If I play with settings I may be able to get Smooth Sync as clean but for now I think FNT is the way to go. I am playing with both though to see what other issues may arise and to see if the basic FNT can be made as smooth as the FNT Child is. Not sure why one works better but removing the empty parent game object and putting the code directly on the object seems like it would be better. The version I'm using is an experimental release so...Maybe Puns next release will work without a hitch.

Update: It did!