Unity Editor and Virtual Desktop

Heya
Im Developing in Unity (latest ver) and am starting to work on VR games. One point, i havent been using a link cable because ive been streaming games to my Quest via VD so i havent needed to buy one.

What i want to know is has anyone had any luck connecting Unity Editor to the Quest via Virtual desktop?

ive followed a few tutorials online but they are outdated as the VR plugins are being superceded by the XR plugin so its super confusing.

i have no problem connecting to Steam VR but have no idea if i ccan edit my Software and test on the fly inside the quest.Im hoping this is possible otherwise im screwed by the fact that i cant get a link cable on the planet atm.


TYIA

 

I like to have the same. But i've a cable for Link. Trick is that Oculus will Hook when we go to PlayMode with Editor. It always says no HMD. Thats why the Hook goes from Virtual Desktop to Oculus. So you've to start an App via Virtual Desktop. Thats tricky with the editor and only works with a build. I wonder there is somehing possible...

 

I managed to run Unity Editor bypassing Hub VD injection problem. So sweet to have wireless VR development ability again! All you have to do is open unity project using command line with VD injection.
Like this:

cd C:\Program Files\Virtual Desktop Streamer\

VirtualDesktop.Streamer.exe "C:\Program Files\Unity\Hub\Editor\2020.2.0b2\Editor\Unity.exe" -projectpath D:/Projects/XR-Interaction-Toolkit-Examples/VR -useHub -hubIPC -cloudEnvironment production

PS: I don't know exactly what these arguments mean, I just captured them using API Monitor

 

Hey @dnnkeeper, tried the command, but that just opens the Unity Hub, and I could not get the Unity Editor to "see" my headset or controllers. Could I trouble you for some step by step process?

 

Try launching Unity Hub manually prior to executing this command. Then hit Play through the VD or make sure headset is active and connected at this moment. Your project must have XR Management plugin for Oculus installed and enabled.

 

After a bit of back and forth I finally got it working, thanks @dnnkeeper! For you others stumbling in here wondering about the same, I'll post what I did.

First of all: Localize your Virtual Desktop (VD) folder, specifically the "Virtual Desktop Streamer" folder. We're gonna need the exe path here, so usually it's
"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe". Take note of the path.

Second, find your project folder, i.e. "S:\UnityProjects\VRToolkitUnity", only the "top" path of the folder is needed. It's VERY important that the proejct path does NOT contain any white spaces. Even if its enclosed I couldn't getting it to work, so I had to remove any whitespace in the actual folder names in Windows.

Next is the Unity Editor executable, the one installed through the hub. In most cases it's located like this: "C:\Program Files\Unity\Hub\Editor\2020.1.3f1\Editor\Unity.exe"
Replace "2020.1.3f1" with the version your specific project uses.

So let inject things here:
Open a cmd window (Windows + R, write "cmd", or search for "command" or "cmd" in Start menu search. so here's the kicker: With paths we've just located paste in this, in this syntax: "<Virtual Dekstop Streamer Exe Path> "<Unity Editor executable path"> -projectpath "<Unity project path".

An example here:

Code (Boo):

1. "C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2020.1.3f1\Editor\Unity.exe" -projectpath "S:\UnityProjects\VRToolkitUnity"

Don't forget the " " when pasting in your own paths!

Then Unity starts, if you hit play when VD is active on the Quest, it runs! Even if you start VD after you've started Unity, it still works!

EDIT: Some reported that adding "-useHub -hubIPC -cloudEnvironment production" after the project path cleared up some HMD detection issues.

Project path would then be:

Code (Boo):

1. -projectpath "S:\UnityProjects\VRToolkitUnity -useHub -hubIPC -cloudEnvironment production"

Again creds to @dnnkeeper!

 

It worked! I even made a batch to be able to call this command from the context menu! Thanks guys!

 

I bought the VD Quest app and it works, but even though I can run the commands above and get the Unity editor running, it won't "preview" in Quest as VR, it will preview as if I had no HMD connected, in the flat editor screen. Any thoughts?

 

It worked for me too, but the hand controllers are not being detected as expected... I do see them in the XR input inspector, but when I do "InputDevices.GetDevices(allDevices);" in the code, only the headset appears, and not the hands! With the cable it finds the 3 of them.... Is anyone else having this problem? Any idea or possible solution? Thank you very much

 

Adding the "-useHub -hubIPC -cloudEnvironment production" seems to have solved the problem in case someone is struggling with this

 

Strange and interesting, I found the "-useHub -hubIPC -cloudEnvironment production" not making any difference for me.
I'll update my answer with this!

 

I've installed countless Unity Editor versions, patched VD via SideQuest and still nothing happens when I click play. Tried with and without the "-useHub -hubIPC -cloudEnvironment production", no success at all. Other than downloading VD, patching it with SideQuest, is there something else?

 

Have you tried using Oculus Link with this project? Sometimes even wired connection doesn't work properly. If you able to run it with usb cable then you should be able to run it with VD injection. Try opening Oculus app before starting editor.

 

Turns out I was putting the extra commands 'outside' the project path argument. It works now and I'm the happiest programmer on earth Thanks everyone!

 

Unity Editor just crashes for me. =( Any suggestions?

I added the sidequest file, then I made a .bat file of:

"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2020.1.16f1\Editor\Unity.exe" -projectpath "C:\MarshallVisions\GitHub\WizardsKeep -useHub -hubIPC -cloudEnvironment production"

This causes Unity Editor to run. Then when I click play, a second Unity icon appears on the taskbar for a couple seconds, then disappears then the Editor crashes.

 

This sounds fantastic. Brilliant I am going to give it a shot in the morning.

That would be exactly what I am looking for, maybe better than ALVR and not having to be connected with a Link cable.



Does this support the controllers too?

 

Did you ever get that working? I tried adding that, but doesn't seem to work, where when I try InputDevices.GetDevices, it only returns 1 device (QuestHeadMounted, TrackedDevice).

I used the following:

Code (csharp):

1. "C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2020.2.0f1\Editor\Unity.exe" -projectpath "D:\UnityProjects\TestUnity1 -useHub -hubIPC -cloudEnvironment production"

Be good if that's working... appreciate anyone's help, thanks!

It's actually working now. Turns out I just need to make sure I have the Oculus software opened, and then it all worked like normal. Thanks everyone here! It's so awesome to be able to code remotely.

 

Did this for my specific project and it started VD desktop, started the unity project, I put a XR room scale camera in the scene, pressed play and it just crashes unity.
So we need more info on which version of unity, XR intergration or legacy Oculus?, any other settings.
It would be nice if you could chat on the Virtual Desktop Discord. I am there too.

this was my console to start everything

"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2019.4.14f1\Editor\Unity.exe" -projectpath "C:\UnityProjectsNov2020SSD\QuestURPDez2020"

 

Apparently some are having some luck with this solution. So if anyone else manages to get this solution or another working, then it would be great if you could share your success with us all and how you got it working. Maybe even let us know on the Virtual Desktop discord.

Thanks in advance to any feedback.

 

I cannot get this to work for me. Using Unity 2020.2.1f1

Here's my cmd line.

Code (Boo):

1. "C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2020.2.1f1\Editor\Unity.exe" -projectpath "D:\GitHub\VRProject2\VR-Unity-2020 -useHub -hubIPC -cloudEnvironment production"

I have the Oculus app open. I've tried with VD already running and opening it after. It opens Unity to the correct project. The project has the XR stuff set up, but when I press Play, it doesn't go to my Quest. It's just the normal Playback in the editor, that I can see on VD.

 

You don’t need any of those commands.

You need SteamVR installed on your Desktop, also SideQuest and Virtual Desktop.

On your quest you need to buy and install Virtual Desktop, then use side quest to install the Virtual desktop patch.

In Unity in a fresh project just install the SteamVR Plugin from the asset store and that will set you up and it will also install your XR Management.

Start then Virtual Desktop in your quest, connect to your PC and click the left menu button of the controller to see the VD menu. There you will see start Steam. Now your quest will be in the steam environment.

In Unity just press play and there you go, your scene will run in the quest wirelessly with controllers

 

Followed your instructions which worked, but the camera using XR Rig for action based, doesn't work properly. The entire scene seems to be locked on the camera instead of camera moving individually and tracked

 

Did you fix it?

 

did you enable the oculus plugin?

 

Thanks for the previous work on this @dnnkeeper and @Eddiman !
I managed to get wireless play-mode on Quest 2 running using Virtual Desktop 1.18.9, Unity 2020.2.2f1 and the XR Plugin Management package.
It took me some time to figure everything out, but here is the solution:

Prerequisites [Unity]
Packages: "XR Plugin Management" and optional "VR Interaction Framework"
Enable the Oculus Plugin: Edit - Project Settings - XR Plug-in Management - Enable "Initialize XR on Startup" and "Oculus"
@Selaphiel included a screenshot in the previous post of how this should look like in the end.

Procedure

1. !!! Neither launch Oculus nor Steam VR and make sure Unity Hub and all your Unity instances are closed!!!
2. PC: Launch Virtual Desktop Streamer
3. Quest 2: Start Virtual Desktop app and connect with PC (PC and Quest 2 have to be on the same Wifi!)
4. Execute this command in cmd.exe (make sure to adapt the paths!)
   

   "C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2020.2.2f1\Editor\Unity.exe" -projectpath "C:\Repos\proj-stardew-vr-health -cloudEnvironment"

   
   Hint: the parameter [-cloudEnvironment] is needed, as otherwise, Unity won't accept input from the Quest (via Virtual Desktop) the first time. So you would have to close Unity and start it via the command again
5. Inside Quest 2 press Unity's Play-button (Make sure you have a scene open where the XR rig is set up correctly).

Fast procedure via my .bat script
Place the following lines of code inside a .bat file (make sure to adapt the paths!):

start "" "C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe"

echo "Inside your Oculus Quest start the Virtual Desktop App and connect with your PC, then inside the Quest click inside the CMD window, open the virtual keyboard (X-Button on left controller) and press a key"

timeout 60

"C:\Program Files\Virtual Desktop Streamer\VirtualDesktop.Streamer.exe" "C:\Program Files\Unity\Hub\Editor\2020.2.2f1\Editor\Unity.exe" -projectpath "C:\Repos\proj-stardew-vr-health -cloudEnvironment"

Then every time you want to get started, close everything (Unity, Unity Hub, Virtual Desktop, Oculus, SteamVR) and simply start the .bat.

 

@Captain_Breadbeard @unity_zFXZPrrcQEU5Uw Look at my instructions, this should help you!

 

Launches the hub and Unity welcome window, but Unity does not launch. At what point could I be wrong?


upd. This works for me when I launch bat file before launching SteamVR and Oculus Quest Virtual Desktop.

 

I wrote you a PM, let's look into this there.

 

I've been able to get the Quest working with Virtual Desktop by installing the XR Toolkit, OpenXR Plugin and setting the Editor instance to run through SteamVR, everything works fine except that it won't recognise the primary2DAxis. It's really strange.

Not been able to figure if it's Valve's lacking OpenXR implementation or if it's an issue with Unity's OpenXR plugin

 

If i use the bat i get it to render on the headset and move the camara in sync with the headset, but i have not been able to get the controllers working.

 

I can confirm that thumbsticks on controllers don't work if Unity editor is run from command line. This is actually really bad for the whole workflow. Can someone please advise of any solution?

EDIT: The solution was to turn off the conflicting controller device. 3Dconnexion mouse in my case.

 

Hey guys. Got this working and even using the bat. All is working but I have no controllers. In my OVRCameraRig I do have the OVRhand prefab added, so it works normaly in Unity but not in VD mode. Any tips?

 

Ok, I got it all working and it is very nice to have controllers again.

What I am now going to do is go through different install situations with versions of unity OpenXR, OculusXR, Inputs manager, Interaction framework and all the possibilities to get the very basics what is needed, as I do not think you need all this .bat stuff and you don't even need the OculusXR in your scene.


So I will get back here at the latest tomorrow and post a really working easy solution.

And even, how I do all this without using a router. I just use a Wi-Fi stick that has Hotspot functionality and the Quest connect to that. Runs very smoothly and it is a working solution for wireless VR development. I will post that Wi-Fi adapter here later too.

 

You have to start an App via Virtual Desktop. That's tricky with the editor and only works with a build.

 

I am curious about your latency and throughput when using the hotspot. Plz post a screenshot of that data. Up to now using a router was recommended over using a hotspot given that "The hotspot creates a NAT and your home router creates another NAT so you end up with a double NAT situation which sometimes won’t allow connections through" (stated by the dev. of Virtual Desktop itself https://steamcommunity.com/app/382110/discussions/0/3435626645078755484/)

If you find an easier solution to get the Unity VR play mode debugging working using the VR Interaction Framework and Virtual Desktop you can always post it. Until then I'll keep using my .bat solution.

 

No, you don't have to create a build. Using my .bat above in combination with e.g. the VR interaction framework (I haven't tried anything else so far) you can basically tell Virtual Desktop that Unity is the app you want to start and then also the Unity play mode in combination with Virtual Desktop (so wirelessly) is working directly inside VR including headset and controller movement. A build is just an easy solution to try out your game inside VR using Virtual Desktop, but when developing a larger VR game it just takes so much time to create a build over and over again. So you should definitely try to get the play-mode working inside VR.

 

Follow instructions to the dot. Camera looks like it works but in my oculus its all black despite looking outside on my unity editor looks like its following head movement perfectly. Also controllers aren't tracking properly despite Input.getdevices seem to debug it perfectly that I'm using oculus controllers.

Unity Build Number 2020.2.0b7

EDIT: After retrying the instructions above. It actually works, just some project issues causing problems. Thanks for the insturctions.

 

Everything was working then suddenly I was getting the same symptoms as longarm123 (screen was black but I could see on my laptop that the head/hand tracking was fine). I updated the streamer, then restarted my pc and oculus and when I loaded everything back up, the editor was fully working with the headset again.

 

Mine started working in 2020.3.0f1

 

If you are stuck in the Unity Hub and/or you want to use whitespaces in your project path: The project path is passed to Streamer as an argument which then passes this on to Unity. Therefore you have to add an extra \" around your project path e.g.:

Code (csharp):

1. "C:\...\VirtualDesktop.Streamer.exe" "C:\...\Unity.exe" -projectpath "\"C:\...\Project Name\""

PS:
Thanks @dnnkeeper. Works like a charm.

 

I tested SteamVR plugin +ALVR. And the result amazing, no need for commands, just clicking play button, then editor play mode appears in my Quest 2.

 

You guys are awesome, the batch script works perfect. How do you even reach this kind of solutions? You must have great insight in both Unity and the streaming technology for the VD. The "\" is also a total new thing when it comes to batch scripting for me.
Again, thanks. I paired one of my old computers from 2010 coupled with a OQ2 with VD and it works just fine.
I used this setup to create a development platform to go through the new Create Vith VR course by Carl.D on Unity learn.

 

I was just able to get this working using the bat @johanneskopf posted and making the tweaks that @TheFoundation suggested.

I did have to take one extra step. My project is not using the Oculus Integration Kit and is instead a straight up Unity XR project. In order to actually launch into my project when I press play I had to open the Oculus app and leave it open.

 

Heya, everyone.

For months, I've been using Virtual Desktop Streamer and my Oculus Quest 2 to alpha test my game straight from the Unity editor. However, as of today, 5/5/21, the controllers no longer seem to work via this method. Head tracking is still functional, but the hands are no longer tracking nor does their button/analog stick input seem to reach Unity.

It was working as recently as yesterday.

I've already tried restarting the app and entire computer. I've already confirmed everything is working fine outside of Unity (i.e. Steam games, including a build of my own game). It is just Unity that seems to be having this problem.

Anyone have any ideas? I do not see any errors or anything in Unity.

Edit - Solution:
I discovered that somehow the input bindings profile on Steam was disabled. I don't know how that happened, but I just needed to re-enable it. So, if anyone else experiences their controllers no longer tracking nor input getting recognized, check to make sure you have an active SteamVR input binding profile.

 

used this method before through the command line and it worked fine. Now It's stopped working. Unity will show the editor launching window (the one with the graphic) for a few seconds and then it will just close. Any ideas. I haven't changed the methodology in any way.

 

This works. Thanks for the guide, as it saved some time.

However, it doesn't matter if you press "Play" from headset or from desktop. It fires up either way.

 

Any chance you could explain how to do this?? I'm not familiar with batch scripting!

 

@adrian-taylor09 I have made a written blog post here about a similar bash-script, if that helps you? https://jonassandstedt.se/blog/connect-adb-wirelessly-to-a-oculus-quest/

 

Just a note:
Have been using this bash-method for a while now. But recently I started a completely new project using the new Open XR API instead, and now I can open the project directly from the Hub without any command-line arguments! The only thing is that is opens Steam VR in between, so need to quit Steam VR after each test to get back to Virtual Desktop. But it's very fast, so not a big problem.

 

I still can't get Virtual desktop to with Unity through the batch file, maybe because I have some custom installation paths. I changed the path for Virtual desktop streamer and unity exe. However, I'm really confused with this path: "C:\Repos\proj-stardew-vr-health -cloudEnvironment", which I don't have in my files. Can someone help me identify what \Repos\proj is?