SteamVr HTC Vive weird FPS issue

Hey, i've encountered a very weird issue with htc vive and steamVr.

The setup is simple - camera and a high-res plane with custom shader. The shader works fine without vr(tested in Game View with disabled VSync):
(1000+fps) - Full Hd resolution
(400-600 fps) - at 2160x1200 (native VIve resolution)
(100-150fps) at 4k
(30-40fps) at 8k

but with Vive connected(resolution in game view is set to 960x540) I get 40-50fps only, and with camera set to render only one eye - 100fps (no Vsync).
renderScale is set to 1
And changing any quality or player settings does not make any significant difference (<5%)

Does SteamVr override any settings that might affect fps?
Thanks

 

I looked at their source on the weekend. The SDK was put together by c++ developers that know little about c# and less about unity.

It could do a lot of things better. It's the slowest VR SDK we use, I had to rewrite large portions to make it usable with anything real.

For example they iterate every camera, every frame, with nested functions using GetComponent (a slow and expensive process) and GameObject.Find (extremely expensive). Just to find references they found 1ms ago. It just needs someone who knows what they are doing to clean it up. I'm surprised more people haven't mentioned that to them.

 

Have you mentioned it to them?

 

Several times I finally reworked a chunk of their SteamVR stuff especially around Camera.Render();. I'm seeing about a 30% improvement overall in game speed. But there are big sections I still need to rewrite and lots to test.

Maybe there is a reason they did it the way they did it. Their code has been improving version over version. But there is a reason their VR SDK in Unity is the slowest of the bunch.

That said they have enough money and staff. If they care to fix it, they'll fix it.

 

I have a similar problem. Even on an empty scene, I get these dips to below 60fps about 5 times per second on the profiler. 90-95% of the CPU time is taken up by VR.WaitForGPU. The GPU usage says up to 22ms but the overview doesn't even add up to 1 ms. What would cause such periodic drops in frame rate?

 

First, I need to preface this by saying I'm not a fan of SteamVR. I love our HTC Vive, but their SDK ... yeah. I'll leave it at that.

VR.WaitForGPU, is when it sends something off to the graphics card, and waits to hear back. It was something new they added for 5.4 in conjunction with the Unity folks, and neither Unity nor the SteamVR team have been willing to put in the time or effort to give us details on it ... Best guess, is the GPU is taking its sweet time rendering / working with something.

I'm seeing that same thing on our R9 card and capping even empty scenes at 60 FPS.

 

So after seeing lloydsummers posts I decided to poke into SteamVR, and make sure I was understanding the issues he was having. It does seem as though the package works very differently on Unity 5.4 with the latest SteamVR version(1.1.1). This is good news for me as my current project is working on the latest version, but I am not actually certain if this configuration has better performance. I would be intrigued to know what pain points might still exist and I'd really like to know if there are ways we(Unity SteamVR users) can squeeze more performance out.

 

Good idea, I'm working around the clock for a deadline for mid-september. I'm seeing some abnormalities on 5.4, not sure if it is my changes, or Unity and I can't really investigate for a while.

Once I'm caught up, I'll clone the repo and share the link. Right now its embedded in a project and I don't have time to cut it out.

I'm seeing some strange behaviours with Unity 5.4.

But for performance changes, some things to suggest that worked well for me. Take a look at any of the loops using GetComponent or FindGameObject --- those are relatively expensive operations to do on Updates or Couroutines with WaitForFPS, and do repeat the process every single frame is just poor planning (although it is more defensively coded). In addition, the couroutine they are running, can be accessed different ways, and technically a LateUpdate could suffice and gives a bit more of a performance improvement over CoRoutine using WaitForEndOfFrame (based on my testing). Some of the matrix stuff they are doing on the inner could actually be offloaded to a separate CPU thread, which I haven't dug until yet, but there is a lot of calculations that could be offloaded elsewhere.

Right now, they are doing everything under new RenderTexture.

If you want to maximize Unity performance, it should actually be RenderTexture.getTemporary. On that note, the way they are using RenderTexture, I've tried to do that in the past for commercial medical applications as a screen-capture source, and found it to behave intermittently on Windows / Mac and Linux and browser changing when using WebGL. Because they are Windows focused, non-browser based, they haven't bumped into that. But they will if they decide to one day support more platforms.

To clarify, it isn't anything wrong - not like "shame on the SteamVR team for doing this" - just improvements they don't know about because they aren't used to Unity yet IMO. If they really wanted to propel it forward, they should throw some money at the Sensics guys, or another team that has built some really transformational stuff in Unity VR space. You should have seen the earlier SteamVR stuff, that was straight C++ that looked like a program converted it to C#.

 

This is not great, as always the users muts put the effort on plugins because Unity doesn't make enought.

 

I can also confirm a drastical drop in performance (HTC Vive, HTC Unity plugin) against the Oculus (DK2, Unity native VR support).

 

Any news??
@ lloydsummers Can you share your modified SteamVR??
Regards and thank you

 

My SteamVR is far deviated and buried in the project. I've now wrapped it into 3 other VR SDK's that the Unity team hasn't bothered with to support - including non-VR. Kind of a giant inhouse input and display system.

Most of my work is going to be contributed back to OSVR.

Currently we are still under fire for some rough deadlines --- so I haven't had time to carve it out and make it nice. But it is on my todo list. If our timelines don't deviate, most likely end-of-month-ish.

On that note - use the VRTK as well and check for the actual FPS during play. Unity Editor VR testing is currently unusable in 5.3 + 5.4 with any type of serious sized game. 5.5 id say is somewhat functional in editor, but extremely dangerous beta to upgrade to.

Speculation: It's a shame because HTC started strong and innovative. But once launched, they transitioned everything to some ops teams and dropped the ball badly both in development and on customer facing teams. Based on what I'm seeing, they are preoccupied with a new vive predecessor. They obsess on video card specs but their true bottleneck to me, with SteamVR, appears to actually be cpu and sequential operations. Unity collected a multimillion dollar cheque to take on SteamVR natively, but outside of some basic performance changes, seems to have done little to actually meet the commitment so far.

 

It's a real shame so little work is done on drivers and software.

 

hi all. My research pointed me here, and I am wondering if my problem is related. See imgur screen shot of my profiler:
http://imgur.com/a/mmOnJ

The game runs totally fine most of the time, and then every so often, without any sign of changes in cpu or memory, the gpu gives a bunch of huge spikes. Nothing like 50+ms of wait-for-GPU to get you sick in vr.

We have tried a lot of things, about to try lab renderer. If that fails, we might ditch and go to Unreal.

The unity profiler is great at profiling scripts, but terrible with GPU issues: just a giant blob of "other" and "vsync" even though vsync is set to off.

Any thoughts?

 

I've also found some bad implementation in Unity's SDK. Garbage is implemented every frame if you load the controllers with dynamic enabled. I ended up finding the OBJ file of the controllers and extracting them into my game to stop the garbage generation. Turn off "Update Dynamically" on SteamVR_RenderModel. The foreach loop happens in there.

I've also noticed running Unity's profile has an impact on performance. Turning that off seems to stabilize the number of late starts to almost nil. And I've also found that often a physics update will need to be run twice which then pushes the wait for GPU past the next frame and causes a skip.

I am still not entirely sure if WaitForGPU is set by SteamVR, Unity or something else. It seems to be fairly consistent and may be set by SteamVR's api. I believe it waits till the last second before sending the info to the GPU for the next frame. Here is a great GDC talk on it. http://www.gdcvault.com/play/1021771/Advanced-VR.

Or it may be that my render thread is just too long to have two physics calculations done within the few milliseconds each round.

And I've found Unity 5.4 works pretty well with SteamVR natively. Hope that Unity 5.5 is even better.

 

Good tip and point.

 

Found some more garbage reduction places. SteamVR_UpdatePoses.cs creates 88 bytes of garbage every frame in order to call a set of events to update the pose. It ca

SteamVR_Updateposes uses events and calls params object[] args. Calling params object[] args will always create a new array. In SteamVR_UpdatePoses, every round a new event is sent to update the poses of the tracked objects.

There is no easy way to fix this. The listeners are all tied into using generalized parameters object[] args which is not good memory management. I bashed my head against it for a while, but I am not great at events (I don't really like them). Anyone else who is smarter than me figure out a solution to this?

 

Figured it out. I had to duplicate the event handler.

Duplicate the Event class inside SteamVR_Utils.

Code (CSharp):

1.   public class TrackedObjEvent
2.     {
3.         public delegate void Handler(TrackedDevicePose_t[] poses);
4.  
5.         public static void Listen(string message, Handler action)
6.         {
7.             var actions = listeners[message] as Handler;
8.             if (actions != null)
9.             {
10.                 listeners[message] = actions + action;
11.             }
12.             else
13.             {
14.                 listeners[message] = action;
15.             }
16.         }
17.  
18.         public static void Remove(string message, Handler action)
19.         {
20.             var actions = listeners[message] as Handler;
21.             if (actions != null)
22.             {
23.                 listeners[message] = actions - action;
24.             }
25.         }
26.  
27.         public static void Send(string message, TrackedDevicePose_t[] poses)
28.         {
29.             var actions = listeners[message] as Handler;
30.             if (actions != null)
31.             {
32.                 actions(poses);
33.             }
34.         }
35.  
36.         public static void Send(string message)
37.         {
38.             var actions = listeners[message] as Handler;
39.             if (actions != null)
40.             {
41.                 actions(null);
42.             }
43.         }
44.  
45.         private static Hashtable listeners = new Hashtable();
46.     }

Go into SteamVR_UpdatePoses.cs and change the current Event.Send to the code below.

Code (CSharp):

1.               SteamVR_Utils.TrackedObjEvent.Send("new_poses", render.poses);
2.             SteamVR_Utils.TrackedObjEvent.Send("new_poses_applied");

Now you'll need to find every place where "new_poses" or "new_poses_applied" is added as a listener or removed and update it to use the new TrackedObjEvent. They are currently in SteamVR, SteamVR_TrackedObject, and SteamVR_Ears. Note, any other 3rd party systems that use Steam's weird event system will need to be updated too (Like NetwonVR). You may have to duplicate some functions if they aren't playing nicely with the new event system.

Hope that helps! Its a temporary hack and I hope the SDK is updated to be garbageless in the next version.

Edit: Formatting for code

 

Thanks.
It would be nice if Unity guys could take a look at your solution to improve next versions

 

Found the last bit of garbage and I don't know if there is a solution for this. It relates to private IEnumerator RenderLoop() in the SteamVR Renderer. SteamVR is starting a couroutine (which is fine) except that couroutines create 16 bytes of garbage per use. I tried moving it to the late update, but I was getting some bad juddering and I am not sure the reason they wanted to use a Couroutine. Maybe to ensure that this function is fired right as the frame ends?

If Unity fixed the issue where Coroutines did not generate garbage (which I found a reference to) everything would be groovy. @lloydsummers where did you move the RenderLoop to? Is there a better way to call the renderloop?

 

Nevermind. I fixed it! You need to stop Unity from generating a new WaitForEndOfFrame every frame.

Code (CSharp):

1.  
2.     private WaitForEndOfFrame waitingForEndOfFrame = new WaitForEndOfFrame();
3.  
4.     private IEnumerator RenderLoop()
5.     {
6.         while (true)
7.         {
8.             yield return waitingForEndOfFrame;
9.  
10.             if (pauseRendering)
11.                 continue;
12.  
13.             var compositor = OpenVR.Compositor;
14.             if (compositor != null)
15.             {
16.                 if (!compositor.CanRenderScene())
17.                     continue;
18.  
19.                 compositor.SetTrackingSpace(trackingSpace);
20.             }
21.  
22.             var overlay = SteamVR_Overlay.instance;
23.             if (overlay != null)
24.                 overlay.UpdateOverlay();
25.  
26.             RenderExternalCamera();
27.         }
28.     }

You can call me the Recycle man because you don't need that garbage!

 

Nice

One small item...

Instead of assigning compositor, and then assigning overlay, move the references up one level to "carry" those between frames.. Please note as well, I didn't actually test the code above. But this way, only when those references are null, then assign them.

I changed a lot of those all over the place in my SteamVR hybrid for anything running on a loop, and it seemed to help a lot.

Code (CSharp):

1.  
2.     private WaitForEndOfFrame waitingForEndOfFrame = new WaitForEndOfFrame();
3.  
4.     private IEnumerator RenderLoop()
5.     {
6.  
7.         var overlay = SteamVR_Overlay.instance;
8.         var compositor = OpenVR.Compositor;
9.  
10.         while (Application.isPlaying)
11.         {
12.             yield return waitingForEndOfFrame;
13.  
14.             if (pauseRendering)
15.                 continue;
16.  
17.             if (compositor == null)
18.                 compositor = OpenVR.Compositor;
19.  
20.             if (compositor != null)
21.             {
22.                 if (!compositor.CanRenderScene())
23.                     continue;
24.  
25.                 compositor.SetTrackingSpace(trackingSpace);
26.             }
27.  
28.             if (overlay == null)
29.                 overlay = SteamVR_Overlay.instance;
30.  
31.             if (overlay != null)
32.                 overlay.UpdateOverlay();
33.  
34.             RenderExternalCamera();
35.         }
36.     }

 

Hmm, I assumed that SteamVR was assigning those every round for a reason. You are saying you can cache it successfully? I noticed that this is used everywhere in the SDK. I just assumed they needed to reassign it every frame. How much performance improvements did you see from this? A ms?

 

Yup! We did a tradeshow recently, with the game running steadily for ~6 hours without any issues, including needing to be restarted, and we use just about every SteamVR command in the book. They might do it for some strange edge case though - its the mass load of changes like that that deviated my SteamVR so badly.

Where you also want to look is where they have used things like GetComponent. It all adds up! As you work through, you'll find have more nested methods than it first appears. Some are more expensive than others.

I made all my changes at once over a period of about a week, while hybriding it with all our other VR frameworks. But all-in-all, we went from ~15FPS to +90. Not all of that was SteamVR, I did an end-to-end performance cleanup and walked away when we go the FPS I needed.

 

Cool. Looking through the latest version of the SDK, they definitely removed the get component problems. Although I think the example scripts are not well optimized still. Doesn't matter as I wrote my own input methods mostly.

I think I'll hold off on major changes. Currently the speed is pretty good once I got rid of those garbage allocation issues. What version of the SteamVR SDK did you diverge from?

 

Hi 8bitgoose,
can you "pull request" your code to main github SteamVR?
Is this the code: https://github.com/ValveSoftware/op...Assets/SteamVR/Scripts/SteamVR_Render.cs#L153 ?

I think that is very important for all us!
Good work!

 

@Vytek Maybe I can? I looked at the pull requests and there are already 15 in there. Not sure how active they are on accepting submissions from outside sources. I also am not using this repo to update my Steam SDK. I'd have to do a bunch of work to make the pull request happen.

Good news is they updated it 7 days ago. Bad news is the garbage generation is still there.

I did make a post on SteamVR's hardware developer page. Hopefully they take notice. https://steamcommunity.com/app/358720/discussions/0/341537671986065508/

 

@8bitgoose I link you explanation in a issue that I start: https://github.com/ValveSoftware/openvr/issues/280

I hope this helps too!!

Now I am trying to contact Joe Ludwig (I think that it is one of the owner of github repo) on Valve main site. I think that you make a superb works! (I am using this link: http://valvesoftware.com/email.php?recipient=Joe+Ludwig)

What repo do you use for Steam DSK? Could you post your version, I am trying to patch my SteamVR SDK but it doesn't works :-(

Thank you for your help.

P.S.: I believe in OpenSource model, SteamVR will be better ;-)

 

Awesome. I hope that works out.

I am using the SDK right off the Asset store. My SDK has also had the Unity 5.3 and below text stripped out. I don't know if any of the changes I've made will impact pre Unity 5.3 stuff. I'd rather not go to the trouble to fix anything below 5.4 as it doesn't affect me.

Where did you run into an issue?

 

I create an issue on github openvr repository linking all your post. Please see this: https://github.com/ValveSoftware/openvr/issues/280

Developer aleiby was assigned by JoeLudwigan hour ago to this issue ;-)

It is time to post all your patch ;-)

 

No news :-(

@8bitgoose @lloydsummers

Can you create a patch or share your code? The problem still occurse in Unity 5.5 :-(

Please let me know. Thank you in advance.

 

I have created a fork and I recreate your code for pull request on SteamVR repo.

Please visit: https://github.com/Vytek/openvr/blob/master/unity_package/Assets/SteamVR/Scripts/SteamVR_Render.cs

Any help is appreciated!!

 

I would also appreciate the optimized SteamVR code a lot. I think it would really help everyone. Specially people too new to programing (like me) to do this kind of major clean up.

 

The SteamVR plugin is getting some update love soon it seems:
https://steamcommunity.com/app/358720/discussions/0/341537671986065508/#c142261027580807962

 

Great news!! @8bitgoose @lloydsummers

Please help us!! I create a pull request with poi .3 of your post:

3. The render loop (which is generated every frame) creates a new WaitForEndOfFrame() object. Instead create a single class at the start and reference that. Check the link for all the details.

https://github.com/ValveSoftware/openvr/pull/360

 

I've taken a quick scan through on the repo linked above, it seems fairly solid to me... I can see a few spots where it can still be optimized, but I'm not convinced it will help much. It looks much cleaner than what we (Red Iron was locked to an old SteamVR build) were running with when I posted.

Some spots that could still help though for example with the GetComponent... SteamVR_Camera during creation needs to find its own GetComponent<Camera> [line 153] then throws away the reference. However, going forward SteamVR_Render iterates through RenderLoop, which feeds to RenderEye, which is doing multiple GetComponents per-frame back to the SteamVR cameras [line 235]. If that reference is stored, all the better.

Again, just mentioning that RenderTexture GetSceneTexture is running per-frame, and returning a new RenderTexture. When we messed with it, I moved it over to RenderTexture.GetTemporary(). But again, note it causes behavior issues on WebGL for iOS, so comes with a cost.

https://docs.unity3d.com/ScriptReference/RenderTexture.GetTemporary.html

When I refactored our package (just to mention - we don't use our hybrided SteamVR anymore - in 5.5beta our performance was greatly improved except in editor play mode) I cleaned up a lot of the redundant assignments in the unlikely event that Unity was doing some getter/setter magic behind the scenes. It is a long shot without digging into the Unity source code again and reviewing their internal methods, but may have had an influence. For example, SteamVR_CameraMask.Set, gets executed twice a frame from SteamVR_Render [line 225] even if nothing changed.

My theory at the time was this - in VR - RAM seems cheap, CPU/GPU seems expensive. So I'd rather store a reference (even overly abundantly or at risk of carrying a bad reference) than executing query. And not even perform an assignment or query if the value hasn't changed. I think it was gross overkill though, our biggest performance increases were likely from our own non-steamvr refactorings.

 

Looks like the SteamVR plugin has been updated with my garbage collection suggestions. Thanks everyone for your help especially @Vytek who put the github pull request together (because I was too lazy to do it). And also @lloydsummers who took a look at it as well.

Again @Vytek, sorry I couldn't do more. Have been working on a huge feature and didn't have the energy to put together a pull request. You the real MVP.

 

I would like to thanks @8bitgoose for your superb code/test and @lloydsummers for advise and help! 8bitgoose can you test the new version with profiler? Is really better? I hope so ;-)

Now there is a new repository for Unity SteamVR Plugin: https://github.com/ValveSoftware/steamvr_unity_plugin

http://steamcommunity.com/app/358720/discussions/0/142261352645468158/

Changes for v1.2.0:
* Updated to SteamVR runtime v1481926580 and SDK version 1.0.5.
* Replaced SteamVR_Utils.Event with SteamVR_Events.<EventName> to avoid runtime memory allocation associated with use of params object[] args.
* Added SteamVR_Events.<EventName>Action to make it easy to wrap callbacks to avoid memory allocation when components are frequently enabled/disabled at runtime.
* Fixed other miscellaneous runtime memory allocation in SteamVR_Render and SteamVR_RenderModels. (Suggestions by unity3d user @8bitgoose.)
* Integrated fix for SteamVR_LaserPointer direction (from github user @fredsa).
* Integrated fixes and comments for SteamVR_Teleporter (from github user @natewinck).
* Removed SteamVR_Status and SteamVR_StatusText as they were using SteamVR_Utils.Event with generic strings which is no longer allowed.
* Added SteamVR_Controller.assignAllBeforeIdentified (to allow controller to be assigned before identified as left vs right). Suggested by github user @chrwoizi.
* Added SteamVR_Controller.UpdateTargets public interface. This allows spawning the component at runtime. Suggested by github user @demonixis.
* Fixed bug with SteamVR_TrackedObject when specifying origin. Suggested by github user @fredsa.
* Fixed issue with head camera reference in SteamVR_Camera. Suggested by github user @pedrofe.

See you soon!

 

I shall test it when I get around to it (which may be some time )