Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug Why is the new input system so slow?

Discussion in 'XR Interaction Toolkit and Input' started by funkyCoty, Feb 25, 2022.

  1. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    On a quest 2 game, it's managing to eat up more CPU time than the entire player loop. It's almost as much time spent as the entire rendering pipeline?! What on earth is going on here? How can we resolve this?

    Unity_d04Ffe7zlC.png Unity_tF5V9iUFZr.png
     
    Deleted User and Vaupell like this.
  2. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hi @funkyCoty,
    Are you able to provide a more zoomed in view of one of those icicles for us to take a look at? It would be helpful to get the Hierarchy view instead of Timeline (there is a dropdown to the left below the Profiler Modules). This can provide a deeper drill-down of the functions being called and will help us determine what is going on.

    You can also try importing the Profile Analyzer package to help dig a bit deeper.
     
  3. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
  4. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    Hierarchy above ^
    But wanted to note, of course it's not nearly as bad in non deep profile builds. Still eats up a decent chunk of time that should otherwise not be eaten up, though. It also seems that any call to read any inputs are fairly slow.
     
  5. Riscii

    Riscii

    Joined:
    Nov 14, 2012
    Posts:
    4
    I'm seeing similar profiling issues with our implementation of the new input system mapping paired with Oculus XR Plug-in. It seems to be providing two pose updates per device per frame. It takes over 2ms of our 11ms budget when profiling on the quest 2.
     
    weizgit and funkyCoty like this.
  6. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hi @funkyCoty,
    Can you re-run the profiler against a release mode vs debug mode? We are curious if you still see the same delay. You can enable release mode by clicking the little 'bug' icon in the bottom right and then clicking the Release Mode button. Here's more information about this: ManagedCodeDebugging
     
  7. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    I was profiling from an actual quest 2.

    In the editor, it's already in release mode.
    6Yj1sHnxDF.png Unity_KbRcs5gZJD.png
     
  8. meisi11

    meisi11

    Joined:
    Jul 18, 2016
    Posts:
    1
    We're also running into the same problem, and at this point are wondering if we have to ditch the input system entirely. Are there any fixes?
     
  9. emrys90

    emrys90

    Joined:
    Oct 14, 2013
    Posts:
    755
    Any news on this? I'm having the same performance issues on a Quest 2. I really do not want to have to switch away from the new Input System
     
  10. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    In our quest 2 game, we ended up just removing the input system entirely. Just using the oculus api directly instead (~3ms -> ~0.00ms on profiler to do the same stuff). The input system was eating up more time than all of URP and Unity has not given any sort of hints as to why or how to fix it.
     
  11. kalineh

    kalineh

    Joined:
    Dec 23, 2015
    Posts:
    240
    You just want the OVRPlugin.GetControllerState4() or OVRPlugin.GetNodePoseStateRaw() calls to get position & button state, you just need to flip the position values (+x,+y,-z) and the orientation values (-x,-y,+z,+w) because it's not the same space as unity (that includes velocity and angular velocity).
     
    hippocoder likes this.
  12. DriesVrBase

    DriesVrBase

    Joined:
    Mar 24, 2020
    Posts:
    63
    @VRDave_Unity Any updates by any chance? Having the same issue
     
  13. ErThu

    ErThu

    Joined:
    Jul 1, 2015
    Posts:
    4
    Same issue here, we're considering not using the new Input System for Quest because of the huge performance hit it takes.
     
  14. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hey all,
    We are definitely looking into this issue. We don't want people blocked from using the new Input System for VR development. I will post back here when we have news to share about the status of this.
     
  15. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hey everyone,
    I have some results to share from our Input Team. They ran through a number of different build configurations on Quest 2. Here are the results for running against our XRI Samples project with Unity 2021.3.4f, Input System 1.3.0, and XRI 2.0.2:
    • Mono Release + Script Debugging + Deep Profile :
      • Input System: ~3.0-4.0ms
      • XRI: ~7.0-9.0ms
    • Mono Release + Script Debugging + No Deep Profile :
      • Input System: ~0.6ms
      • XRI: ~1.0ms
    • IL2CPP Release + Script Debugging + No Deep Profile :
      • Input System: ~0.3-0.5ms
      • XRI: ~0.6-1.0ms
    • IL2CPP Release + No Script Debugging + No Deep Profile :
      • Input System: ~0.06ms
      • XRI: ~0.14ms
    So as you can see, with deep profiling turned on, the results are higher than expected. When checking perf, Script Debugging seems to add a lot of overhead (We'll be looking into some options for optimization here), so unless you absolutely need it for some reason, we recommend turning it off for your perf runs. You can also see a big gain with IL2CPP builds (almost 10x gain).

    Hopefully this provides a bit of guidance when looking at performance numbers and some ways to interpret what we are seeing.
     
  16. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    I really don't what else to tell you, but these numbers just seem unrealistic from what I've seen in a real project. IL2CPP debug + no script debugging = was slower than the entirety of URP's render loop. 3ms was the fastest I've seen it. I don't know if we hit some edge case (where others in the thread seem to have hit it too) or what. All I can say for sure is after we ripped out the unity input system and just used their input api directly, we have not had any input performance issues.
     
  17. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hey @funkyCoty,

    Are you able to share the script that does the
    NewInputUpdate
    and
    NewInputBeforeRenderUpdate
    calls? We would love to dig deeper on a more appropriate real-world example. If you are unable to share on a public forum, feel free to message me and we can connect another way such as email.
     
  18. Mbenzacharia

    Mbenzacharia

    Joined:
    Nov 16, 2021
    Posts:
    1
    in my project i noticed that it only happens when two quest controllers are connected/active. with only 1 controller i see normal frame times. when two controllers are connected the per-frame time goes up to about 25ms...
     
  19. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hey @Mbenzacharia,
    Does this happen regardless of which hand is connected first (right hand first vs left hand first)? Are you able to share your
    .inputactions
    asset files for us to take a look at? This should definitely not be the case. Are you using your own code to handle the inputs or the XRI built-in:
    XR Controller (Action-based)
    ?
     
  20. theinlein

    theinlein

    Joined:
    May 29, 2021
    Posts:
    4
    @VRDave_Unity I'm curious if any headway has been made on this. We're also seeing the same (on a Quest 2 project). The unity input system being non-viable for standalone VR headsets is... disappointing.
     
  21. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hey @theinlein,
    How many input bindings are you currently using? We have seen performance issues where there are a lot of input bindings active at the same time. Have you tried turning off Script Debuggin, turning off Deep Profiling and building with IL2CPP ? As shown above, this helps dramatically improve input performance.

    That being said, we have been working on some tooling around this area of the code to help track down the performance problems and make improvements but also to ensure future updates do not cause a performance regression. I hope to share more news soon.
     
  22. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    We ripped it out of our project pretty quick, so no I don't have this stuff on hand. However if you email us (hello@funktroniclabs.com) i think @kalineh will be able to get you more information, maybe check back farther in git to get the scripts you want to reference.
     
    Walter_Hulsebos likes this.
  23. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    Update: We're (unfortunately) using the New Input System for the PC version of our Quest 2 game, still. We had reports along the lines of "after a few hours of playing, the game's framerate becomes unbearably slow," so of course I looked into it ASAP. What a surprise, once again it was the input system biting us in the butt. Here's what I uncovered.

    Take a look at this short video.



    This is me scrubbing through a unity profile timeline of snapshots taken across the game over about 30 or 40 minutes of playtime. In a non deep profile in a debug build, things start out okay on PC. Roughly 0.1ms or so. Way too high, but you know, it's manageable. Towards the end, we're looking at well over 1ms over overhead. Something is up. After some debugging via modifying the input systems source (looks like we're on version 1.5.0) - I noticed that the internal list of "maps" was growing in size. There were a LOT of "null" entries and a lot of duplicates of the VR HMD and controllers. So, I modified the following lines:

    upload_2023-2-13_17-19-28.png

    This seems to have resolved our "performance degrades over time" issue entirely, as long as I call DisableAllActions occasionally (loading screens). Yay! Insane that I had to do so much debugging to find this, but I'm glad. However, I'm still pretty upset that it's eating up 0.1ms so I started debugging that a bit too.

    What I found was that most of the time was spent in DeferBindingResolutionWrapper's Dispose() function. It looks like every single action callback has a using(DeferBindingResolutionWrapper), which gets auto disposed. Unfortunately in that Dipose it seems to be handling input rebindings... every... time??????

    So I made the following change:

    upload_2023-2-13_17-21-30.png

    With this one change, the input system's update is now ~0.05ms, which is still too slow but I'm a lot happier about.

    upload_2023-2-13_17-22-18.png

    I wish I didn't have to spend so much time debugging this. I lost an entire day of work just to make a few changes; but hey I'm glad our game is going to run faster and longer for many players now.
     
  24. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Hey @funkyCoty,
    Thank you for finding this and posting all this info, we really do appreciate the effort and diagnostics you have done on these performance issues. I will say that this has been a hot topic the last couple of days here and we are getting all the right people involved to look into it. Could you please file an official bug report (in Editor go to Help > Report a Bug...) with your findings and anything else that might be helpful for us to reproduce on our end, such as logs or information related to scene loads or setup/teardown of devices. This will help escalate the issue and get it resolved sooner.
     
    glenneroo, Arnold_2013 and kalineh like this.
  25. Shizola

    Shizola

    Joined:
    Jun 29, 2014
    Posts:
    467
    mgear likes this.
  26. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    Shizola likes this.
  27. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    709
    I'll try to when I find a chance. Honestly we're trying to ship a game (live this wednesday!!) on quest/ps5/pc all at the same time so we're actually incredibly swamped at the studio right now. Maybe after the launch stuff calms down I might find a chance to do this. The only reason I dug so deep was because our PC beta testers were having serious issues.
     
    ImpossibleRobert likes this.
  28. VRDave_Unity

    VRDave_Unity

    Unity Technologies

    Joined:
    Nov 19, 2021
    Posts:
    258
    Thanks for the update. It looks like you guys made it over the line, congratulations! The game looks fantastic by the way, we've been excited about it over here.

    As far as the bug you discovered is concerned, I have created an internal ticket and escalated this forum post to our Input team and the XR Foundations team, so both sides are taking a look at it, making sure we're using the Input System APIs the right way from the XR side and fixing things from the Input System package side as well.
     
  29. ImpossibleRobert

    ImpossibleRobert

    Joined:
    Oct 10, 2013
    Posts:
    517
    Congratulations on the launch! This looks really great. Reviews seem great and looks like an overall success. I hope the sales are what you hoped for :)
     
    kalineh, VRDave_Unity and funkyCoty like this.
  30. emrys90

    emrys90

    Joined:
    Oct 14, 2013
    Posts:
    755
    Any news on this?
     
  31. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    IMO the problem with the Input System is overengineering. Those call stacks are far too deep, with layers upon layers of unnecessary abstraction between reading the low level input and it finally informing the game about it. This is the kind of code that shows the weakness of IL2CPP: deeply nested virtual/delegate/interface calls prevent a lot of compiler optimizations due to the extra cruft needed by the dotnet execution model.
     
    Unifikation likes this.
  32. peter_hall

    peter_hall

    Unity Technologies

    Joined:
    Sep 3, 2019
    Posts:
    15
    Hi! I work on the Optimization team and I've recently been starting to investigate this. I haven't yet found any projects using the input system that show this slowdown, and I haven't been able to build a sample that shows the problem either. If anyone here has a sample project that shows this slowdown it would be really useful if you could share it (ideally by filing a bug with the project attached, or send me a DM) so that I can try and fix the problem.
     
  33. Arnold_2013

    Arnold_2013

    Joined:
    Nov 24, 2013
    Posts:
    273
    I had a problem with the input system performance for VR and when I found this thread I thought it was related. I've just created a bug report (IN-42801). But while making the small demo sample it looks like a different issue. My problem seems to come from not disabling inputs when switching scenes, which resulted in a 17ms "PreUpdate.NewInputUpdate" after a few scenes loads :confused:.
    I post this here because I don't understand enough about the input system to definitely say its a completely unrelated problem.
     
  34. peter_hall

    peter_hall

    Unity Technologies

    Joined:
    Sep 3, 2019
    Posts:
    15
    This does sound like it could be related - we'll try testing with your sample and see if we can work out what's going on. Thank you for reporting it!
     
    ericprovencher likes this.
  35. Arnold_2013

    Arnold_2013

    Joined:
    Nov 24, 2013
    Posts:
    273
    After doing some more tests I found out that just having many components getting their own input in code destroys the performance. 50 components all getting position and rotation of the left and right controller would cost me ~1ms per frame, even without a scene reload and correctly disabling the input. While they should all get the exact same values (input mode is on dynamic, so I am not manually repolling the newest values from the device, as far as i know.)

    I've added this as a comment under the Bug report, since its mostly the same setup.
     
  36. ericprovencher

    ericprovencher

    Unity Technologies

    Joined:
    Dec 17, 2020
    Posts:
    219
    Have you tried using the caching and fast path optimization flags that shipped with 1.5? The caching optimization in particular should alleviate the 50 component issue, but I’m curious to see how it affects your scene in practice.
     
  37. Arnold_2013

    Arnold_2013

    Joined:
    Nov 24, 2013
    Posts:
    273
    The package manager says I am on version 1.5.1 of the Input system.

    I've added a GameObject to the scene with a component with this code :

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using UnityEngine.InputSystem;
    5.  
    6. public class ApplyPerformanceInhancerCodeForInputSystem : MonoBehaviour
    7. {
    8.     private void Awake()
    9.     {
    10.         Debug.Log("Optimise!!");
    11.  
    12.         InputSystem.settings.SetInternalFeatureFlag("USE_OPTIMIZED_CONTROLS", true);
    13.         InputSystem.settings.SetInternalFeatureFlag("USE_READ_VALUE_CACHING", true);
    14.         InputSystem.settings.SetInternalFeatureFlag("PARANOID_READ_VALUE_CACHING_CHECKS", true);
    15.      
    16.     }
    17. }
    18.  
    From the changelog + twitter it seems like this can be run anywhere to activate... Or should I run it in the input script or at a special time? It looked like a global setting.

    The "Optimise!!" from the debug line is shown in console, but nothing else.


    The performance looks the same, not better not worse. Please let me know if I need to apply this code in some other way, since it does not seem to do anything now.

    When I look in the "Optimized" column of the Device Debugger I do see + signs. (here is a screenshot)
    The scene in the bug report has the input of the controllers Position and Rotation set to PointerPosition and Pointer Rotation. These have a '+'.
    upload_2023-6-3_21-44-54.png
     
    Last edited: Jun 3, 2023
    ericprovencher likes this.
  38. ericprovencher

    ericprovencher

    Unity Technologies

    Joined:
    Dec 17, 2020
    Posts:
    219
    I would remove the paranoid value check, I think that one slows things down for debugging.

    Indeed if the + is there the control is optimized, it should work on all controls that don’t have any processors on them.

    Thanks a lot for providing information.
     
  39. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,073
    @ericprovencher is there any chance we users could get a low level, absolutely minimal input system capable of optimal levels of low latency and MIDI input recording friendly rates of polling?

    Upon this, all sorts of performance critical input could be managed.
     
    ericprovencher likes this.
  40. Arnold_2013

    Arnold_2013

    Joined:
    Nov 24, 2013
    Posts:
    273
    I've commented out the paranoid check. I've reimported the Input System (just in case). I've restarted Unity.

    Performance is roughly the same, ~1ms per frame for input.

    Sorry, I know this is not what anyone wants to hear when there is code in place that should make this case work. And it might still be something that I am doing wrong. So hopefully you (or someone at unity) can reproduce the bug from the bug report and fix this, or tell me what I am doing wrong. Really appreciate the direct communication.
     
    ericprovencher likes this.
  41. DaveLloyd

    DaveLloyd

    Joined:
    Nov 15, 2011
    Posts:
    21
    I'm in the middle of a port from PC/SteamVR to the Quest 2 and I'm using the new InputSystem with OpenXR as this seemed to be the recommended approach. I'm using Unity 2022.2.18.

    I'm seeing the same issues as above: when viewed with the profiler, I can see substantial chunks of time taken out by the input system:
    * 2.7ms in PostLateUpdate.PlayerSendFrameStarted > FrameEvents.NewInputBeforeRenderUpdate > NativeInputSystem.NotifyUpdate
    * 2.7ms in PreUpdate.NewInputUpdate > NativeInputSystem.NotifyUpdate

    As a quick hack I added a button to disable inputs through the InputActionManager. When clicked, framerate went up to 72fps and the profiler shows the scripts taking about 5ms less time.

    As others have noted, this is quite a performance killer!

    I really don't want to have to port onto the old Input system or use the Oculus specific inputs but is there anyway to make the new input system performant? Is it a matter of having too many actions, too many maps, too many listeners, ...? I've mostly followed the same pattern as I used with SteamVR which was perfectly performant on a PC at least.
     
    Unifikation likes this.
  42. ericprovencher

    ericprovencher

    Unity Technologies

    Joined:
    Dec 17, 2020
    Posts:
    219
    do you have a repro project to test this change with? We’re always looking for real world benchmark projects to help test with.
     
  43. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,073
    Are you going to reason about and analyse the performance of the new input system without a repro?

    This has been a problem for VR devices since the release of the new input system, and widely discussed. I'm yet to see anyone from Unity even admit there might be an issue, let alone that they're going to investigate the call stack and processes of the new input system.

    Please do!
     
  44. ericprovencher

    ericprovencher

    Unity Technologies

    Joined:
    Dec 17, 2020
    Posts:
    219
    The input optimization flags were an attempt to improve things at the lower level. The optimized controls flag specifically reduces the call stack for controls that do not use any processors.

    The challenge with this problem is there are many ways to use the input system, and our internal test projects don't take up anywhere near 1ms for input processing and updating. If that's what you're encountering with your project, we're happy to take a look and help identify which botlenecks you're running into so as to better address them.

    Also, filing bugs with clear reproduction cases does help a lot.
     
  45. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,073
    Please, do you think you could get someone at Unity to audit, reason about and then explain the absolute most optimal pathways for usage of The New Input system? For touchscreen devices with polling rates above 120Hz and all VR applications it is, I think, essential that the engine both provide (and explicitly detail) how to optimally get at all forms of input on low power, high frame rate and low latency environment devices - ie VR/XR and mobile.

    At the very least, show us your internal testing rigs so that we can see how you're getting timings you're happy with, and so that we can see what's the ideal rigging to use, as envisaged and utilised (by Unity) of your New Input System, and the latencies/rates you've considered "good enough".

    Speaking just from a design perspective, it's imperative to understand exactly what's the best possible input resolution, latency and rates. Please provide some of this insight into The New Input System!

    VR is about to take a huge leap forward, and many (including me) are now going to be trying to create the optimal experience knowing that Apple's entered the chat.
     
    Last edited: Jun 5, 2023
    OGMagi, Arnold_2013, Shizola and 2 others like this.
  46. DaveLloyd

    DaveLloyd

    Joined:
    Nov 15, 2011
    Posts:
    21
    I'll see if I can produce a slimmed down repro - I can't really send the whole project I'm working on (it's big and has quite a narrow path to test so far).
     
  47. DaveLloyd

    DaveLloyd

    Joined:
    Nov 15, 2011
    Posts:
    21
    I've made a surprisingly small change and reduced the timings to 1ms in each phase. So the input system is now consuming a little over 2ms instead of most of 10ms.

    I had been loading the InputActionAsset everywhere I wanted to use it and then selecting the ActionMap I wanted to bind into. However making that a static load so the InputActionAsset is a global singleton has massively reduced the time spent handling input.

    I had loaded the asset like that to make it easy to discard a load of bindings when changing scene or context so I'll now need to manage that. Are there likely to be any other issues with keeping the action asset live forever?

    Also, I think I've got at least one other instance as some bindings are still using the XRI Default Action asset as opposed to my own. I haven't looked under the hood, but does the ActionBasedController etc load the asset every time?
     
    ericprovencher likes this.
  48. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,073
    Handling input shouldn't take more than 0.01ms. It's scraping a cache somewhere unique on each device type and platform, that's got the input flags in it of some sort, and objects and/or structs, and then surfacing these to us and/or calling functions.

    Am I wrong about this?

    Why is it anywhere near 1ms?

    We have a total of 8ms, on mostly constrained hardware, on the main thread, that we can't realistically go anywhere near using all 8ms when wanting stable frame rates, and my tests show that even a primitive use of the New Input System is eating more than 10% of this.

    Why?

    How do we get it down to tiny fractions of a millisecond to handle input?
     
    OGMagi, ElasticSea, ontrigger and 2 others like this.
  49. efficientTech

    efficientTech

    Joined:
    Jul 23, 2021
    Posts:
    6
    I'm having the same issue and I'm not sure what the solution is from reading this post. We are getting input times of 3-4ms and it's absolutely tanking our performance.
     
  50. ericprovencher

    ericprovencher

    Unity Technologies

    Joined:
    Dec 17, 2020
    Posts:
    219
    Please do share a repro project with this issue. Having case studies helps us improve build the case for the performance concerns.