VR Controller Input Stops Working After Using Mouse in Scene

Has anybody experienced an issue where the VR controller inputs for the VR Editor stop working after you manipulate an object in the scene with the mouse? So far, the only fix I've found is restarting the editor, which is in exactly a fix

Any ideas? Thanks