Oculus VR overriding the unity's input System

I have a project with 2 scenes , one with VR Supported and another with non-VR . In vr user wears the headset and play the game. in NON -VR he has to remove the headset then interact with unity ui using mouse and keyboard on the moniter.
.
The game starts with non vr scene , where user will interact with ui on moniter, adjust some settings for the game. Then he enters the scene with VR.
.
WHEN I COME BACK TO NON-VR scene from VR-Scene , i am not able to interact with the unity ui unless my vr headset is enable.(by enabled i mean it should be wore).
.

I face this problem in build file. it is working fine in unity editor.
.
please share your views.