Bug VR game runs fine on oculus, but desktop app shows up as black screen

I have a problem that my VR application runs fine on the oculus device itself, but when I look at the desktop app at the same time the desktop app is completely black. The default VR scene seems to work fine if I build it and use it in the same way. How can I fix this?

Edit: I can see that the application shows up fine until I put on my headset and it switches the view to the headset.