Empty Project - Bad Performance - AMD RX 580 + i4690k + 8GB DDR 3 - Oculus Rift S

Hello,

I have a problem with the performance of my project.
It is a empty scene with only a the OVRPlayerController from the Oculus SDK.
Ingame I just have like 50 FPS and sometimes it drops even lower, very choppy.

The GPU utliziation reaches a maximum of 3 % and the CPU is working with 35 % on the unity thread.
So there is no hardware bottleneck at all. The Rift S just have a resolution of 2560x1440 nothing a 580 could not handle in a empty scene. E

I've tried out everything. Disabled Vsync, trying to switch to SinglePass Rendering etc.
I even lowered the Graphic settings but nothing has an effect on the FPS.

Any ideas whats the problem may be?

 

Mh no one have an idea?