Top VR SDK reverts to None after level change

If I manually drag the OpenVR SDK to the top of the SKD list, it all works fine until I change the scene. In the newly opened scene, "None" is back on top. I tried enabling OpenVR from code in each scene, but I'm getting the "Assertion failed: Assertion failed on expression: 'IsMatrixValid(matrix)'" error. Below is the coroutine I'm using. How can I fix it?

IEnumerator SwitchToVR()
{
string desiredDevice = "OpenVR";
if (UnityEngine.XR.XRSettings.loadedDeviceName == desiredDevice)
{
yield break;
}
XRSettings.LoadDeviceByName(desiredDevice);
yield return null;
XRSettings.enabled = true;
}

 

Anybody know why this happens? Super annoying.