5.4 collider gizmos drifting in VR

Encountered a strange problem today - since the 5.4 update (which is awesome in many ways) it became pretty hard to debug VR scenes that rely on raycasting and colliders (vector forward from camera position) because in the editor, all colliders drift around with the HMD movement (rift cv1) where they should be completely static.
The raycasting itself still works as expected, but the gizmos are way off, and moreover invisible in the hmd when activated in the editor - which used to be useful in older versions.

What's going on here, am I missing something?

Reproduceable every time: new empty scene, a cube with box collider somewhere, enable gizmos, select cube, hit play, move HMD around -> green gizmo flying all over the place...