SteamVR + HTC Vive = floating rectangle

So, I'm working on a scene using SteamVR, and in every scene I go into, there is a floating rectangle under the ground and vertical. Sometimes it's above the ground. But it is always RIGHT in front of me. It seems to have the same texture as the main object under me (ground etc) and moves when I teleport around.
In one level it's sticking out of the ground in front of me.

What's weird though, is it doesn't show up in a screenshot. SteamVR screenshot shows the ground as normal. But standing in the Vive I can see it.

https://imgur.com/a/cbiOuku

This is a photo taken directly in the headset.
The attached image on the thread is the same spot taken with Steam VR.

I can't see it in the 2d debug view. And I can't see anything weird attached to the SteamVR Player object.

It stays where it is relative to the playspace though. So I can walk around it to see behind it. But a teleport brings it with me.

Has anyone got a clue what this is?