Question [Bug] Ray interactor flashes behind an interactable when moving it fast

When using XR Interaction Toolkit (even in Unity's example project) when you move an object the ray interactor flashes behind the object and the controller vibrates as if it's hovering over the interactable for the first time, here's an example:

See the Ray Interactor is usually blocked by the object your holding, but when you move it it flashes behind the object. This happens for Instantaneous, Kinematic, & Velocity tracked. It's problematic not just visually but the constant controller vibration is very distracting.

What's the best way to fix this, I've considered writing code that detects the distance the interactable is away from the interactor and sets the ray interactor to this length, I'd also need to write code to fix the vibration, but this feels hacky, is there a better/easier way to fix this?