Scene Pan BUG: Scene view moves randomly after releasing middle mouse button (pan)

I am encountering a frustrating bug in the Unity Editor Scene view.
Whenever I pan with my middle mouse button, the scene view oftentimes, but not always, automatically pans away from wherever I just panned towards when I release the middle mouse button.
These movements don't seem to be related to the distance of the middle mouse pan. Sometimes I get "pushed" only a few centimeters in one direction , other times a couple meters.

This makes focusing and panning around certain objects an absolute nightmare.

Does anyone have an idea what could have caused this issue? Other co-workers who are working on the same scene on different computers don't experience this problem.

 

Maybe try a different mouse?
Check your computer for errant input devices?

 

I did try one of my colleagues mice and it does seem to solve the problem.
I wouldn't know why a mechanical issue like that causes the pan view to snap back to where my mouse was though. It seems as if it always moves the part of the screen my mouse is hovering when releasing the middle mouse button back to the center of the scene view. How can a mouse cause that.?