Question Two parallel box colliders Sliding - Unexpected behavior

So there are two physics bodies, one is static while another is dynamic.
On the dynamic body I added the "Limit DOF joint" from the sample, the dynamic body have all 3 rotation axes locked and Y position axis locked, and everything is working.

The problem is when I translate towards static collider with the dynamic one, the dynamic one goes +/-90 degrees from the direction it was going (but with smaller speed).
Both colliders axis aligned, and the player movements are always axis aligned, and both colliders have "Bevel radius" set to 0 , so I don't understand why it is happening , and how to prevent that.

Here is how it looks.

Also I tested it on regular rigidbody and this is not happening.