Rigidbody moves by Y when against the collider

Here is the video for quick understanding:

https://webmshare.com/AX1Mj

As you can see, when I put player object against the static box collider with acceleration on X it starts to slowly descend on Y asix.

Controller code is nothing special:
https://hastebin.com/xutayetoxu.go

Any idea why this might happen?

 

To me, that movement on Y axis falls in the rounding errors and/or numerical limits categories. I don't think it can be removed by just relying of physics. If you need the player to stay perfectly steady on the Y coordinate while moving in the X axis you should constrain the position explicitly (i.e. by setting transform.position with a rounded Y coordinate).