Rotate to velocity but ignore an axis.

I'm currently trying to implement sliding similar to 3d platforming games like Super Mario 64. My character is face down on their belly (their local forward vector is facing the ground). I have a simple sloped surface to act as my slide. I want the character to slowly rotate towards their velocity. I have this bit of code:

Code (CSharp):

1.     //Rotate towards velocity direction when sliding
2.     public void RotateSlideToVelocity(float turnSpeed)
3.     {
4.         Vector3 dir = rigidbody.velocity;
5.         dir.y = -90;
6.  
7.         if (dir.magnitude > 0.1f)
8.         {
9.             Quaternion dirQ = Quaternion.LookRotation(dir);
10.             Quaternion slerp = Quaternion.Slerp(transform.rotation, dirQ, dir.magnitude * turnSpeed * Time.deltaTime);
11.             rigidbody.MoveRotation(slerp);
12.         }
13.     }

This works, but they wobble on their x axis. I'm sure this is due to the dir.y = -90 line. That line at least guarantees that the player is facing down while sliding. If I remove that line however, the player will be somewhat upright - I'm guessing this is because it takes into consideration the y velocity. Setting dir.y to 0 directly makes the character stand perfectly upright. I want physics to handle the rotation of the player's local x axis, not the move rotation line.

Any ideas on how to do this?

 

A quick and dirty way might be to get the x euler angle before doing the MoveRotation and then just reassign it afterwards.