Conservation of Momentum lost in simple collisions

I set up 2 spheres aimed at each other inside two kinematic walls with no gravity. Each sphere has a velocity of 5 in the x direction towards the other sphere. After the collision they bounce off the walls and then keep repeating the collisions. During this experiment momentum is conserved. In the info of the rigidbody each sphere speed stays constant at 5.

However if I raise one sphere slightly higher than the other and they collide off center then they lose speed in each collision with each other but not with the walls. They actually seem to gain angular momentum during the first collision but it seems to be lost when hitting the walls.

So I think that the momentum is lost into the walls as angular momentum loss.

I've tried adjusting all the rigidbody, physics material, and physics project settings. Nothing seems to affect the momentum loss.

Is there a way to prevent the loss of angular momentum?

 

The problem was on my end. I had some attached script that was changing the friction after I set it. So I thought it was 0 but it changed to .2 immediately afterwards. But resetting it in the inspector after play had started did not alter the friction even though the inspector said it was altered. So that threw me off. So now they seem to be bouncing and colliding without losing angular velocity or regular velocity.