Capsule Collider and Sliding off the edge

I'm making a first person controller but the capsule collider is driving me nuts. This is not just Unity problem, I see it in many games that use capsule as character collider.

The problem is if the character oversteps capsule's center by even a tiny bit he will start to slide off the edge. Same happens when you jump from a platform and you don't land perfectly on center of capsule but instead and edge and slide off the platform.

Box collider is no go because it is rectangular and thus if you stand near a wall and rotate you will involuntarily move or some other bad things will happen.

I thought about using cylinder as collider but since it is not an integrated shape you cannot change radius and height like you can for capsule.

Suggestions are welcome