So I was using a cylinder shape with a non uniform scale but it was unnecessarily complicating some transformations. So I just duplicated the cylinder mesh using existing tools to the right scale. But after some wtf moments I figured out doing so messed up the collider orientation. LocalToWorld Forward is like 30 degrees off from the collider forward shown in the physics debug display. Must be some logical explanation but I'm missing it.