Hinge Joint as an axle

I am attempting to use hinge joints to connect tires to a body, it works decently, but the wheels end up getting pulled off of the axis when turning. The wheels axis isn't supposed to move like in a car, instead the wheels just turn in different directions. (Think like tank controls)

Is there anything I can do to fix this?

 

I attached a gif which will probably illustrate the problem better than my words. Ignore the artifacts.

Thanks for any help anyone can offer

 

Anyone got any advice?

This is really annoying me

 

If your Wheels have a rigidbody, you could freeze every rotation, except for the one that you need.

 

But then the vehicle won't be able to tip over, etc.

I definitely need that for what I am working on.

 

Any help?

 

I would try making a Physic material and play with the friction values. You could also increase the physics iterations to get some more accurate calculations.