Question Issue with VR plane physics

I am using Oculus SDK.
I have a plane that has a steering wheel
I attached a rigidbody and hinge joint to the wheel.
The plane has a rigidbody. I use a force to move the plane.

If I don't attach the wheel to the plane the steering wheel goes flying off.
When I attach the wheel to the plane, if I turn the wheel with my hands it turns the plane.
I am not sure how to set this up so my hands don't turn the plane.

I have tried everything I can think of. Using kinematic. Using dummy objects with rigidbodies. Using fixed joints. Nothing works. Either I can't turn the steering wheel at all or the plane turns too.
Thanks,