Pull a hinge joint

Hi guys,
im currently developping with the HTC vive.
I want to make a door that can be pushed to open it as well as pulled to close it.
The pushing part was easy but I have no idea how to pull it due to the fact that I can't parent the door to my htc vive controller object.

Do you have any recommendations?

Regards,
Yannik