How do I setup hinge joint physics?

In my game the player is able to hold up a lantern for some light, however it is currently very static and I don't like this. I wanted to add some physics to the lantern by using a hinge joint component, but I can't get it working properly. Basically, the players arm and lantern are made up of 3 parts: the arm, the handle (which is a child of the arm), and the lantern (which is a child of the handle). When I add the hinge joints to the handle and lantern the physics become super strange, they rotate slowly into strange positions and don't seem to react to the players movement, almost like they have no weight. Help would be appreciated