Trembling colliders!

Hi!

I would like to ask your help in a very strange problem... I just working on a system for maintenance-training with LEAP Orion and Oculus when the problems encountered, you can see in the video: https://1drv.ms/v/s!Ajgvr-zdn1HagsMN2ywCp-7nPwUy9A
I want to pull the nut on the axis, the movement is defined by colliders for the most realistic feeling. The first version of the project don't have this strange phenomenon, then I put the active parts under GameObjects as shown in the picture below.
One layer is the "axis", the other the "nut" colliders and they contact each other.
Someone knows, what is the solution?

If I don't understandable, please ask!
Thank you so much!

 

In this older version the collision works grater! https://1drv.ms/v/s!Ajgvr-zdn1HagsMT0VhCJj5V999GEA
What could be the problem, what can be the difference?

Thank you very much!

 

Hi!

When I create a simple Cube in the scene and add some Colliders directy to the "cube" GameObject (like the video) the problem does not occur! Thats strange, but unfortunatly it's not slove my porblem becouse I can't use the same layer for the part collison and for grabbing layer :/

https://1drv.ms/v/s!Ajgvr-zdn1HagsMUXRPj2o1TMOXJnQ