Question Advices on how to implement physics-based grab with Oculus hand tracking

Hi all!
I have recently started using unity, for now I have used it for a few university projects, and I am new to the world of 3d graphics in general. I'd like to implement a physical grab system (not the default oculus one), i.e. to grab an object you have to bend your fingers and grab using hand tracking from Quest 2. Specifically, I would like to map hand tracking to an avatar's hands and use these to grab objects. The ultimate goal is to record a demonstration and to train an agent with MLAgents on the same avatar (that's why I want to map the hand tracking input to the avatar and not directly use the bones of the hand tracked by the quest).
Can you recommend any guides or tutorials for inspiration? Especially on the mapping between hand tracking and avatar.
Thanks!


EDIT:
I'm doing it this way: taking the Transforms from OVRSkeleton and copying the bone rotations into the avatar Transforms. It seems to work except for some strange behavior in the thumb and pinky (they are the two fingers with 4 bones instead of 3 like the others, unlike the avatar from mixamo which has 4 parts for each finger, so I definitely need to figure out the correspondence).

 

I will just like to follow up on this question. Have you been able to do something about this? Also, what materials did you use for this?

 

Hello! If you need a mapping from hand tracking to avatar, maybe you should try this toolkit: https://github.com/jorgejgnz/HPTK. It's super easy to setup and it works like a charm.
You can follow the docs for the physics settings and the examples for the materials.

If you need help, the creator has a Discord and he's super helpful and kind.

Good luck!