Search Unity

Moving bones at runtime for Finger level IK in VR?

Discussion in 'Animation' started by alexchesser, Jan 17, 2020.

  1. alexchesser

    alexchesser

    Joined:
    Sep 15, 2017
    Posts:
    147
    Hi All,

    I've made some progress with arm presence in VR with OculusQuest hand tracking (see video). My next step is to try to make my model's fingers move in time with the OVRSkeleton's finger movements.

    At a high level, the OVRSkeleton has a list of bone-objects representing a transform for each finger/joint in the hand. I've created a mapping of the MIXAMO bone to OCULUS bone object such that they are roughly equivalent.

    The algorithm I'm trying is very brute force but essentially I have a function call in the OnAnimatorIK which moves each MIXAMO finger bone's transform to the matching position on the OVRSkeleton.

    Now the problem:

    It seems like simply moving the transform of the MIXAMO bone is insufficient to force the movement of the underlying model ... *SO* with all that said, my question here is how do I move a bone at runtime in order to ensure the model moves with it?

     
    alexchesser, Jan 17, 2020
    #1
    InsaneDuane likes this.

  2. alexchesser

    alexchesser

    Joined:
    Sep 15, 2017
    Posts:
    147
    if anyone is interested - I made some progress

    Thumbs are a little off, but other than that it's in (IMO) pretty good shape.

    Since this is built to support a MIXAMO character rig, you could pretty easily swap out any character model you want in this space. My next step is to do a bit of finesse on the thumb-tuning - maybe a tool to adjust the offset digits while in the app, since the work to fine tune rigging adjustments is pretty significant.




    Oh! and by the way, the answer was to move the bones within LateUpdate.
     
    alexchesser, Jan 20, 2020
    #2
    FlightOfOne, InsaneDuane and User340 like this.

  3. User340

    User340

    Joined:
    Feb 28, 2007
    Posts:
    3,001
    That is so cool. It's gonna be so nice not having to pick up controllers anymore.
     
    User340, Jan 20, 2020
    #3

  4. PFH

    PFH

    Joined:
    Nov 5, 2018
    Posts:
    2
    Hey alexchesser, looks amaziing what you done! This is exactly what i need for our Projects. Are you willing to share your efforts?
     
    PFH, Jun 8, 2020
    #4

  5. alexchesser

    alexchesser

    Joined:
    Sep 15, 2017
    Posts:
    147
    Hi PFH, I just realized I never replied to you. If you look through my post history here I've done a write up on how I did this and linked to the github repo where all the code exists.

    Warning though, it hasn't been touched since that sample and both UNITY and VR has moved forward, so I would be surprised if it still worked out of the box. I think you'd have to start from scratch.

    edit - easier for me to post it to you than to make you search out the answer @PFH https://forum.unity.com/threads/oculus-handtracking-with-custom-hand-model.810678/
     
    alexchesser, Feb 16, 2021
    #5