Question Creating an interactable AR/VR avatar using animation rigging.

I would like to be able to animate a rigged humanoid avatar in AR/ VR by simply being able to grab different touch bones, position the model and record the keyframes of my avatar in the different positions while in play mode.
I also what to set constraints so the avatar can only be positioned in a way a real human could be positioned (e.g. constraining the head to only rotate by x deg on each axis etc).
Essentially I want to be able to interact with a humanoid model in AR/VR similarly to how you can interact with it in Blender while creating animations.
I'm thinking about using this package and just wanted to ask if anyone has some ideas/ recommendations on how I should go about this? Has someone done this before? I think I'll start with this amazing project:

but there aren't any muscle constraints so the model can be brought into all kinds of different unnatural positions. I really want if to feel like/ look like I'm animating/ interacting with a real human in AR/VR.
Thank you!