Get correct vertices from scaled skinned mesh renderer

Hi guys,

I am using the k4a azure kinect examples sdk for unity to create a virtual fitting room type experience.

What i am trying to accomplish is: when a user is detected, spawn virtual objects on his/her body and them stick to the user mesh (so they continue to stick on the user's body). I have achieved this mostly by taking the positions of the vertices from the baked mesh result of SkinnedMeshRender. To ensure that the fitting is exact for a person of any height and weight, I am using a script in the k4a sdk called AvatarScaler.cs. This script scales the skinnedMeshRenderer mesh to the exact shape of the user's body. I took a peek into the script and saw that the avatar is scaled by scaling each bone (I think, not 100% sure).

My problem is: my objects are following the user body just fine. The problem is that when the avatar scales, it seems like the mesh that is baked by skinnedMeshRenderer does not take into account such scaling, so the vertex positions im using to place virtual objects are off by a constant scale factor.

My question is: how do I get vertex positions of a skinnedMeshRenderer after bones are scaled?

 

Just to be clear, you are using SkinnedMeshRender method BakeMesh to get the actual mesh?
https://docs.unity3d.com/ScriptReference/SkinnedMeshRenderer.BakeMesh.html

Because in my case, and I tested it now, this method reflects scaling of bones' transforms.
Though I don't think it will help your case, this method has an overload (see the documentation link) with parameter useScale. However, the default behavior for skinned mesh is to ignore transform of skinned mesh render component (if it is not a parent).
It would help to post some screenshot to better understand your problem.

 

Just fixed it. I think you are correct, the vertices are already corrected with bone scale. What caused the offset for me was the parent's tranform, I had to apply the parent's transform manually to the vertices from the skinnedmeshrenderer