Question Compute mesh deformation, without dots animation

Hi,

With a game object, I can move a character bone (for example: stretch the head in "Y") and the mesh will deform.

but with entities, if I manually modify the translation of a bone in runtime there will be no visual change to the mesh, Why?

I have Dots Animation installed and I am able to make my character run. (But my goal is to modify a bone without using dots animation plugin)

How to make the mesh deform went a bone is being moved?

 

I wrote a custom solution for this. I don't recommend using this version for anything other than a reference implementation since it is a prototype. The final version will have a better API, docs, and more support for different prefab setups, as well as a few surprise features. [0.4.2] My personal DOTS framework - Latios Framework

 

You look like you know what you are doing with animation, can you help me with my problem?

 

I found a piece of the puzzle.

SkinnedMeshRenderer are converted to DynamicBuffer<SkinMatrix>

SkinMatrix is a float3x4

went I change a value in that SkinMatrix it does not propagate to the children, for example: If I rotate the knee, the foot gonna stay on the ground.

so I have new questions:
How do I change the float3x4 properly?
How to rotate the bones correctly with all the children?

 

I can't help you with the animation package since I don't use it currently, but I can help you if you want to provide raw skinned matrices to the Hybrid Renderer. But you'll have to be much more specific about what you are struggling with and also be patient since I am going to be spending time with family here shortly.

But to at least get you started, in the entities package there is an assembly called Unity.Deformations. And inside that there is an IBufferElementData called SkinMatrix. On a SkinnedMeshRenderer, you can find a bones array and a bindposes array which have one-to-one index correspondence. Now suppose you have a Blender-style model in which the mesh and the root bone are both separately parented to the armature object. The formula for a SkinMatrix is simply

Code (CSharp):

1. var skinMat     = math.mul(boneToArmature, bindPose);
2. skinMatrices[i] = new SkinMatrix { Value = new float3x4(skinMat.c0.xyz, skinMat.c1.xyz, skinMat.c2.xyz, skinMat.c3.xyz) };

Or to describe what is actually happening, the bindpose matrix converts a mesh vertex into the bone's local coordinate space pre-deform. And then the boneToArmature matrix converts it back to mesh space but using the deformed bone position.

The module I am writing handles all of this and lets you just write to the bone transform components and it handles everything else, including bounds culling.

 

Make sure to perform your calculations after the TransformSystemGroup so that you have up-to-date LocalToWorld components from which to calculate your boneToArmature matrices.

 

Ok, I will do that.

I will try to explain better.

Code (CSharp):

1.        
2. JobHandle job = Entities.ForEach((Entity entity, ref DynamicBuffer<SkinMatrix> skinsMatrix, in TestComponent test) => {
3.  
4.             SkinMatrix skin = skinsMatrix[test.boneIndex];
5.  
6.             skin.Value.c2.x += 1 * deltatime;
7.  
8.             skinsMatrix[test.boneIndex] = skin;
9.  
10.         }).Schedule(Dependency);
11.  

test.boneIndex = 25 (Thats my character right shoulder)

With the script above I am able to rotate the shoulder but the elbow, hand, fingers do not move with the shoulder, resulting in a weird deformation.

I do not understand the SkinMatrix very well, what does C0, C1, C2, C3 refer to... and is that local dimension, because it doesn't match the LocalToWorld (See screenshot below)
SkinMatric.Value.C0 = Scale i think?
SkinMatric.Value.C1 = Translation i think?
SkinMatric.Value.C2 = Rotation?
SkinMatric.Value.C3 = ???

Shoulder Skin Matrix

Shoulder Entity


So to conclude I don't understand:
SkinMatrix C0,C1,C2,C3 and I don't know how to move the hand, elbow & shoulder all together.

 

Correct, because you are using unmodified skin matrices for the other bones. When working with the skin matrix buffer, you have to handle the hierarchical information yourself.

SkinMatrix is actually several matrices multiplied together, similar to LocalToWorld. The c0, c1, c2, and c3 just refer to the columns of the matrix.

To do what you want to do, you either need to spend some time learning how transform matrices work, or wait for me to release my module and use that.

 

Based on my research I agree with this. If you want to know how Unity does it you can take a look at the Unity.Animation.ComputeRigMatrices to see how they update the Unity.Animation.AnimatedLocalToRoot and Unity.Animation.AnimatedLocalToWorld. For my work i decided to not remake the Animation system and use whatever it generates for my work. So like @DreamingImLatios said

 

Dots Unity.Animation was overriding my rotation & translation.

So I decided to remove DOTS Unity.Animation and make myself an Animation system.

So I add to make myself a "SkinMatrixSystem" to modify my skin matrix, a "SkeletonMatrixSystem" to compute the LTW and LTP of each bone, and finally, an "AnimationSystem" that provide an offset matrix for the current animation.

Now I am having a problem converting unity AnimationClip, but that is for another post.

Thanks, DreamingImLatios & NT_Ninetails. for pointing me in the right direction.