Struggling to procedurally set correct orientations to generated mesh bones

Hey there. So, I have a procedural snake mesh that is generated and animated at runtime, extruding itself around an array of points whose positions are controlled with the SmoothDamp() function.

The positions work just fine, but my problem comes when I upgraded the mesh to be only refreshed when the snake grows, and letting an array of bones do the transforms on the mesh for the animation instead.

Here is a video of the issue. The first bone seems to always be twisted and fixed in one direction, no matter what rotation I assign to it at first or at the update.
https://cdn.discordapp.com/attachme....6f1_Personal___DX11__2022-07-22_20-04-32.mp4

Also, when it turns, it’s possible to see that at times the orientation of other bones seem to be fixed, depending on which position they were assigned during a mesh update; but not always.
The reason the root bone bones[0] isn’t parented to anything is that when it was the case, the movement update from the parenting did conflict with the one of the damping function.

Overall, I am under the impression I am missing something quite simple and not understanding exactly how rotations are intricated in each other.

What I want to achieve is to have the forward orientation of bone[0] always aligned to the forward orientation of the head, and having the following forward orientations of the bones directed at the position of the previous bone, which in theory seems to be what this code is trying to achieve.

I think this has to do with the bindPoses and local / global space, but I’m not really understanding how. There is an old post here that tries to explain them pretty well, but I’m still not getting exactly how this can apply on my case. When I do respect the procedure from the documentation and use local positions and rotations, the just goes all over the place. And there is virtually no documentation about rigging that doesn’t use the Animation Rigging pack, which isn’t what I want to do.

My intuition is that I should reform the function as to work in local space rather than in global space, as it’s not very logical to have a global mesh that follows something unrelated, but I’m unsure of how to proceed at this point.

Here is the code (I edited non-relevant bits out, but I can send the whole thing if needed).

Code (CSharp):

1. void Start()
2.     { ...
3.         bones = new Transform[maxSegmentsCount];
4.         for (int i = 0; i < bones.Length; i++)
5.         {
6.             bones[i] = new GameObject($"Spine_{i}").transform;
7.         }
8.         bindPoses = new Matrix4x4[maxSegmentsCount];
9.         rend = GetComponent<SkinnedMeshRenderer>();
10.  
11.         mesh = new Mesh();
12.         mesh.name = "Snake Body";
13.         rend.sharedMesh = mesh;
14.         rend.bones = bones;
15.  
16.         GenerateBodyMesh();
17.  
18.     }

Code (CSharp):

1. void Update()
2.     {
3.         segmentPoints[0].position = head.position;
4.         segmentPoints[0].rotation = head.rotation;
5.  
6.         rend.bones[0].position = head.position;
7.         rend.bones[0].rotation = head.localRotation;
8.  
9.         for (int i = 1; i < currentSegmentsCount; i++)
10.         {
11.             Vector3 target = segmentPoints[i-1].position;
12.             Vector3 current = segmentPoints[i].position;
13.             Vector3 bufferDist = -head.forward * segmentsInterval;
14.             Vector3 dir = target - current;
15.  
16.             segmentPoints[i].position = Vector3.SmoothDamp(
17.                 current,
18.                 target + bufferDist,
19.                 ref segmentPoints[i].velocity,
20.                 movementDamping + i / trailResponse);
21.            
22.             rend.bones[i].position = segmentPoints[i].position;
23.            
24.             segmentPoints[i].rotation = Quaternion.LookRotation(dir);
25.             rend.bones[i].rotation = segmentPoints[i].rotation;
26.         }
27.     }

Code (CSharp):

1.  private void GenerateBodyMesh()
2.     {
3.        ...
4.  
5.         // populate vertices
6.         for (int slice = 0; slice < currentSegmentsCount; slice++)
7.         {
8.             OrientedPoint localOrigin = segmentPoints[slice];
9.  
10.             ...
11.  
12.             // assigning bones positions to the local origin point of the mesh
13.             bones[slice].position = head.InverseTransformPoint(localOrigin.position);
14.             bones[slice].rotation = Quaternion.identity;
15.             bones[slice].localRotation = Quaternion.identity;
16.  
17.             for (int i = 0; i < shape.VertCount; i++)
18.             {
19.  
20.                 // ... assigning other vertex data ...
21.                
22.                 // assign weight data
23.                 BoneWeight currentWeight = new BoneWeight();
24.                 currentWeight.boneIndex0 = slice;
25.                 currentWeight.weight0 = 1;
26.                 weights.Add(currentWeight);
27.  
28.             }
29.         }
30.  
31.         // ... drawing triangles ...
32.  
33.         // ... assigning data to mesh ...
34.        
35.         arr_weights = new BoneWeight[currentSegmentsCount * vc];
36.         for (int i = 0; i < weights.Count; i++)
37.         {
38.             arr_weights[i] = weights[i];
39.         }
40.  
41.         //bones[0].parent = head; // caused the wonkiness
42.         bindPoses[0] = bones[0].worldToLocalMatrix;
43.  
44.  
45.         for (int i = 1; i < bones.Length; i++)
46.         {
47.             bones[i].parent = bones[i-1];
48.             bindPoses[i] = bones[i].worldToLocalMatrix;
49.  
50.         }
51.  
52.         mesh.boneWeights = arr_weights;
53.         mesh.bindposes = bindPoses;
54.         rend.bones = bones;
55.         rend.sharedMesh = mesh;
56.     }

If anyone has ideas, that would be lovely!

 

Mmh, bah, anyway, I ended up finding OpenGL resources that lay down transformation matrices much better that Unity's documentation.

So now I realise that in fact the matrices have to multiply recursively through the bone's hierarchy then multiply by the invert of the world space. In this case though I opted for leaving the hierarchy aside, since the model is now physics based, so the matrix operation was easy with only two terms, just as in the scripting reference.

The real fix was to set the bindposes right after the generation of a ring of vertices. Now it works impeccably.

The Mesh.bindPoses API documentation could be a bit more verbose about what it is and point towards the right reference though. All good information I found was from people writing engines, not using them.

 

Hi @exitshadow I've been looking at your code, and the documentation about bindposes: https://docs.unity3d.com/ScriptReference/Mesh-bindposes.html
and I just can't wrap my head around the missing parts of your code.
no expectation you just hand them to me, I also would like to think about it. but maybe some general questions

1. snake "head" is that a mesh you prepared ahead and imported to unity?
2. except for segments and the extrusion, did you import an initial snake mesh to unity?
3. in GenerateBodyMesh() - I see a head reference, head seems to be a transform, do you instantiate the head and then apply any following data relative to the head?

4. OrientPoint - is that a struct you've created which holds variables such as: position, velocity?
5. Segments are actually OrientPoints, does a segment holds any mesh related data?
6.

is this the tedious work of writing each vertices, uv, and triangle of the mesh, resulting in long arrays, and finally applied to a mesh?
7.

Wonder what happens at the beginning of this method
8.

three dots again.. maybe a comment about what should happen here...

curious what "other" data should I consider here

Thanks in advanse