Help combining and manipulating skinned mesh renderers imported from Blender

Two things I want to do:

1. Take a bunch of skinned mesh renderers and combine them
2. Create a flipped version of a skinned mesh object - for example select a left arm and flip it and have it be a right arm with all the animations flipped (you can do this by setting the scale to negative on a parent but previously this would mess up colliders - it seems to work now, so maybe I don't need this)

Important: these meshes are imported from blender and have the structure where the armature is parallel to the skinned mesh renderer and the bones are underneath the armature.

The issue I'm running into on both of these things is setting up the bind poses correctly. I don't quite understand how the bind poses are derived for skinned meshes imported through Blender. They definitely aren't derived like the Unity examples where it's something like:

bones[0].worldToLocalMatrix * transform.localToWorldMatrix;

What I want is a version of the SkinnedMeshCombiner you can find in various places (http://wiki.unity3d.com/index.php?title=SkinnedMeshCombiner) that works for SkinnedMeshes imported from .blend files, and some understanding of how the bind poses work for blender imports so that I can write the flipper correctly. (Assuming that setting scale to negative still breaks colliders - looking into that now)

 

Ok so I kind of have this figured out I think...

Going to state the problem and then the solution.

Problem: The way the bind pose on a skinned mesh works in theory is that it maps the mesh verts to world space and then back to bone-local space. But in a Blender model the bones are not children of the Skinned Mesh Renderer, they are parallel to it and children of the Armature. If you examine the generated bind poses for a Blender-imported model they are NOT the typical:

bones[0].worldToLocalMatrix * transform.localToWorldMatrix;

Instead they somehow take the Armature transform into account as well.

I spent some time trying to figure out how these bind poses are constructed and ultimately I have no idea. Zeroing out the rotation and translation on the armature before doing the above matrix operations works in some cases but not all. Zeroing out the translation and rotating by x=90 also seems to sometimes work but falls apart in more complex cases. I can't reproduce the math that creates the bind poses.

Solution:

I want to construct a combined skinned mesh and fix up these bind poses. My solution is take the old bind pose and just multiply it by the original skinned mesh worldToLocalMatrix.

New bind pose is this:

oldBindPose * transform.worldToLocalMatrix

Where "transform" is the old transform that was on the original skinned mesh object.

In theory the oldBindPose was:

bones.worldToLocalMatrix * transform.localToWorldMatrix

However we know that that's not quite right, since doing that math doesn't result in the imported bind pose. So instead let's say the oldBindPose was this:

mysteryMatrix * transform.localToWorldMatrix

When we multiply that by transform.worldToLocalMatrix we're just left with mysteryMatrix. We don't know what that is or how it is derived, but we do know that it maps verts into local bone space correctly. (Or does whatever it needs to do to make the mesh skinning algorithm work correctly)

So effectively we've corrected for the fact that the old skinned mesh was somewhere in the hierarchy and now we've relocated to the root by just removing the localToWorldMatrix from the original bind pose. Whatever is left, the mysteryMatrix, still correctly does the job of mapping verts to bone space, though we have no idea how it does that.

The new code is very similar to the old skinned mesh combiner on the wiki, the difference is that instead of trying to reconstruct a new bind pose based on the assumption that "bones[0].worldToLocalMatrix * transform.localToWorldMatrix" is the right way to do that we just take the old bind pose and modify it in the way described above.

Code (CSharp):

1. using UnityEngine;
2. using System.Collections.Generic;
3.  
4. public class SkinnedMeshCombiner {
5.  
6.     private static List<Transform> bones = new List<Transform> ();
7.     private static List<BoneWeight> boneWeights = new List<BoneWeight> ();
8.     private static List<CombineInstance> combineInstances = new List<CombineInstance> ();
9.     private static List<Texture2D> textures = new List<Texture2D>();
10.     private static List<SkinnedMeshRenderer> smRenderers = new List<SkinnedMeshRenderer> ();
11.     private static List<Matrix4x4> bindPoses = new List<Matrix4x4>();
12.  
13.     public static void CombineMeshes(GameObject o, Material m) {  
14.         bones.Clear ();
15.         boneWeights.Clear ();
16.         combineInstances.Clear ();
17.         textures.Clear ();
18.         smRenderers.Clear ();
19.  
20.         o.GetComponentsInChildren<SkinnedMeshRenderer>(true,smRenderers);
21.  
22.  
23.         int boneOffset = 0;
24.         for( int s = 0; s < smRenderers.Count; s++ ) {
25.             SkinnedMeshRenderer smr = smRenderers[s];      
26.  
27.             BoneWeight[] meshBoneweight = smr.sharedMesh.boneWeights;
28.  
29.             // May want to modify this if the renderer shares bones as unnecessary bones will get added.
30.  
31.             for (int i = 0; i<meshBoneweight.Length; ++i){
32.                 BoneWeight bWeight = meshBoneweight[i];
33.  
34.                 bWeight.boneIndex0 += boneOffset;
35.                 bWeight.boneIndex1 += boneOffset;
36.                 bWeight.boneIndex2 += boneOffset;
37.                 bWeight.boneIndex3 += boneOffset;            
38.  
39.                 boneWeights.Add( bWeight );
40.             }
41.  
42.             boneOffset += smr.bones.Length;
43.  
44.             Transform[] meshBones = smr.bones;
45.  
46.  
47.  
48.  
49.             for (int i = 0; i < meshBones.Length; ++i) {
50.                 bones.Add (meshBones[i]);
51.  
52.                 //we take the old bind pose that mapped from our mesh to world to bone,
53.                 //and take out our localToWorldMatrix, so now it's JUST the bone matrix
54.                 //since our skinned mesh renderer is going to be on the root of our object that works
55.                 bindPoses.Add (smr.sharedMesh.bindposes [i] * smr.transform.worldToLocalMatrix);
56.             }
57.  
58.             if (smr.material.mainTexture != null) {
59.                 textures.Add (smr.material.mainTexture as Texture2D);
60.             }
61.  
62.             CombineInstance ci = new CombineInstance();
63.             ci.mesh = smr.sharedMesh;
64.             ci.transform = smr.transform.localToWorldMatrix;
65.             combineInstances.Add( ci );
66.  
67.  
68.  
69.             GameObject.DestroyImmediate( smr );
70.         }
71.          
72.  
73.  
74.         SkinnedMeshRenderer r = o.AddComponent<SkinnedMeshRenderer>();
75.         r.sharedMesh = new Mesh();
76.         r.sharedMesh.CombineMeshes( combineInstances.ToArray(), true, true );
77.  
78.         Texture2D mainTexture = textures.Count > 0 ? textures [0] : null;
79.         Vector2[] originalUVs = r.sharedMesh.uv;
80.  
81.  
82.  
83.         Material combinedMat = m;
84.         combinedMat.mainTexture = mainTexture;
85.         r.sharedMaterial = combinedMat;
86.  
87.         r.bones = bones.ToArray();
88.         r.sharedMesh.boneWeights = boneWeights.ToArray();
89.         r.sharedMesh.bindposes = bindPoses.ToArray();
90.         r.sharedMesh.RecalculateBounds();
91.     }
92. }

 

Hi Thank's for script i'm looking for help
for combine MeshFilter to SkinMeshRenderer
Model got animation any help is welcome
thank's

 

You're in luck. (Maybe)

This code will take a list of skinnedmeshrenderers and a list of meshfilters and combine them.


One thing that's weird about this, and makes it longer than it needs to be, is the code around the "altMaterial". In our game if a mesh has a certain material on it (the one passed to the function) we make it a separate sub-mesh with a different material.

If you look at the code that says "//now combine the face renderer if there was one" you probably don't need any of that. So you could delete "Materal m" from the function and then remove the code that relies on it.

The theory of this code is that a MeshFilter is basically a skinned mesh that just has one bone. (The transform it's attached to) So it adds one bone to the combined thing, and adds a bone weight for each vertex.

It's a bit complicated but hopefully it helps. I'm not sure about the TODO in there - the code works for me so I think ti's correct...

Code (CSharp):

1.  
2.    
3. using UnityEngine;
4. using System.Collections.Generic;
5.  
6. public class SkinnedMeshCombiner {
7.  
8.     private static List<Transform> bones = new List<Transform> ();
9.     private static List<BoneWeight> boneWeights = new List<BoneWeight> ();
10.     private static List<CombineInstance> combineInstances = new List<CombineInstance> ();
11.     private static List<SkinnedMeshRenderer> smRenderers = new List<SkinnedMeshRenderer> ();
12.  
13.     private static SkinnedMeshRenderer altMaterialRenderer = null;
14.     private static List<Matrix4x4> bindPoses = new List<Matrix4x4>();
15.  
16.  
17.  
18.     public static SkinnedMeshRenderer CombineMeshes(GameObject o, Material m, List<SkinnedMeshRenderer> renderers, List<MeshFilter> meshFilters) {    
19.         bones.Clear ();
20.         boneWeights.Clear ();
21.         combineInstances.Clear ();
22.         bindPoses.Clear ();
23.         altMaterialRenderer = null;
24.  
25.  
26.         Material altMaterial = null;
27.  
28.         int boneOffset = 0;
29.         for( int s = 0; s < renderers.Count; s++ ) {
30.             SkinnedMeshRenderer smr = renderers[s];
31.  
32.             //if the skinned mesh renderer has a material other than the default
33.             //we assume it's a one-off face material and deal with it later
34.             if (smr.sharedMaterial != m) {
35.                 altMaterialRenderer = smr;
36.                 altMaterial = smr.sharedMaterial;
37.                 continue;
38.             }
39.  
40.             BoneWeight[] meshBoneweight = smr.sharedMesh.boneWeights;
41.  
42.             // May want to modify this if the renderer shares bones as unnecessary bones will get added.
43.  
44.             for (int i = 0; i<meshBoneweight.Length; ++i){
45.                 BoneWeight bWeight = meshBoneweight[i];
46.  
47.                 bWeight.boneIndex0 += boneOffset;
48.                 bWeight.boneIndex1 += boneOffset;
49.                 bWeight.boneIndex2 += boneOffset;
50.                 bWeight.boneIndex3 += boneOffset;              
51.  
52.                 boneWeights.Add( bWeight );
53.             }
54.  
55.             boneOffset += smr.bones.Length;
56.  
57.             Transform[] meshBones = smr.bones;
58.  
59.  
60.  
61.  
62.             for (int i = 0; i < meshBones.Length; ++i) {
63.                 bones.Add (meshBones[i]);
64.  
65.                 //we take the old bind pose that mapped from our mesh to world to bone,
66.                 //and take out our localToWorldMatrix, so now it's JUST the bone matrix
67.                 //since our skinned mesh renderer is going to be on the root of our object that works
68.                 bindPoses.Add (smr.sharedMesh.bindposes [i] * smr.transform.worldToLocalMatrix);
69.             }
70.                
71.  
72.             CombineInstance ci = new CombineInstance();
73.             ci.mesh = smr.sharedMesh;
74.             ci.transform = smr.transform.localToWorldMatrix;
75.             combineInstances.Add( ci );
76.  
77.  
78.  
79.             GameObject.DestroyImmediate( smr );
80.         }
81.  
82.         //now combine the non-skinned renderers
83.         for( int s = 0; meshFilters!=null && s < meshFilters.Count; s++ ) {
84.             MeshFilter filter = meshFilters [s];
85.             MeshRenderer renderer = filter.gameObject.GetComponent<MeshRenderer> ();
86.             if (renderer.sharedMaterial != m) {
87.                 continue;
88.             }
89.  
90.  
91.             // May want to modify this if the renderer shares bones as unnecessary bones will get added.
92.  
93.             int vertCount = filter.sharedMesh.vertexCount;
94.             for (int i = 0; i < vertCount; ++i) {
95.                 BoneWeight bWeight = new BoneWeight ();
96.  
97.                 bWeight.boneIndex0 = boneOffset;
98.                 bWeight.boneIndex1 = boneOffset;
99.                 bWeight.boneIndex2 = boneOffset;
100.                 bWeight.boneIndex3 = boneOffset;              
101.                 bWeight.weight0 = 1;
102.  
103.                 boneWeights.Add (bWeight);
104.             }
105.  
106.  
107.             boneOffset += 1;
108.  
109.             bones.Add (filter.transform);
110.  
111.             //TODO figure out what this should be
112.             bindPoses.Add (filter.transform.worldToLocalMatrix);
113.  
114.  
115.             CombineInstance ci = new CombineInstance();
116.             ci.mesh = filter.sharedMesh;
117.             ci.transform = filter.transform.localToWorldMatrix;
118.             combineInstances.Add( ci );
119.  
120.             GameObject.DestroyImmediate( filter );
121.             GameObject.DestroyImmediate( renderer );
122.         }
123.  
124.  
125.  
126.         SkinnedMeshRenderer r = o.GetComponent<SkinnedMeshRenderer> ();
127.  
128.         if (r == null) {
129.             r = o.AddComponent<SkinnedMeshRenderer> ();
130.         }
131.  
132.         Mesh skinnedMesh = new Mesh();
133.         skinnedMesh.CombineMeshes (combineInstances.ToArray (), true, true);
134.  
135.    
136.         //now combine the face renderer if there was one
137.         if (altMaterialRenderer != null) {
138.             combineInstances.Clear ();
139.             CombineInstance mainInstance = new CombineInstance ();
140.             mainInstance.mesh = skinnedMesh;
141.             mainInstance.transform = Matrix4x4.identity;
142.             combineInstances.Add (mainInstance);
143.  
144.             BoneWeight[] meshBoneweight = altMaterialRenderer.sharedMesh.boneWeights;
145.  
146.             // May want to modify this if the renderer shares bones as unnecessary bones will get added.
147.  
148.             for (int i = 0; i<meshBoneweight.Length; ++i){
149.                 BoneWeight bWeight = meshBoneweight[i];
150.  
151.                 bWeight.boneIndex0 += boneOffset;
152.                 bWeight.boneIndex1 += boneOffset;
153.                 bWeight.boneIndex2 += boneOffset;
154.                 bWeight.boneIndex3 += boneOffset;              
155.  
156.                 boneWeights.Add( bWeight );
157.             }
158.  
159.             boneOffset += altMaterialRenderer.bones.Length;
160.  
161.             Transform[] meshBones = altMaterialRenderer.bones;
162.  
163.  
164.  
165.  
166.             for (int i = 0; i < meshBones.Length; ++i) {
167.                 bones.Add (meshBones[i]);
168.  
169.                 //we take the old bind pose that mapped from our mesh to world to bone,
170.                 //and take out our localToWorldMatrix, so now it's JUST the bone matrix
171.                 //since our skinned mesh renderer is going to be on the root of our object that works
172.                 bindPoses.Add (altMaterialRenderer.sharedMesh.bindposes [i] * altMaterialRenderer.transform.worldToLocalMatrix);
173.             }
174.  
175.  
176.             CombineInstance ci = new CombineInstance();
177.             ci.mesh = altMaterialRenderer.sharedMesh;
178.             ci.transform = altMaterialRenderer.transform.localToWorldMatrix;
179.             combineInstances.Add( ci );
180.             GameObject.DestroyImmediate( altMaterialRenderer );
181.  
182.             //this is kind of clunky but you can't combine a mesh into itself
183.             //so we just make a new one.
184.             //we could pool these meshes and use clear() I guess instead of constantly making new ones
185.             //and pool combine instances as well - not sure how much it matters
186.             skinnedMesh = new Mesh ();
187.             skinnedMesh.CombineMeshes (combineInstances.ToArray (), false, true);
188.         }
189.         skinnedMesh.RecalculateBounds ();
190.  
191.         r.sharedMesh = skinnedMesh;
192.  
193.         if (altMaterial == null) {
194.             r.sharedMaterial = m;
195.         } else{
196.             Material[] newMaterials = new Material[]{m,altMaterial};
197.             r.sharedMaterials = newMaterials;
198.  
199.         }
200.  
201.         r.bones = bones.ToArray();
202.         r.sharedMesh.boneWeights = boneWeights.ToArray();
203.         r.sharedMesh.bindposes = bindPoses.ToArray();
204.  
205.         //clear at end so we can free garbage
206.         bones.Clear ();
207.         boneWeights.Clear ();
208.         combineInstances.Clear ();
209.         bindPoses.Clear ();
210.  
211.     return r;
212.     }
213.  
214.     public static void CombineMeshes(GameObject o, Material m) {    
215.         smRenderers.Clear ();
216.         o.GetComponentsInChildren<SkinnedMeshRenderer>(true,smRenderers);
217.         CombineMeshes (o, m, smRenderers,null);
218.     }
219. }
220.     }

 

Thank's i will check

 

That's work for me !! Thanks a lot.