Dual quaternion skinning for Unity

Hi, I just tried this new version, I saw it on reddit. but I am getting this error with the shader:


I'm using unity 2018.2.20f1, so i'm not sure if maybe the shader is only for 2017?

I tried the "Minimal Skinning" version but it seems a bit odd....

 

The minimal skinning version is not supposed to be used in games. It's just there to show how passing the data from skinner works.

I tested the script with unity 2018.3.3.f1.

I will test with your version and write back.

 

Alas my unity hub refused to download the version 2018.2.20f1 with this error:


I tried several times and it's always the same.
So i downloaded the closest version i could find which is 2018.1.9f2

There was indeed an error but a completely different one.

It would seem that the code of standard shader changed a lot throughout last several versions and thus earlier versions of unity are incompatible with my altered version from 2018.3.3.f1.

Interestingly according to the feedback I received HackedStandard.shader works fine with unity 2018.2.16 which is supposed to be an earlier version than yours. Though I did not test it myself.

I will look for a way to make a more universal variant though i'm not sure it's quite possible. After all I'm editing a built-in shader.

You can try to make the same alterations to the Standard shader from your unity version if you really need to. I commented my edits in HackedStandard.shader so it should be pretty straight forward. The source code of Standard shader can be found here.



Meanwhile the script works properly with unity 2018.3.3.f1 (at least for me)

I will see if I can download your unity version tomorrow to be sure what's happening there.

 

Thanks for all your efforts. It is strange you can't get it to download. It's been the latest version for several weeks now.

I will try looking at the shaders myself tomorrow, I thought you might know a simple fix. But to my knowledge the shaders only changed between 2018.1 ->2018.2 and then 2018.2 -> 2018.3 I don't think they changed anytime during the 2018.2 cycle.

 

Looks like I have unknowingly lied. I told the guy whom I received feedback from to check whether the script worked with his version (2018.2.16) and if it doesn't try it with 2018.3.3.f1

I looked at the screenshot he sent me and turns out he was running 2018.3.3.f1
So most likely it did not work with 2018.2.16 either ¯\_(ツ)_/¯

 

I did a Hash of the standard.shader and the UnityShadowLibrary.cginc from 2018.2.20 and 2018.3.3 and they are identical. So the difference must be deeper it is odd because as you said, it does compile in 2018.3

Actually I can get it to compile in 2018.2.20, I see. but it's only as soon as I put it on an object in a scene, that it suddenly throws the errors. hmm

EDIT: okay I found the problem. It is working on 2018.2.20. But the shader seems to break if you set the Rendering mode to Cutout/Fade/Transparent. If it's set to Opaque, it works fine.

 

Also as an aside, I'm not sure this speeds anything up, but it makes for cleaner code:

Code (CSharp):

1. using UnityEngine;
2.  
3. public static class MatrixExtensions
4. {
5.     public static Quaternion ExtractRotation(this Matrix4x4 matrix)
6.     {
7.         //Vector3 forward;
8.         //forward.x = matrix.m02;
9.         //forward.y = matrix.m12;
10.         //forward.z = matrix.m22;
11.  
12.         //Vector3 upwards;
13.         //upwards.x = matrix.m01;
14.         //upwards.y = matrix.m11;
15.         //upwards.z = matrix.m21;
16.  
17.         //return Quaternion.LookRotation(forward, upwards);
18.  
19.         return Quaternion.LookRotation(matrix.GetColumn(2), matrix.GetColumn(1) );
20.     }
21.  
22.     public static Vector4 ExtractPosition(this Matrix4x4 matrix)
23.     {
24.         Vector4 position;
25.         //position.x = matrix.m03;
26.         //position.y = matrix.m13;
27.         //position.z = matrix.m23;
28.         position = matrix.GetColumn(3);
29.  
30.         return position;
31.     }
32.  
33.     public static Vector3 ExtractScale(this Matrix4x4 matrix)
34.     {
35.         Vector3 scale;
36.         //scale.x = new Vector4(matrix.m00, matrix.m10, matrix.m20, matrix.m30).magnitude;
37.         //scale.y = new Vector4(matrix.m01, matrix.m11, matrix.m21, matrix.m31).magnitude;
38.         //scale.z = new Vector4(matrix.m02, matrix.m12, matrix.m22, matrix.m32).magnitude;
39.  
40.         scale = new Vector3(matrix.GetColumn(0).magnitude, matrix.GetColumn(1).magnitude, matrix.GetColumn(2).magnitude );
41.         return scale;
42.     }
43. }

I changed ExtractPosition to Vector4 so you don't have to redo it a second time in DualQuaterionSkinner.cs:

Code (CSharp):

1.             //Vector3 pos = bindPoses[i].ExtractPosition();
2.             //bindDqs[i].position = new Vector4(pos.x, pos.y, pos.z, 1);
3.             bindDqs[i].position = bindPoses[i].ExtractPosition();

These changes seem to work fine for me.

 

I actually copy-pasted that piece from somewhere in the forum

Will add this change, thanks.

 

Do you think there's a way to fix the shader to work with transparent modes? I kind of need that.

 

I'm 90% sure it's possible. Most likely the vertex input structures I used are from opaque mode while in actual shader source code there is an #ifdef thats chooses the right input structure depending on the current mode.

 

Actually I found the issue. Parameters shouldn't include the 'out' keyword. Change the shader to this and it works for all modes:

Code (CSharp):

1.                 vertShadowCaster(
2.                     v,
3.                     opos
4.  
5.                     #ifdef UNITY_STANDARD_USE_SHADOW_OUTPUT_STRUCT
6.                         , o
7.                     #endif
8.  
9.                     #ifdef UNITY_STANDARD_USE_STEREO_SHADOW_OUTPUT_STRUCT
10.                         , os
11.                     #endif
12.                 );

(note 'out' keyword was removed from 'o' and 'os' variables)

 

omg I would read through entire shader archive several times before checking this.

Kudos to you for finding it.

*uploaded the fixed version to github

 

I read in another thread that you have a solution for integrating DQ with Pre-Integrated Skin Shader. Could you write what needs to be changed?

 

I'm afraid that solution is for older version of this script (before the blend shapes were added).
That version is still available on bitbucket but it is bugged and outdated. I do not recommend using it.

I proposed to add hooks into Pre-Integrated Skin Shader that would allow toggling DQ with a #define, but the author of Pre-Integrated Skin Shader never responded.

 

Yes this shader looks like it has been abandoned. Or maybe you could consider writing a short set of tips on how to integrate DQ with custom shaders in the future. HDRP / ASE support would also be great. I know it's a lot of work that's why I keep my fingers crossed. I think you could publish your asset on the asset store. I would buy it for sure.

 

Alas in Ukraine receiving money from asset store (or doing any other business outside the country) would require so much paperwork and taxes it's not worth it unless you can make at least several hundred bucks every month. That's why I released the script for free.

HDRP support would require almost completely rewriting the script from scratch. Unless I decide to work with HDRP for my own project it is not likely that I will add HDRP support.

Regarding ASE - I have not worked with it. It could be easy or it could be hard. Once again unless I use it for my own project it is unlikely that I will add support.

Regarding the tutorial - i commented heavily the changes I made to the standard shader and marked all the places where alterations were made. For someone who has enough experience with shaders it should be enough. For those who don't - alas it's not simply copy-pasting several lines so there is no way to create a simple and straight-forward tutorial.

 

Hi, this doesnt work with unity 2018.2.12f1, which version should i use for this?

 

I get a lot of errors here are the errors i get:


Assets/AssetStore/DQ-skinning-for-Unity-master/Code/DQ skinning/DualQuaternionSkinner.cs(117,6): error CS1644: Feature `interpolated strings' cannot be used because it is not part of the C# 4.0 language specification

Assets/AssetStore/DQ-skinning-for-Unity-master/Code/DQ skinning/DualQuaternionSkinner.cs(118,6): error CS1644: Feature `interpolated strings' cannot be used because it is not part of the C# 4.0 language specification

Assets/AssetStore/DQ-skinning-for-Unity-master/Code/DQ skinning/DualQuaternionSkinner.cs(440,17): error CS1644: Feature `null propagating operator' cannot be used because it is not part of the C# 4.0 language specification

Assets/AssetStore/DQ-skinning-for-Unity-master/Code/DQ skinning/DualQuaternionSkinner.cs(441,19): error CS1644: Feature `null propagating operator' cannot be used because it is not part of the C# 4.0 language specification

Assets/AssetStore/DQ-skinning-for-Unity-master/Code/DQ skinning/DualQuaternionSkinner.cs(442,26): error CS1644: Feature `null propagating operator' cannot be used because it is not part of the C# 4.0 language specification

Assets/AssetStore/DQ-skinning-for-Unity-master/Code/DQ skinning/DualQuaternionSkinner.cs(443,27): error CS1644: Feature `null propagating operator' cannot be used because it is not part of the C# 4.0 language specification

And like 10 more with the same end "cannot be used because it is not part of the C# 4.0 language specification", please help.

 

I've used it in 2018.2.20 and 2018.3.5

you should be on 2018.2.20 anyway, there's no reason to use an old subversion, there's only bugfixes, not project breaking changes.

 

These are just because it's using some newer c# features. You can fix the string ones by going back to old style parameters {0}, {1} in the string and putting the variables after the string and the others by explicitly testing for null instead of using the shortcut (I think, haven't actually compiled, just looked at the code.)

 

Well, now i´m using Unity 2018.3.3f1, still not working.

EDIT: I think i found how to fix that:
You have to enable .NET 4.x in the Editor in order to use the interpolated strings. feature.

Go to Edit --> Project Settings --> Player --> Other Settings --> Configuration --> Scripting Runtime Version --> .NET 4.x Equivalent.

 

Now i have problem with the model :/

 

If you are on version 2018.2.20 you can send me a minimal project that demonstrates the problem and I'll look into it.

 

I had that too, it's because the original skinned mesh didn't have transform position set to 0,0,0 just fix that, set position to 0,0,0 and all is fine.

 

I can probably fix the script to work as intended in such situation though. Thanks for mentioning this.

 

Hello @mr_madcake, suppose I need to compute the skinning on the CPU for various reasons. For the first step I tried extracting the new vertex positions from rtSkinnedData_1. I used a Texture2D and then test if the colors match the vertex position without any actual rotation performed. Since there was no rotation for any bone the vertex positions should not be modified. I guess I'm doing something wrong because for my test case the vertex with index 0 has coordinates (-0.02237231, 1.686474, 0.07623848) but the corresponding pixel in the Texture2D has values for rgba: (0, 1, 0.07450981, 1). I modified the code in the Update method like this:

Code (CSharp):

1.  
2. this.shaderDQBlend.Dispatch(this.kernelHandleDQBlend, numThreadGroups, 1, 1);
3.  
4. texture = new Texture2D(textureWidth, textureHeight, TextureFormat.RGB24, false);
5. Rect rectReadPicture = new Rect(0, 0, textureWidth, textureHeight);
6. RenderTexture.active = rtSkinnedData_1;
7.  
8. // Read pixels
9. texture.ReadPixels(rectReadPicture, 0, 0);
10. texture.Apply();
11.  
12. mf.mesh.GetVertices(verts);
13.  
14. //for (int i = 0; i < mf.mesh.vertexCount; i++)
15. for (int i = 0; i < 100; i++)
16. {
17.      int x = i % textureWidth;
18.      int y = i / textureWidth;
19.  
20.      Debug.Log(verts[i].x + ", " + verts[i].y + ", " + verts[i].z);
21.      Color c = texture.GetPixel(x, y);
22.      Debug.Log(i + "-> " + x + "; " + y + " = " + c.r + ", " + c.g + ", " + c.b + ", " + c.a);
23. }
24.  
25.  

Question: how would you go about extracting the modified vertex positions from rtSkinnedData_1 after applying the DQS algorithm (that is, after running this.shaderDQBlend.Dispatch(this.kernelHandleDQBlend, numThreadGroups, 1, 1);)?

 

I'm not completely sure I understand what are you doing but the texture format is wrong
It's not RGB24 but ARGBFloat

Look here

 

@mr_madcake - do you think it would be feasible to use the concept (DQ) to auto-rig clothing?

What I have in mind is starting with a rigged/animated avatar, but have all the clothing as unrigged models, and using the "skeleton" of the avatar to skin the clothing.

Obviously this is quite different than the current logic, but I was wondering if it could be the basis for something like this.

 

If I understand correctly what you are trying to achieve you might want to transfer weights from the character mesh to the clothing.

 

Hi,

It's me again. For a new project I recently tried to use your DQ scripts with Unity 2018.3.14.f1, and after downloading the latest version of your scripts on your git project, I encountered a problem with scales again, which I wanted to share with you.

As you told me last time, I have no scale on the mesh object or on the skeleton objects, but I have a scale of (0.01, 0.01, 0.01) on my parent game object. When I apply your script this is the result I get



However, as the script works when the scale is (1,1,1) I started to look into the code, and I noticed that your script works if I use the following line of code in the DualQuaternionSkinner.cs instead of what is shown in yellow in the picture

Code (CSharp):

1. this.shaderComputeBoneDQ.SetVector(
2.         "parent_scale",
3.         new Vector4(
4.             this.transform.parent.lossyScale.x,
5.             this.transform.parent.lossyScale.y,
6.             this.transform.parent.lossyScale.z,
7.             this.transform.parent.lossyScale.x
8.             )
9. );

At the moment it does work for me, but i wanted to share this bug in case you have an idea how to fix it more properly.

Cheers,

 

Thank you very much for reporting this!

I was trying to catch this bug for quite some time before my schedule went crazy and I gave up. This might help me to finally pin it down.

 

Hopefully fixed the scaling issue (release v1.2). There should be no more limitations of this kind now (as far as I'm aware).

 

Hi, Thank you for your efforts.

For the model that has flipped UV, tangent.w is used as sign value to calculate binormal for normal-mapping.
So, I tried to pass the original vertex_infos.tangent.w to skinned_data_3.y in DQBlend.compute and it worked for my flipped UV case.

 

Hi, I'm in interested in your DQ. btw, I read the threads, Which version of Unity3d do you recommend to use for latest DQ(1.4)?

 

The readme on github is always relevant to the latest release. The unity version for DQ v1.4 is 2018.3.3.f1

I'm planning to update it for current version of unity in about 1-2 weeks (this will be v1.5). It will also include a fix for tangent.w:

 

Candy-Wrapped joints can be fixed using regular unity`s skinning + twist / roll bones.

 

Ofc, there are different solutions to the problem.

Roll bones work well for arm/leg twist, not so well for collapsing shoulder. Also the volume loss happens for bending as well as twisting, roll bones only work for twisting.

DQ is not perfect (bulge artifact), but in most cases can achieve better results than linear + roll bones without requiring a more complex rig.

I'm actually working on a bulging compensation method for DQ skinning as my thesis.

 

Its Interesting, character setupwithout roll bones.
I guess you used Daz3d Character for sample pic in git. could you share example for a buldging compensation method?

 

The character was sent to me by another forum member whom I helped with setting up the script. I don't know where did he get it.

Regarding bulging compensation - it will take a lot of time before it can be used for real-world applications. All I can show at the moment is a proof of concept.

I manually entered several parameters that will have to be automatically estimated (this estimation is what I'm working on currently, among other things). This means final result will look worse than the proof of concept I have now, because the estimations will not be perfect. Though it will still look significantly better than default DQ.

When I get home I'll post a screenshot with a bent cylinder that I used to demonstrate the difference between default and bulge-compensated dq.

--------------------------------

Edit: found some gifs on my phone: Imgur

Final algorithm will look somewhere in-between this ("proposed") and normal DQ.

 

I've found a bug in conversion matrix4x4 -> DQ which caused divsion by zero in shader when rotating a bone exactly 180 degrees (which caused the model to become invisible).

This is fixed in version v1.5

Edit:

Fixed tangent.w (v1.6):

Please report any bugs.

Edit 2:

Tested with unity 2019.2.0f1, seems to work fine as-is

 

Good job, I will try soon.

 

Uploaded version v2.0-alpha.

It is a pre-release with bulge compensation. In order for bulge compensation to work, the bones must be oriented along the joints. The picture below demonstrates a model which would not work properly:


Select bone orientation axis and adjust the amount of bulge compensation with the slider:


Selecting wrong orientation will deform the model in weird ways (this is normal). If bulging is increased instead of decreased, choose same axis with different sign (Y => negativeY).

Areas which are likely to be problematic include face, shoulders and hands (in-between fingers).

-----------------------------------------------------------

Feedback is highly appreciated. If you have an undesired deformation with the script, please upload the model you used and a screenshot of the problem.

-----------------------------------------------------------

Edit:

In version v2.0-alpha.6 several major bugs were fixed, providing much better deformations, especially for areas, affected by more than 2 joints.

Version v2.2 was tested on a range of motion demo from this project and seems to be working as intended. Multiple bugs fixed since previous versions.

 

Hi, can I ask someone to post basic installation steps to get this script working? I'm an experienced developer, very familiar with VS, but brand new to Unity.

Thanks!

 

Thanks @mr_madcake for continuing to support and improve this! It is very helpful to use

 

My article, which details the bulging-compensation method I developed has been published and is freely accessible for anyone interested.

Notes:

• In formula (6) there is a mistake, x should be replaced by w_2 (there is no variable named x)
• Since publishing the article I found a better polynomial instead of (9), the coefficients are as follows: 2.2; -10; 11.2. The relevant commit: github
• In formula (12) there are weird brackets around V_bisector. This is a software bug, they should be ignored. This happens multiple times later in the article.
• The stretched region of the model in Fig (7), that looks weird even after applying the fix (though is no longer jagged), is actually caused by using improperly prepared armature. The spine and the shoulder have different local axes aligned along the joint. This contradicts one of the requirements listed at the top of page 17.

 

This is a nice work! I only wish you would provide a HDRP version of this.... would be awesome.

 

Thanks!

My problem with HDRP is this:

I really like customization and being able to implement myself things that were not provided by the engine (like DQ skinning). Seeing how implementing custom skinning for builtin render pipeline was quite a journey, I imagine implementing it for HDRP would be even more so.

Not to mention, I would have to update and support both versions separately, as HDRP is so different, I will have to re-write the entire thing from scratch.

It confuses me why would HDRP make a big thing out of giving the developers access to the source code, but then it turns out changing anything there is a pain in the ass, but it seems to be the case.

 

Hey there! this is really just a shot in the dark, to be honest most of what you do is beyond me, but would there be a way to modify the component to work with older versions of unity? 2017+ specifically,
Honestly just interested in modding older titles, this is honestly really amazing.

 

I'm 99% sure it's possible to backport the DQ script, but I have no idea how hard it will be. Could be done in 10 minutes, or could take weeks.

I might look into it someday, but in the foreseeable future I'm very busy and updates are unlikely. I'm defending a thesis on the bulging compensation method I developed for the script

Inserting this system in a game without having the source code, though, is going to be VERY hard, if at all possible.

 

Hi mr_madcake!

This is a great project! I wanted to show you my results. Notice the armpit area and the glute area keep the volume much better than the traditional linear skinning. I was able to remove about 24 "twist" bones and get a better result.


I also have a feature request. In 3ds Max you can limit the areas that use DQ skinning by painting the influence, screenshot below. All the black is linear skinning and the white blends into dq skinning. It's a good way to prevent knees and elbows from ballooning. But I'm guessing this may make the shader more expensive at it will need to calculate DQ and Linear skinning for the verts.

If you're able to do this, the vertex alpha is a good place to read the influence value.