I found out how to do Zbrush matcap shading in Unity.

techmage · Jan 21, 2010

I have a found a very easy way to make Zbrush matcap materials work in unity with pre-existing shaders.

If you are not familiar with what zbrush matcap shading is, here is the simple of it.

You can load an image of a rendered, shaded sphere into Zbrush like these
http://omploader.org/vM2JlYQ

Just a basic sphere bitmap image, no special image data or anything is kept on those spheres. Like those images right there are actual matcap materials that could be loaded directly into zbrush.

Zbrush will then take that rendered sphere, and shade an actual model with it. Based on the normals of the model. So like say, the normal on the model is facing 90x,0z,0y. Zbrush will then look up the normal 90x,0z,0y on the rendered image of the matcap sphere, and then apply it to the actual model.

if that explanation doesn't make any sense, it basically means you can make models look like this
http://omploader.org/vM2JlZA
for essentially zero rendering calc. Because those models are not actually lit by any light sources, or even any shaders, it's just applying that rendered sphere to the model, matching the colors based on normals.

ANYWAYS. You can do this in Unity via utilizing the Toon Lighted shader.

Take a matcap material image like this one
http://omploader.org/vM2Jlbg

Load it into unity by putting it in your assets folder. In the texture importer properties set Generate Cubemap to 'SimpleSpheremap', you must use simpleSpheremap.

Then make a material with the toon lighted shader. And for toon cubemap, select the cubemap you just made.

Now the toon lighted shader lights your model based off of the direction of lights. So you must add one directional light, only one light for the whole scene, then point your directional light the same exact direction your camera is facing. And ideally you would only want to use this with orthographic cameras.

You should see your model now being lit like a matcap material. Although, it is probably upside down, so you need to rotate you directional light 180 degrees on the proper axis to make the matcap material appear rightside up.

You can see it in action here

As you could probably guess, this technqiue could be quite awesome for lighting 2D sidescroller games. Maybe if you get creative it could be used in 3D ones as well.

But there is one problem.

The cube on the very left of that image is a diffuse material using the matcap material as it's diffuse color. The sphere and two characters on the right are using the toon lighted shader with the matcap cubemap.

If you notice, the toon lighted shader is washed out in comparison to the diffuse shaded square. And the diffuse shaded square is the color it should actually be.

Could someone please help me figure how to make the toonlighted shader not make the cube map appear washed out? I think this would be a really awesome technique for alot of people if this washed out problem could be fixed.

techmage · Jan 21, 2010

Ok. I just found the solution to my problem. Gotta set the ambient light to 0

heres it is fixed and with a different matcap

However. I could still use some help on one thing.

Would it be possible in anyway whatsoever to have a model lit with this matcap material as I have just explained, and then have a point light be able to add diffuse lighting onto the model as well?

Also for some reason the outlines are not showing in the game viewport. Whats up with that?

KaelisAsur · Jan 21, 2010

Whoa. That ... that looks and sounds amazing!

A way to have pretty shaders, without actually having to code them - my dream come true! Albeit with limitations, but still!

Thank you for sharing this!

techmage · Jan 22, 2010

It is amazing! It's even more awesome to see it actually moving. Or with a really hi-res model. Because it so effectively can make something appear like it has a really fancy shader with subsurface scattering and a bunch of blurry reflections for less rendering time than even a diffuse shader. (If implemented properly, I dunno if using the toonshader in Unity does) But still quite fast.

Now if only someone could implement support for diffuse lighting over the top, maybe making it respond to normal mapping and letting you layer another texture on top of it. I don't wanna like ASK for someone else to do work for me but like... if you just so happen to think this could be of slight use to you, well I wouldn't object to you doing it and sharing it to the community, and alot of people would probably get alot of use out of it as well. Actually, to be quite frank, this could be a selling point for Unity, Unity Devs, you might actually want to do this yourself, I'm sure it wouldn't be hard if you knew what your doing. Matcap materials have alot of potential in games, especially where graphics card speed can be an issue. I mean imagine a little bullet point on the page explaining materials in unity "Has the ability to emulate any complex shaders with Subsurface Scattering and Blurry Reflections in 2D games with little performance hit even on SM1.1!"

Deleted User · Jan 22, 2010

Oo that's awesome! Thanks for sharing.

I wonder if it would work similarly with 3D Coat as well?
It's very similar to Zbrush, I use it more often because it's much more easier to use than Zbrush

techmage · Jan 22, 2010

3D coat uses matcap materials just like zbrush, 3Dcoat got the idea from zbrush. It is the same thing.

But you don't need either 3Dcoat or zbrush to make a matcap material. You can render a sphere in any application. You could make it in photoshop...

funshark · Jan 22, 2010

nice finding
But it will be really interesting if we can map this on a normal map 8)

Daniel_Brauer · Jan 22, 2010

I can probably write a shader this weekend that does this without any of the extra overhead that the toon shader might have, and I can provide normal map support.

Daniel_Brauer · Jan 23, 2010

Oh, also the toon shader outlines don't work with orthographic cameras.

Daniel_Brauer · Jan 23, 2010

Shazam!

Code (csharp):

Shader "MatCap" {

Properties {

_Color ("Main Color", Color) = (0.5,0.5,0.5,1)

_BumpMap ("Bumpmap (RGB)", 2D) = "bump" {}

_MatCap ("MatCap (RGB)", 2D) = "white" {}

}

Subshader {

Tags { "RenderType"="Opaque" }

Blend AppSrcAdd AppDstAdd

Fog { Color [_AddFog] }

Pass {

Name "BASE"

Tags { "LightMode" = "Always" }

CGPROGRAM

#pragma vertex vert

#pragma fragment frag

#pragma fragmentoption ARB_fog_exp2

#pragma fragmentoption ARB_precision_hint_fastest

#include "UnityCG.cginc"

struct v2f {

V2F_POS_FOG;

float2 uv;

float3 TtoV0;

float3 TtoV1;

};

uniform float4 _BumpMap_ST;

v2f vert (appdata_tan v)

{

v2f o;

PositionFog( v.vertex, o.pos, o.fog );

o.uv = TRANSFORM_TEX(v.texcoord,_BumpMap);

TANGENT_SPACE_ROTATION;

o.TtoV0 = mul(rotation, glstate.matrix.modelview[0][0].xyz);

o.TtoV1 = mul(rotation, glstate.matrix.modelview[0][1].xyz);

return o;

}

uniform float4 _Color;

uniform sampler2D _BumpMap;

uniform sampler2D _MatCap;

float4 frag (v2f i) : COLOR

{

// get normal from the normal map

float3 normal = tex2D(_BumpMap, i.uv).xyz * 2 - 1;

half2 vn;

vn.x = dot(i.TtoV0, normal);

vn.y = dot(i.TtoV1, normal);

float4 matcapLookup = tex2D(_MatCap, vn*0.5 + 0.5);

return _Color*matcapLookup*2;

}

ENDCG

}

}

}

Here are some nice matcaps I found on Google. Didn't pay attention to who they belong to, so don't go trying to make money off them:

techmage · Jan 23, 2010

Awesome daniel.

Theres alot of matcaps in the zbrush matcap library
http://www.pixologic.com/zbrush/downloadcenter/library/

Although I think those are actual zmaterial files, so you'll need to open them in zbrush to get them out.

I think you can get straight up images out of this thread
http://www.zbrushcentral.com/showthread.php?t=46175

Although, these are other peoples matcaps, to do all your own work its probably best to render your own.

Daniel I can't thank you enough for your help on this. But I have to ask, can you mix that with the diffuse shader to allow a person to mix a texture map onto the model (using UVs) and also add highlights from spot lights or point lights?

Deleted User · Jan 23, 2010

Awesome.
Quick question.. When you apply this shader and matcap texture to object and then rotate it or rotate camera, you can see the white/ black edges from the matcap texture. Any way to fix this or is it unfixable?

techmage · Jan 23, 2010

Maybe try turning off mip maps?

If you look at the very first matcap I posted, it extends the colors from the edges a bit, that could probably help as well.

Daniel_Brauer · Jan 23, 2010

djoscard said:

Awesome.
Quick question.. When you apply this shader and matcap texture to object and then rotate it or rotate camera, you can see the white/ black edges from the matcap texture. Any way to fix this or is it unfixable?
Click to expand...

I thought I would see this problem but I never did. To guarantee it doesn't happen, even with mipmaps still on (in Photoshop):

1. Make sure your sphere touches each edge of the texture
2. Select the non-sphere parts of the image and make them transparent.
3. Use Flaming Pear's excellent free Solidify filter, available in the free package here.

This will extend the edges of the sphere smoothly to the edges of the texture.

Also, turn anisotropic filtering way up for your matcaps.

Daniel_Brauer · Jan 23, 2010

eem said:

Daniel I can't thank you enough for your help on this. But I have to ask, can you mix that with the diffuse shader to allow a person to mix a texture map onto the model (using UVs) and also add highlights from spot lights or point lights?
Click to expand...

That can definitely be done, although you should note that this shader is not the cheapest due to having to calculate the viewspace normal from the normal map for every pixel. Its real advantage is how easy it makes it to provide a material definition. As soon as you add real-time pixel lights, you might as well graduate to a full-fledged reflective cube map shader.

As for mixing in a base texture, how would you want to do that? Because lighting and material properties are baked into the matcap, simply multiplying a base texture in has gross-looking results.

techmage · Jan 23, 2010

Daniel Brauer said:

As for mixing in a base texture, how would you want to do that? Because lighting and material properties are baked into the matcap, simply multiplying a base texture in has gross-looking results.
Click to expand...

I was thinking that it would add the color of the point light on top of the matcap coloring. So then if you have a character with a matcap material on him and he is standing next to a red point light, the side of him facing the light would be tinted red, but the rest of his lighting would be unchanged. The matcap itself would still account for the majority of making the character look how he should look. The diffuse lighting would be used just to make him appear reactive to the lighting in the surrounding environment by adding intensity and tinting to his color.

Then all of that multiplied by a texture placed by the UV's, like the detail channel on diffuse detail.

Daniel_Brauer · Jan 24, 2010

I don't really like it, but here it is:

Code (csharp):

Shader "MatCap Normal with Extras!" {

Properties {

_Color ("Main Color", Color) = (0.5,0.5,0.5,1)

_BumpMap ("Bumpmap (RGB)", 2D) = "bump" {}

_MatCap ("MatCap (RGB)", 2D) = "white" {}

_SpecColor ("Specular Color", Color) = (0.5, 0.5, 0.5, 1)

_Shininess ("Shininess", Range (0.03, 1)) = 0.078125

_MainTex ("Base (RGB) Gloss (A)", 2D) = "white" {}

}

Subshader {

Tags { "RenderType"="Opaque" }

Blend AppSrcAdd AppDstAdd

Fog { Color [_AddFog] }

Pass {

Name "BASE"

Tags { "LightMode" = "Always" }

CGPROGRAM

#pragma vertex vert

#pragma fragment frag

#pragma fragmentoption ARB_fog_exp2

#pragma fragmentoption ARB_precision_hint_fastest

#include "UnityCG.cginc"

struct v2f {

V2F_POS_FOG;

float2 uv;

float2 uv2;

float3 TtoV0;

float3 TtoV1;

};

uniform float4 _BumpMap_ST;

uniform float4 _MainTex_ST;

v2f vert (appdata_tan v)

{

v2f o;

PositionFog( v.vertex, o.pos, o.fog );

o.uv = TRANSFORM_TEX(v.texcoord,_BumpMap);

o.uv2 = TRANSFORM_TEX(v.texcoord,_MainTex);

TANGENT_SPACE_ROTATION;

o.TtoV0 = mul(rotation, glstate.matrix.modelview[0][0].xyz);

o.TtoV1 = mul(rotation, glstate.matrix.modelview[0][1].xyz);

return o;

}

uniform float4 _Color;

uniform sampler2D _BumpMap;

uniform sampler2D _MatCap;

uniform sampler2D _MainTex;

float4 frag (v2f i) : COLOR

{

float4 texcol = tex2D(_MainTex, i.uv2);

// get normal from the normal map

float3 normal = tex2D(_BumpMap, i.uv).xyz * 2 - 1;

half2 vn;

vn.x = dot(i.TtoV0, normal);

vn.y = dot(i.TtoV1, normal);

float4 matcapLookup = tex2D(_MatCap, vn*0.5 + 0.5);

return texcol*_Color*matcapLookup*2;

}

ENDCG

}

UsePass "Bumped Specular/PPL"

}

}

My issue is that the matcap encodes both diffuse and specular lighting, as well as whatever else you simulated, but because they're already combined you can't discriminate between them. Multiplying in the base texture affects specular in exactly the same way as it affects diffuse, reflection, emission, sub-surface scattering, etc. Unless you use a very simple matcap, I think a base texture is always going to make it look worse.

techmage · Jan 24, 2010

Thank you so much. This helps me considerably as CG code is the only thing in the unity development pipeline that I still stare at in complete confuzzlment and the only thing left in my game that I really couldn't comprehend how to do. This is a huge favor to me. If you ever need a favor, like a model or something rendered nicely, feel free to ask me. I can model, texture and make nice renders of pretty much anything.
Heres some stuff I've made
http://eem.cgsociety.org/gallery/
http://www.iconarchitect.com/portfolio/

And I know it is potentially kind of nasty in some ways. It is for sure faked lighting. I hold a background in archviz and mental ray with all physically accurate rendering. I know how nasty it is from that point of view. But I am going to be doing considerable amount of pseudo-lighting techniques in my game. And this is just perfect.

Daniel_Brauer · Jan 24, 2010

Cool. I'm not down on the technique in general. I just can't picture the base texture addition looking good. I am interested to see your results, though, because it looks like you know what you're doing!

RElam · Jan 25, 2010

I'm with Daniel, I think in the end you might find this to fall well short of your expectations. I think what you might have seen initially is the quality improvement from an image based lighting technique, but I think you'd be much happier with a more traditional image based lighting technique. Not sure if someone has written a shader for that for Unity, though.

Daniel_Brauer · Jan 25, 2010

RElam said:

I think what you might have seen initially is the quality improvement from an image based lighting technique, but I think you'd be much happier with a more traditional image based lighting technique.
Click to expand...

What sort of more traditional IBL technique are you referring to?

RElam · Jan 26, 2010

Daniel Brauer said:

What sort of more traditional IBL technique are you referring to?
Click to expand...

I guess traditional may not be the best word. But I'm referring to using a reflection cube (or light probe, whatever you like to call it), then down sampling and blurring it to create diffuse lighting cube, then using those cubes to generate your global illumination (essentially, replacing your ambient calculations, since they are the same information, one just looks alot better ). I'm just not sure of an external program that can create the maps for you. Most rendering applications seem to do all that internally using a different technique, since technically the only source material you need is the reflection cube (HDR preferably, or you lose bright lights).

But I'd done it dynamically to create the cubes, then downsampled and blurred, and used 4 cube maps when doing the actual object lighting. I'd just blend the values depending on the gloss level for the specular (each cube represents a different gloss level) and lookup with reflection vector, then add the diffuse looked up with the normal. The end result was a pretty acceptable approximation of the HDR/IBL stuff you'd see in rendering applications.

Only problem I had was seams in my reflection cubes, but I got them to a point where they weren't terrible, and hardly noticeable on 'real' content (stuff that wasn't a solid color sphere ). Something like http://developer.amd.com/gpu/cubemapgen/pages/default.aspx might fix that though.

So, if you could find a program that could create the images you need well enough, the actual math to do the lighting is fairly straight forward (downsampling cube shader, not fun at all!). I'd done it as a screen pass in a deferred shader, so I didn't do it per object, but speed shouldn't be that much different here, it's awful fast for how good it looks.

leeman1000 · Dec 24, 2010

Thank you very much this is awesome! For some reason the matcap with extras doesn't work for me, don't know why but here's basically the same thing with the texture working with the matcap and normal map.

Shader "Matcap/MatCap Bumped" {
Properties {
_Color ("Main Color", Color) = (0.5,0.5,0.5,1)
_MainTex ("Base (RGB)", 2D) = "white" {}
_BumpMap ("Bumpmap (RGB)", 2D) = "bump" {}
_MatCap ("MatCap (RGB)", 2D) = "white" {}
}

Subshader {
Tags { "RenderType"="Opaque" }

Fog { Color [_AddFog] }

Pass {
Name "BASE"
Tags { "LightMode" = "Always" }

CGPROGRAM
#pragma exclude_renderers xbox360
#pragma vertex vert
#pragma fragment frag
#pragma fragmentoption ARB_fog_exp2
#pragma fragmentoption ARB_precision_hint_fastest
#include "UnityCG.cginc"

struct v2f {
float4 pos : SV_POSITION;
float2 uv : TEXCOORD0;
float3 TtoV0 : TEXCOORD1;
float3 TtoV1 : TEXCOORD2;
};

uniform float4 _BumpMap_ST;

v2f vert (appdata_tan v)
{
v2f o;
o.pos = mul (UNITY_MATRIX_MVP, v.vertex);
o.uv = TRANSFORM_TEX(v.texcoord,_BumpMap);

TANGENT_SPACE_ROTATION;
o.TtoV0 = mul(rotation, UNITY_MATRIX_IT_MV[0].xyz);
o.TtoV1 = mul(rotation, UNITY_MATRIX_IT_MV[1].xyz);
return o;
}

uniform float4 _Color;
uniform sampler2D _MainTex;
uniform sampler2D _BumpMap;
uniform sampler2D _MatCap;

float4 frag (v2f i) : COLOR
{
float3 normal = UnpackNormal(tex2D(_BumpMap, i.uv));
// normal = normalize(normal);

half2 vn;
vn.x = dot(i.TtoV0, normal);
vn.y = dot(i.TtoV1, normal);

float4 matcapLookup = tex2D(_MatCap, vn*0.5 + 0.5);

matcapLookup.a = 1;

matcapLookup.rgb += tex2D(_MainTex, i.uv).rgb*0.5-0.4;

return _Color*matcapLookup*2;

}
ENDCG

}
}
}
Click to expand...

techmage · Dec 28, 2010

Unity3 broke the matcap with extras, something to do with how the specular pass is called changed.

DanjelRicci · Jun 18, 2011

Sorry to bring back an old post like this, but I would like to ask one thing...
Leeman1000 posted the working MatCap Extra shader for Unity 3.x, but I would like to add both Texture and Color alpha to it. Since I never wrote a Vertex Fragment shader, can someone lend an hand? Thanks.

EDIT: is it also possible to add a simple vertex lighting shading?

EDI2: Oh, I managed to add Alpha myself. Here you are, you may find it useful somehow.

Code (csharp):

Shader "SpeedGraphX/Super MatCap" {

Properties {

_Color ("Main Color", Color) = (0.5,0.5,0.5,1)

_MainTex ("Base (RGB)", 2D) = "white" {}

_BumpMap ("Bumpmap (RGB)", 2D) = "bump" {}

_MatCap ("MatCap (RGB)", 2D) = "gray" {}

}

Subshader {

Tags { "Queue" = "transparent" "RenderType"="Transparent" }

Fog { Color [_AddFog] }

Pass {

Name "BASE"

Tags { "LightMode" = "Always" }

Blend SrcAlpha OneMinusSrcAlpha Cull Back

CGPROGRAM

#pragma vertex vert

#pragma fragment frag

#pragma fragmentoption ARB_fog_exp2

#pragma fragmentoption ARB_precision_hint_fastest

#include "UnityCG.cginc"

struct v2f {

float4 pos : SV_POSITION;

float2 uv : TEXCOORD0;

float3 TtoV0 : TEXCOORD1;

float3 TtoV1 : TEXCOORD2;

};

uniform float4 _BumpMap_ST;

v2f vert (appdata_tan v)

{

v2f o;

o.pos = mul (UNITY_MATRIX_MVP, v.vertex);

o.uv = TRANSFORM_TEX(v.texcoord,_BumpMap);

TANGENT_SPACE_ROTATION;

o.TtoV0 = mul(rotation, UNITY_MATRIX_IT_MV[0].xyz);

o.TtoV1 = mul(rotation, UNITY_MATRIX_IT_MV[1].xyz);

return o;

}

uniform float4 _Color;

uniform sampler2D _MainTex;

uniform sampler2D _BumpMap;

uniform sampler2D _MatCap;

float4 frag (v2f i) : COLOR

{

float3 normal = UnpackNormal(tex2D(_BumpMap, i.uv));

// normal = normalize(normal);

half2 vn;

vn.x = dot(i.TtoV0, normal);

vn.y = dot(i.TtoV1, normal);

float4 matcapLookup = tex2D(_MatCap, vn*0.5 + 0.5);

matcapLookup.a = tex2D(_MainTex, i.uv).a*0.5;

matcapLookup.rgb += tex2D(_MainTex, i.uv).rgb*0.5-0.4;

return _Color*matcapLookup*2;

}

ENDCG

}

}

}

Good, eventually I have yet to figure out how to add simple lighting, but this shader is what I was looking for. And it works cheap and perfect on iOS!

leeman1000 · Oct 28, 2011

Cool Thanks! This is useful I was needing this. I did some quick merging with the matcap shader and the toon outline shader in case it's useful to someone. Matcap outline with small 128x128 matcap collection

Ippokratis · Oct 29, 2011

Hi,
Daniel Brauer : I try to figure out how to modify a little your shader so it works without a normal map.
Truth is I understand very little what you are doing there
I have commented with this color the parts I cannot understand.
Could you please provide some help / explanations ? Or perhaps some links ?
Thanks !

Code (csharp):

[COLOR=#b1b100]Shader[/COLOR] [COLOR=#ff0000]"MatCap Bumped"[/COLOR] [COLOR=#66cc66]{[/COLOR]

[COLOR=#b1b100]Properties[/COLOR] [COLOR=#66cc66]{[/COLOR]

_Color [COLOR=#66cc66]([/COLOR][COLOR=#ff0000]"Main Color"[/COLOR], Color[COLOR=#66cc66])[/COLOR] = [COLOR=#66cc66]([/COLOR][COLOR=#cc66cc]0[/COLOR][COLOR=#cc66cc].5[/COLOR],[COLOR=#cc66cc]0[/COLOR][COLOR=#cc66cc].5[/COLOR],[COLOR=#cc66cc]0[/COLOR][COLOR=#cc66cc].5[/COLOR],[COLOR=#cc66cc]1[/COLOR][COLOR=#66cc66])

[/COLOR] _BumpMap [COLOR=#66cc66]([/COLOR][COLOR=#ff0000]"Bumpmap (RGB)"[/COLOR], 2D[COLOR=#66cc66])[/COLOR] = [COLOR=#ff0000]"bump"[/COLOR] [COLOR=#66cc66]{[/COLOR][COLOR=#66cc66]}[/COLOR]

_MatCap [COLOR=#66cc66]([/COLOR][COLOR=#ff0000]"MatCap (RGB)"[/COLOR], 2D[COLOR=#66cc66])[/COLOR] = [COLOR=#ff0000]"white"[/COLOR] [COLOR=#66cc66]{[/COLOR][COLOR=#66cc66]}[/COLOR]

[COLOR=#66cc66]}[/COLOR]

Subshader [COLOR=#66cc66]{[/COLOR]

[COLOR=#b1b100]Tags[/COLOR] [COLOR=#66cc66]{[/COLOR] [COLOR=#ff0000]"RenderType"[/COLOR]=[COLOR=#ff0000]"Opaque"[/COLOR] [COLOR=#66cc66]}[/COLOR]

Fog [COLOR=#66cc66]{[/COLOR] Color [COLOR=#66cc66][[/COLOR]_AddFog[COLOR=#66cc66]][/COLOR] [COLOR=#66cc66]}[/COLOR]

[COLOR=#b1b100]Pass[/COLOR] [COLOR=#66cc66]{[/COLOR]

Name [COLOR=#ff0000]"BASE"[/COLOR]

[COLOR=#b1b100]Tags[/COLOR] [COLOR=#66cc66]{[/COLOR] [COLOR=#ff0000]"LightMode"[/COLOR] = [COLOR=#ff0000]"Always"[/COLOR] [COLOR=#66cc66]}[/COLOR]

CGPROGRAM

[COLOR=green]#pragma exclude_renderers xbox360

[/COLOR][COLOR=green]#pragma vertex vert

[/COLOR][COLOR=green]#pragma fragment frag

[/COLOR][COLOR=green]#pragma fragmentoption ARB_fog_exp2

[/COLOR][COLOR=green]#pragma fragmentoption ARB_precision_hint_fastest

[/COLOR][COLOR=green]#include "UnityCG.cginc"[/COLOR]

struct v2f [COLOR=#66cc66]{[/COLOR]

float4 pos : SV_POSITION;

float2 uv : TEXCOORD0;

float3 TtoV0 : TEXCOORD1;

float3 TtoV1 : TEXCOORD2;

[COLOR=#66cc66]}[/COLOR];

uniform float4 _BumpMap_ST;

v2f vert [COLOR=#66cc66]([/COLOR]appdata_tan v[COLOR=#66cc66])[/COLOR]

[COLOR=#66cc66]{[/COLOR]

v2f o;

o.pos = mul [COLOR=#66cc66]([/COLOR]UNITY_MATRIX_MVP, v.vertex[COLOR=#66cc66])[/COLOR];

o.uv = TRANSFORM_TEX[COLOR=#66cc66]([/COLOR]v.texcoord,_BumpMap[COLOR=#66cc66])[/COLOR];

TANGENT_SPACE_ROTATION;

o.TtoV0 = mul[COLOR=#66cc66]([/COLOR]rotation, UNITY_MATRIX_IT_MV[COLOR=#66cc66][[/COLOR][COLOR=#cc66cc]0[/COLOR][COLOR=#66cc66]][/COLOR].xyz[COLOR=#66cc66])[/COLOR];[COLOR=darkgreen]//What is the purpose here? I understand that you[/COLOR]

o.TtoV1 = mul[COLOR=#66cc66]([/COLOR]rotation, UNITY_MATRIX_IT_MV[COLOR=#66cc66][[/COLOR][COLOR=#cc66cc]1[/COLOR][COLOR=#66cc66]][/COLOR].xyz[COLOR=#66cc66])[/COLOR];[COLOR=darkgreen]// perform an operation in object space but not [/COLOR]

return o; [COLOR=darkgreen] // what exactly.[/COLOR]

[COLOR=#66cc66]}[/COLOR]

uniform float4 _Color;

uniform sampler2D _BumpMap;

uniform sampler2D _MatCap;

float4 frag [COLOR=#66cc66]([/COLOR]v2f i[COLOR=#66cc66])[/COLOR] : COLOR

[COLOR=#66cc66]{[/COLOR]

float3 normal = UnpackNormal[COLOR=#66cc66]([/COLOR]tex2D[COLOR=#66cc66]([/COLOR]_BumpMap, i.uv[COLOR=#66cc66])[/COLOR][COLOR=#66cc66])[/COLOR];

[COLOR=#808080][I]// normal = normalize(normal);[/I][/COLOR]

half2 vn;

vn.x = dot[COLOR=#66cc66]([/COLOR]i.TtoV0, normal[COLOR=#66cc66])[/COLOR];[COLOR=darkgreen]//Here too, what are you using the dot for ? [/COLOR]

vn.y = dot[COLOR=#66cc66]([/COLOR]i.TtoV1, normal[COLOR=#66cc66])[/COLOR];

float4 matcapLookup = tex2D[COLOR=#66cc66]([/COLOR]_MatCap, vn*[COLOR=#cc66cc]0[/COLOR][COLOR=#cc66cc].5[/COLOR] + [COLOR=#cc66cc]0[/COLOR][COLOR=#cc66cc].5[/COLOR][COLOR=#66cc66])[/COLOR];

matcapLookup.a = [COLOR=#cc66cc]1[/COLOR];

return _Color*matcapLookup*[COLOR=#cc66cc]2[/COLOR];

[COLOR=#66cc66]}[/COLOR]

ENDCG

[COLOR=#66cc66]}[/COLOR]

[COLOR=#66cc66]}[/COLOR]

[COLOR=#66cc66]}[/COLOR]

Daniel_Brauer · Oct 29, 2011

The lines you've marked create a rotation from tangent space to view space, and apply it to the normal. It's tricky-looking because it only produces the X and Y components, since the Z doesn't figure into the texture lookup.

Ippokratis · Oct 30, 2011

Hi Daniel,
Thanks for the explanations, they help me a lot.
I try to figure out what is going on but my inefficient knowledge shows.
I wrote what I currently understand from the above lines. If at some point you find the time to comment, that could help me a lot.

Things get tricky when you start with

TANGENT_SPACE_ROTATION;
Click to expand...

directive, for which to my shame I cannot find the documentation page.
I found it in many shaders but not documented somewhere ( have to search more ).

o.TtoV0 = mul(rotation, UNITY_MATRIX_IT_MV[0].xyz);
Click to expand...

Where the rotation came from ? I cannot see it declared, so it must be declared in another file, perhaps related to the above TANGENT_SPACE_ROTATION; declaration. From the syntax it seems that it is a matrix, is this correct ?

UNITY_MATRIX_IT_MV[0].xyz
Click to expand...

This seems to be the member of a matrix ? A specific element on the Inverse transpose of model*view matrix ?
What is really confusing is the [0] and [1]. They indicate two consecutive elements ? The [0] and .xyz are the "instructions" to that particular element of the matrix ?

The output of the UnpackNormal is a "per pixel normal" ?

In the line

vn*0.5 + 0.5
Click to expand...

what do you achieve ? I see this operations on uv coordinates many times, I cannot figure out its use.

Thanks again for the code and explanations,
-Ippokratis

Daniel_Brauer · Oct 30, 2011

TANGENT_SPACE_ROTATION is a macro from UnityCG.cginc. It expands to this:

Code (csharp):

float3 binormal = cross( v.normal, v.tangent.xyz ) * v.tangent.w;

float3x3 rotation = float3x3( v.tangent.xyz, binormal, v.normal )

I recommend looking through that whole file. It is fairly well commented and helps with a lot of details of Unity's renderer implementation. Slight nitpick: "binormal" should actually be "bitangent", because it is another tangent to the surface, rather than another normal. Of course, this is irrelevant to the compiler, but could help someone understand tangent space bases more easily. It would also be nice if "rotation" had a less generic name, for the exact reason that it confused you.

Looking more closely at the following code, I realize that it is a bit more complicated than I remembered. The objective is simple, though: to take the normal from the normal map, and rotate it into eye space. From there, its X and Y components map to the spheremap (matcap) texture. To perform this rotation, we need to make it all the way from tangent space through model and world space, to eye space. Keep in mind that because we are dealing with normals, we only want to rotate, not scale or translate. Also remember that rotation matrices are orthogonal, meaning that their transpose is their inverse.

Rotating from object to eye space is done as usual using the inverse transpose of the ModelView matrix. Since normal from normal maps start in tangent space, though, we need to use the inverse (transpose) of the Tangent space rotation as well:

Code (csharp):

inversetranspose(ModelView)*transpose(Tangent)*normal

There is a problem with doing this straightforwardly, though: the ModelView matrix is 4x4, whereas the Tangent matrix is 3x3, and I don't think it's possible to address a submatrix in Cg if it is not a single row. Also, we only need the first two rows of this result, because we only need the first two components of the transformed normal.

So this is where things get tricky. Remember that:

Code (csharp):

transpose(A*B) == transpose(B)*transpose(A)

So:

Code (csharp):

transpose(inverse(ModelView)) * transpose(Tangent) ==

transpose(Tangent*inverse(ModelView))

There is no UNITY_MATRIX_I_MV. However, because Cg uses a row-major matrix format, we can pull out the first two rows of UNITY_MATRIX_IT_MV using array index notation and they will be implicitly transposed by mul(M, v):

Code (csharp):

o.TtoV0 = mul(rotation, UNITY_MATRIX_IT_MV[0].xyz);

o.TtoV1 = mul(rotation, UNITY_MATRIX_IT_MV[1].xyz);

The result is that we have the first two columns before the final transpose, which will be the first two rows of the matrix we need after the transpose. These columns get passed to the vertex program, where the matrix-vector multiplication is expanded into a pair of dot products, thus implicitly transposing the incoming columns:

Code (csharp):

vn.x = dot(i.TtoV0, normal);

vn.y = dot(i.TtoV1, normal);

Finally, because the texture is in [0,1]^2, and our flattened normal is in [-1,1]^2, we must scale and bias the normal for the texture lookup:

Code (csharp):

float4 matcapLookup = tex2D(_MatCap, vn*0.5 + 0.5);

This took me a while, and left my head spinning (or perhaps transposing) more than once, but the result is a fairly neat transformation and flattening of the normals.

Ippokratis · Oct 30, 2011

Hi Daniel,
Thanks a lot for the explanations, they really help.

Boondock_Saint · Nov 24, 2011

Thanks Daniel for this crystal clear explanation!

dizzlebomb · Jan 30, 2012

what would a simpler modification to this shader look like that didn't use the bump/normal at all?

salmonax · Feb 23, 2012

Whoa, cool thread!

I found it while thinking about (hopefully) cheap ways to make the bump pop a bit more, but without using full blown cubic reflection maps and with the possibility of having easy control over the specular ramp.

About the problems with mixing the diffuse with a matcap: in Blender, the very best result I get in the GLSL viewport is by placing the matcap after everything else, inverting it, and using the "divide" blend mode. It gives the texture kind of a color-dodge effect in brighter areas and seems useful both to simulate post-processing in Unity Free (ie. a kind of bloom or overexposure effect) and to help the normal/bump maps pop a bit irrespective of the lighting. It works well with most texture/matcap combinations I've thrown at it, even that green glass one posted above. ((though the brightest values sometimes break.)

I knocked my head against it a bit to see if I could get it to blend like that in Unity, but to no avail quite yet. It's been my introduction to shaders, though, which has been great.

But I couldn't help wondering, is it really worth doing it this way? or would this method be too expensive for the subtle effect it provides, or best implemented by some other method? For instance, would it be a bad idea to have a scene with many different vertex-shaded, texture-blended matcaps?

wdcstudios · Apr 8, 2012

Has anyone had any luck runing the matcap shader on an iOS device? I always get some strange metalic effect no matter what type of matcap I use.

zbrushsculptor · Apr 8, 2012

Great Thread explains a lot ... there is a way as well to get matcaps to work in blender also ... and many tutorials on these ... I have a question though and it is are these matcaps showing up like they do just in the preview fast render mode in blender or as if they are rendered out ??? Through the use of the new bpr functions in zbrush I can get some very nice renders using the lightcaps etc mixed with the matcaps. But is this just showing up as a matcap before rendering ...

corey.agp · Jul 31, 2012

Hey All,

I love the look it gives to simple objects and it is really fast, I just need it to support batching and work on mobile. Has anyone developed this shader any further?

GiantGrey · Nov 6, 2012

This shader ist really cool! Is it possible to support lighting? Because as for now, the shader does not react on any lights in the scene. That would be awesome!!

Daniel_Brauer · Nov 6, 2012

propeller said: ↑

Is it possible to support lighting?
Click to expand...

If you look at the first page of this thread, there is a version of the shader that includes lighting.

GiantGrey · Nov 6, 2012

hmmm do you mean the MatCap with extras shader? This one seams really heavy.
Would it be possible to include lighting in the simplified MatCap shader?

GiantGrey · Nov 7, 2012

i've tried now all shaders in this thread and none of them works with lights
tried to build one with the strumpy shader editor but i dont even get close to the shader in this thread.
Would be great if anyone could help me out with this, im not a shader programmer

wednesdays · Nov 7, 2012

propeller said: ↑

i've tried now all shaders in this thread and none of them works with lights
tried to build one with the strumpy shader editor but i dont even get close to the shader in this thread.
Would be great if anyone could help me out with this, im not a shader programmer
Click to expand...

The whole point of matcap shading is that you don't use lights. The lighting is already baked into the matcap sample.

GiantGrey · Nov 8, 2012

wednesdays said: ↑

The whole point of matcap shading is that you don't use lights. The lighting is already baked into the matcap sample.
Click to expand...

yes i know! But look at Mudbox or Zbrush, those are using Matcap shaders and you can additionally change the light.

Search Unity

Unity ID

Useful Searches

I found out how to do Zbrush matcap shading in Unity.

Guest

Unity Technologies

Unity Technologies

Unity Technologies

Attached Files:

Guest

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies