Maya CGFX Shader : Inspired by Guilty Gear Xrd GDC Presentation

Junya C Motomura's gave an amazing GDC presentation of how Guilty Gear Xrd achieved it's incredible 2D look. And I was inspired to write a variant of what Junya presented as a Maya Cgfx Shader. Check it out, it's available on Github:

https://github.com/miketon/Maya_Tools/tree/master/Shaders/Toon_xrd

Notable differences :

* Team Red appears to be straight up using a dot product. This shader uses a dot product to do a uv lookup on a 1D ramp texture. Affords more flexibility with changing light-to-shadow palette via texture edit. Tradeoff is lower perf

* Team Red actually altered the normals, I'm using vertex color (red channel) to alter the uv lookup.

 

Nice to see that people have already begun to learn from their experience and experiment

Lit areas was only defined by the normals but shaded areas was defined by both the normals and one of the vertex color channels.
The reason that they did this was so that they could achieve a more dramatic, cartoon-ish and hand-painted shadows that oftentimes doesn't perfectly follow the laws of nature. Examples where they added the vertex paint was knees, below the knees, leg areas with heavy clothing and the characters neck.
Also, they manipulated the normals to achieve smoother shadow lines like Anime characters typically have. I personally see that step as a matter of taste and artistic freedom. Everyone won't want to use that trick because it won't suit every cartoon style

They had two textures because one was the base color and the second was the tint color. They used the tint to both calculate shadows and highlights.
Some clever math in the shader would allow you to define the number of steps in the ramp manually. Thus, you could in your case make a ramp with only two steps to achieve a B&W result of similar quality.

Another reason that they had 2D textures was because of the line-work in the character. Remember how they cut up the characters into additional mesh islands so that they could tweak the island vertices to achieve resolution independent details

 

Simple and nice use of vertex color with NdotL

 

I tried loading this in Maya 2016 and got this error:

cgfxShader -e -fx "C:/Toon_xrd.cgfx" cgfxShader2;
// Error: The compile returned an error. //
// Error: C:/Toon_xrd.cgfx(5) : error C0000: syntax error, unexpected '<' at token "<"
C:/Toon_xrd.cgfx(8) : error C0130: invalid character literal
C:/Toon_xrd.cgfx(99) : error C0130: invalid character literal
C:/Toon_xrd.cgfx(602) : error C0000: syntax error, unexpected '?', expecting "::" at token "?"
C:/Toon_xrd.cgfx(626) : error C0129: invalid char 'D' in integer constant suffix
C:/Toon_xrd.cgfx(630) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(702) : error C0000: syntax error, unexpected '?', expecting "::" at token "?"
C:/Toon_xrd.cgfx(726) : error C0129: invalid char 'D' in integer constant suffix
C:/Toon_xrd.cgfx(730) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(802) : error C0000: syntax error, unexpected '?', expecting "::" at token "?"
C:/Toon_xrd.cgfx(826) : error C0129: invalid char 'D' in integer constant suffix
C:/Toon_xrd.cgfx(830) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(950) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(954) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(954) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(958) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(958) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(962) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(962) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(966) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(966) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(970) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(970) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1322) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1342) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1386) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1422) : error C0000: syntax error, unexpected "+=", expecting "::" at token "+="
C:/Toon_xrd.cgfx(1462) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1940) : error C0130: invalid character literal //
// cgfxShaderNode::cgErrorCallBack //
// Error: The compile returned an error. //
// Error: C:/Toon_xrd.cgfx(5) : error C0000: syntax error, unexpected '<' at token "<"
C:/Toon_xrd.cgfx(8) : error C0130: invalid character literal
C:/Toon_xrd.cgfx(99) : error C0130: invalid character literal
C:/Toon_xrd.cgfx(602) : error C0000: syntax error, unexpected '?', expecting "::" at token "?"
C:/Toon_xrd.cgfx(626) : error C0129: invalid char 'D' in integer constant suffix
C:/Toon_xrd.cgfx(630) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(702) : error C0000: syntax error, unexpected '?', expecting "::" at token "?"
C:/Toon_xrd.cgfx(726) : error C0129: invalid char 'D' in integer constant suffix
C:/Toon_xrd.cgfx(730) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(802) : error C0000: syntax error, unexpected '?', expecting "::" at token "?"
C:/Toon_xrd.cgfx(826) : error C0129: invalid char 'D' in integer constant suffix
C:/Toon_xrd.cgfx(830) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(950) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(954) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(954) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(958) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(958) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(962) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(962) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(966) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(966) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(970) : error C1038: declaration of "UIWidget" conflicts with previous declaration at C:/Toon_xrd.cgfx(950)
C:/Toon_xrd.cgfx(970) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1322) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1342) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1386) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1422) : error C0000: syntax error, unexpected "+=", expecting "::" at token "+="
C:/Toon_xrd.cgfx(1462) : error C0000: syntax error, unexpected '&', expecting "::" at token "&"
C:/Toon_xrd.cgfx(1940) : error C0130: invalid character literal //
// Error: cgfxShader : "cgfxShader2" unable to load effect "C:/Toon_xrd.cgfx" //

Any help would be appreciated!