Assets DD Standard Toon

As part of my research into toon shading I came across David Leon's blog post on cel shading. Someone in another thread mentioned they couldn't get this to work in 2017, so I made this version from the 2017.1.0f3 shader source. Then went back and read the tutorial and added a deferred shader to the one I'd done for the forward path. Since then I've added a number of features to give more control over the stylised lighting.

The DD Standard Toon can be used alongside other shaders in forward and legacy deferred paths. It's a straight swap for the Standard shader on materials.

The image above is forward, linear and uses post processing stack with filmic tone correction, ao and others. It was done before I added outline, so see below posts for forward examples with outlines.

I used a Light Range of 0.05 Black & 0.5 White for this image - this gives half of the light as fully lit, and mostly a soft shadow area. 0.5 Black & 0.5 White would give 50/50 unlit and lit, and you can adjust these to taste to play with different lighting. Confusingly 0.0, 0.0 is fully lit and 1.0, 1.0 is fully dark but otherwise the controls behave as expected including reversing the range with 1.0 Black and 0.0 White.

The DD Deferred Toon shader can be used with Standard shaders and should be chosen as the Deferred shader in graphics setup. There's no point in using Standard Toon for deferred as the material settings for outlines and light ramp are ignored in deferred. You'll need a post effect such as the one in Legacy Image Effects for outlines. The Light Ramp can only be set in the shader code for deferred and will be the same across all models.

The forward versions are the more flexible ones giving per material settings for light ramp and outlines.

 

Had a bit of a play in artist mode:

I changed smoothstep to 0.46, 0.5
I adjusted the lighting and turned Environment Lighting Intensity down to 0.5
I fixed the emission on the clouds.
I also added the Edge Detection from Legacy Image Effects.

I definitely need to bring the smoothstep settings up to the gui

 

Testing deferred with an additional directional light. For such a simple modification it's proving very flexible in getting different styles. For some reason I've always though of toon styles as minimal / no lights, so experimenting with this is somewhat enlightening

I've uploaded a new version in original post which now includes a specular version. I think that covers all the Standard shader workflows now.

 

Testing deferred with specular shader. I'm as speechless as these guys. I've totally underestimated how useful this approach would be.

 

That's really nice looking. Love it!

 

Yet again I forgot to look at the download count before uploading the new version. Oh well.. Feel free to post pictures or suggestions if you do try it out.

There's now a modified gui with controls for the edges of the "Light Range". It doesn't pass through the settings if you are on the deferred render path, for that you can edit the _EdgeWhite and _EdgeBlack variables in DD_StandardToonDeferred.shader file.

I renamed DD_StandardSpecularToon.shader to DD_StandardToonSpecular.shader, so delete the old version after updating your materials.

I tried it out on the UT Corridor and the more realistic materials do kill the cel shaded look somewhat, and GI finished it off. At least I remember now why I think of toon styles as having limited lights (and material colours!)

If anyone can point me at an example of setting a variable in a deferred shader (if it's possible), it would be appreciated!

 

Work in progress:


Added outlines for forward opaque, this also shows a different light range (black 0.6 white 1.0) on the back figure. I'm going to do a bit more work on the outline gui options so it can be disabled easily.

 

I've updated the download with the latest version that now supports outlines. Damn, just remembered I forgot to compare my naming convention with Unitys outline shaders. Might be another update once I do that, but otherwise I think this is pretty much done - unless I get tempted to make outlines work in transparent modes.

Extra controls over standard shader:

 

Looks quite excellent! Don't worry about download count, people will drop by over time Regarding toon, your above shots are quite stylised. A more cartoony look comes from higher ambient (not blacks).

 

Yeah, still very much in programmer art territory here. The whole point of this shader was to allow faster experimentation with different stylised looks, so hopefully more experienced artists will pop in and show what it can really do. In the meantime, here's an experiment with 3 directional lights at 120 degree steps. I should really replace one of those with ambient now you mentioned it

 

I think he's a bit surprised I got fade working!



Latest version in first post now supports outlines in Cutout & Fade modes and _OutlineWidth was renamed to _Outline to match standard assets toon shaders.

 

LOL!

 

Hey man, I forgot that you were developing this one. Seems pretty good! Will give this a shot tonight

 

Thanks! It seemed the best way to get a feel for how a multiple light supporting toon shader would work. I am still planning "DD Enhanced Toon" that will bring in some of the Guilty Gear XRD techniques and other ideas for stylistic rendering.

 

Nice looking effect!

 

I was experimenting with the different styles of shading that adjusting metallic and smoothness could get and I noticed that the specular lighting should really be using the Light Range too. I've uploaded a new version that does this in the first post.

All these have a Light Range of 0.25, 0.25 except for green where it was altered to give the softer look. Otherwise the different looks are from just altering metallic and smoothness.

 

Another style experiment, this one with Unity Chan. The Light Range came in handy here, the eyes have a range of 0.0 - 0.05, the skin has 0.2 - 0.25 and the rest is 0.45 - 0.55, this gives a nice effect going into shadow with the eyes being the last to darken. It does make the skin a bit bright and it'd need texture changes to fix that.

I set the the outlines to a similar colour to the part with just ambient light on and then made them brighter so they show in the shadows.

 

An experiment with a 64 x 64 noise texture on both texture & detail normal slots. I stretched it out in different directions on each one. I used a gradient ambient and two directional lights with the deferred version of the shader, so outlines were done in post. It could benefit from a few more polys, adjusted mesh normals and a proper brush stroke normal map (on detail only?), but for a quick attempt at making low poly vertex coloured meshes look more handpainted, I'm pretty happy with the result.

 

Following on from yesterdays experiment I wanted to see the difference more realistic normal maps would make. I used a single directional light with no shadows, and one spotlight (attached to camera) that does hard shadows. I mostly used negative scales on the normal maps, they felt more toon that way <shrugs>. Some normal maps looked to have too much detail, so I reimported those at lower resolutions. Temporal anti aliasing was a must have post effect to help kill the fireflies.

 

Nice work!

 

I've been mostly in research mode this past week or so. I'm trying to avoid adding a texture for the light controls, I think that is more suited to a non PBR inspired workflow. But I do want more control over the lighting so instead of a simple off - on beyond the smoothstep range, the minimum and maximum light influences could be set. I'm leaning towards combining all the light control settings into a vector4 with a custom drawer. That'll break compatibility with the current version so if anyone has any feature requests, now is a good time to let me know. I kinda want to put the light controls on a light rather than the material, I think I'd have to go to a scripted render loop to do that though.

I did a bit more experimenting with the hand painted style, with some post effects on my previous test. I changed the canopy normal map and added a couple of oil painting effects - one from Vetasoft's Camera Filters and the other from shaderslab. They cancel out each others artifacts, I wasn't happy with either on it's own. I'm starting to develop a preference for the deferred pipeline for this style. It's relying on lots of little changes adding up, and the extra lighting detail in deferred helps with that. The scene felt like it should be raining and Vetasoft helped there too - though it does work against the spotlight a bit

 

Sounds like a good call to me to rely on a light source and not on a material .

 

Yeah, though I'm not so sure now I've updated things. I've uploaded a new version which adds a couple of features, remapping the light range and I made the specular toonification optional. The remapping does kinda make sense on the material even though it affects the lighting. I can see me using it to fake light transmission by lifting the blacks. It's a bit like having a per material ao adjustment. Here's the latest controls layout. Min/Max do the same as the ranges.



Quick deferred test of the Map Range.



Sorry about repeating the ugly choice of photo position, I was watching mip transitions when I set it up to move there and I'm stuck with it for easy A/B tests.

If anyone is wondering what difference this shader makes, here's one using the normal deferred shader.



It kinda makes me want to use different settings on the terrain, which means using the forward path.

Oh and I abandoned the Vector4 idea so it's settings are still compatible with the old version.

 

Updated my Unity Chan test to use the new features. This is in forward with various light Map Ranges (I really need a less confusing name for that!) settings.

 

I can understand how using the lights to drive the shader would be appealing. No idea why Unity makes this so hard.

Perhaps you could somehow just reference the values from a particularly-named light source and insert those into the shader's Inspector GUI, perhaps getting access to this through Reflection? I've always wanted to do this myself, but I didnt have time to learn how to go about it specifically. Just something to think about to avoid scriptable render loops for something so simple as changing a value in the GUI alone.

I've been looking at this technique as a possibility for toon-based shading for some time, and it looks really promising, but there's an issue I've had for a long time with toon shaders and that revolves around 4 things:

1. Not being able to set their outline color on a part-by-part basis (a hat should have a different color than the head/hair and should be able (but not required) to be affected by light on one side and shadow on another, and the artist should be able to define both), and of course, the outline should be optional.
   
2. The fact that having a nice outline depends on having perfectly smooth normals -- In MOST cases with toons, modifying the normals to ensure light pools in all the right places is not only important, but entirely necessary -- especially on anime styles (i.e. look at how hideous Unity-chan's face looks without proper light/shadow poolings) and lots of times, this is handled in one channel for lights and another for shadows (i.e. R for highlights, B for shadow poolings, where R simply sets the normal to a custom value upon rendering, and outlines use the actual model's normals for rendering -- this avoids outlines that appear/disappear at the wrong angles, and genuinely lets a toon look like a toon)
   
3. Not having an extra-dark and extra-light shade (which is blended depending on the light angle) that could be used in the outline color so that it could give a better sense of volume to the 2D-looking toon model.
4. Being unable to change the tint, color, and brightness (hue/sat/lum) of each color in ramp while also affecting how softly or sharply each ramp color blends with the next one up/down from it (or not!)

Do you think you could solve any of these issues? I know deferred is a whole other animal, but these are essential to being able to tweak to cover the huge range of toon styles. As a 2D pixel artist/animator who has done various styles and renderings throughout the past 17 years, I think I've got cartoon styles down to a science. The main thing that changes between them is color, color-count, outline (color, thickness, texture), no outline, and texture -- everything else, at least physics and material-wise, is typically respected (so yay for 3D!) -- and if you want to render them in 3D, assuming you'd like to be comprehensive here, the above points are really all you need to keep in mind outside of what you've already done!

Good luck man, and great job so far! -- I really hope you can find a way to do the stuff above in Deferred rendering! It will be a first if you do!

 

It's my impression that there's a single light class and the render loop sends portions of it's data to the shader depending on the light type. So apart from directional lights where I could store 3 floats in it's position data, or perhaps abusing the cookie texture, I couldn't think of a way to simply add extra data to a light that would be passed to the shader without affecting other features. I could possibly use a script to set shader vars but timing the changes makes the render loop the ideal place for that.

I've mentioned that this is really a test platform to help me plan an enhanced version. I consider any features that need custom data to be for that one. So I really appreciate you taking the time to post your thoughts. I've not done much research into the deferred pipeline yet, it's on my todo list to get the displaced outline method working there and I should have a much clearer picture of what's possible once I tackle that.

One of the problems is trying to find a generic set of data that supports all styles while keeping an eye on performance. I've been leaning towards a variation of a ramp texture that stores a palette of something like: Base Color, Shadow Color, Highlight Color, PBR stuff, Outline Color, Inline Color for each entry. Lookup could be controlled by a 2nd UV set.

UV X would select the palette entry, neighbour color blending would be controlled by subpixel X adjustments and similarly UV Y would control the Inline with subpixel adjustments and negative wrap to the inline. This would let each triangle use up to 3 neighbouring pallete entries with some blending control and optionally have one edge with a scalable inline.

It'd limit the application to vertex colouring, which is why I've been experimenting with how flat colours + a normal map look.

For outlines I was planning to have a separate normal, and I'd kinda dismissed limiting it to one channel out of hand, so it's interesting to hear that might be enough. (Edit: reread your post and am aware it's still two channels really). I am planning feature parity with the Guilty Gear XRD outlines, with per vertex widths and depth modifiers.

I have considered whether I should use the AO channel on this shader as a threshold modifier for the lighting. It's certainly on the must have features for the enhanced version. Wow I wrote a lot before getting to your points!

1. It's likely the Shadow Colour and Highlight Colour will be stored so that it's just a multiply on the base colour, would reusing these on the Outline Colour be enough or would you want Outline Shadow/Highlight Colours adding to the palette? Does tying the Outline Colour to the palette make sense or would you prefer to be able to select a palette entry separately for the outline?
   
2. I think I can solve this one in enhanced version.
3. Unless you are asking for Dark Shadow Colour and Bright Highlight colours on the palette I think I covered this in 1. Later you mentioned outline texture which I'll pick up here, do you have any examples of textured outlines? I'm struggling to picture how they would be used.
   
4. Is this design time or run time for effects?

Thanks. I plan on making things as comprehensive as possible, but there will be compromises I'll have to make. Hopefully having forward only features won't always be one of them.

 

This shader does help a bit with that, but having that control per vertex is planned for the enhanced version.



It does show why being able to choose a shadow colour would be useful though, especially in the bearded sixth shot

 

Sounds like you and I are both on the same page then. That's great to hear -- GGXrD is a great example of what a good toon shader can achieve. However, even the creators of GG said themselves that they had a lot of improvements in the pipeline for their shader, so it's clear their approach CAN be improved. They even mentioned that they had already made some improvements on their approach before they even gave their GDC talk on it. It was watching that talk that gave me faith that a good 2D look in 3D was possible, and as a 2D guy, I looked into studying shaders myself to figure this out.



Anyway, before I answer your questions on my points, I'll first say that, above all, maximum flexibility in changing the overall look and feel is top-priority, so wherever you can give the user a choice that impacts the look heavily (i.e. color, texture, tone, lighting, outline definition) it's best to do so. A good example of this is the newest Unity Chan shader. It is a great base of operations due to its customizability, but it unfortunately lacks the kind of outline coloring/lighting and separate normals from the model's default normals (for outlines) vs lit/unlit normals (for light/shadow pooling control), plus it seems to use textures for everything unfortunately, so performance might be an issue in the longrun.

Being able to define pools of light and shadow via normals that are independent of the model's actual normals is important because the outline depends on the normals being perfectly-defined in order to have a nice, unbroken, outline (due to wonky backfaces when scaling). This is vital, as interpreting the light pools and shadow tendency means interpreting the normals stylistically anyhow to light it properly without artifacts.

For performance, the less textures required, the better, but a normal map and another texture with various channels to determine vertex-based lighting/shadow pooling, (and perhaps an [optional] AO texture to help with explicitly-textured 'rough-surface' normals) would allow for a more 'detailed' mode than anime tends to go for while also still allowing one to ignore more detail and processing than one needs for the style they're going for. The only other texture I could see would be required would be, as you mentioned, a palette texture, but this too could really be used heavily, as only, say, 1/4 of the texture would be necessary for palette entries -- the rest of the texture could be used for other stuff, such as thick/thin outlines, as per the GGXrD specs, or multiply colors for shadow areas (allowing the potential for palette-swapping). The final 1/4 of the texture could be used for logos or other decal-based textures. This could, optionally, be 1/2 of the texture, assuming palette entries could be squished together decently enough. Essentially 3-5 textures, total, increasing only if you want specular/roughness (can be combined) and normal/ao maps (also possible to combine), which is about standard for your typical model.

1. I'd say add them to the palette -- I can envision plenty of scenarios where, for example, I might want a water-colored style look to an otherwise typically-colored character, and the only way to do this is to modify the outline color directly, for both light and shadow portions of the outline. Here's (the middle image) sort of an example of that:
   
   As you can sort of tell, the middle Link would look great if I could tweak the color of the outline on various parts of his body to interesting hues/shades, as well as alter its thickness. The Link on the right uses the Unity Chan shader, which looks really nice with the slightly softer highlight-to-shadow area, but the outline looks horrible on him due to the manually-edited normals that help with light-pooling. Because he uses a lot of textures (each color in the ramp requires me to specify a texture -- which is the same for each shade -- and a color, which it uses to blend the texture with) and he can't use an outline, so ultimately, he isn't able to achieve the look I really wanted.
   That's pretty much where I'm at with my own shader stuff atm.
   
   
2. As you can see in the above image, I didn't include an outline on the right-most Link because it just looked bad as a pure black and in certain zoom levels, it didn't look right without Unity-smoothed normals.
   
   
3. You described exactly what I was talking about here in 1, but I had forgotten to elaborate on the colored-outline portion I sort of described in this reply in my response to 1 here, illustrated by the image above, center Link.
   
   The best example I can give of Textured Outlines is outlines that look like color-crayons or sketchy-styled stuff -- you see these a lot in mock-up styled or architecural/blueprint-looking illustrations, but I can imagine they'd be very useful for actual 2D-looking toon outlines too in some creative cases!
   
   
4. This would be design-time of course -- performance is always an issue with complex shaders like this, but I hope I addressed that issue with the top portion of this post in my suggestion on how this shader might be approached.

I hope this isn't too terribly much to read, but I hope it's all at least relevant enough to be useful. -- If you could manage making a shader like this, I can pretty much guarantee you'll get plenty of support since there's nothing out there like this available to the public at least.

Let me know how you feel about these suggestions.

 

Would it be difficult to write a post processing effect to do the edge detection and draw the lines in the pixel colors ? I haven't tried, just entertaining the possibility.

 

I don't think it'd be easy. The outline is outside the mesh, so it'd pick up the background colour rather than the mesh one. There's other limitations on controlling post outlines such as per vertex scaling not being possible, matching fade to the object, and so on that really limits what styles it can be used for.

 

This *could* be done as a PostFX thing (i.e. for the full screen), but as @DominoM mentioned, it'd be really hard, if not impossible (at least practically), to take into account the model's actual geometry (normals, textures, etc.) using only pure *pixels* while drawing the outline...

On the other hand... since I had considered this myself once...

A potential workaround is to grab the color for the outline from a palette texture + uvs and perhaps store the results of the vertex scaling there, but I think this probably would end up requiring its own custom render-loop to pass around the required data efficiently (if data can be passed around at all, as I'm not familiar enough with custom render loops yet to say for sure if this approach would work). That being said, if this *was* done as a postFX, it would be much faster to do a lot of objects at once, since it'd be a single pass only, and if it were possible to color the outline based solely on the silhouette and nearby vert data, it could create some nice 2D outline effects that could potentially be the best of both worlds. The issue would be finding the silhouette -- but in 2D, I think that could be potentially much easier, especially if you process the verts to the screen pixels and use a sort of vector outline to scale the thickness of each line. But again, performance could be an issue if these silhouettes get too complex. Only testing could tell for sure.

 

We're at least in the same book

It's deciding what point in the pipeline those choices have to be made where we may differ. GGXrd seems to have all the choice made at the art production stage, I doubt the shader has any options to affect the look, it's all done via the data supplied by the artist.

I'm aiming for somewhere similar for the enhanced version, probably just disabling features if data for that isn't provided. Like say an optional albedo modification texture (mapped to -1 to 1 and added to the shaded palette colour) for those occasions when vertex colouring isn't enough.

I worry about the ease of producing the data the shader needs too. It doesn't matter what the shader can do if the artist can't use the features easily. Trying to do everything possible could really hurt ease of use.

There is for example silhouette style outlines which is a image based approach that wouldn't need Outline Normals and isn't on my roadmap at all. Though now I think about it, they would make doing textured outlines easier.. There's so many style dependant choices to make

Can we just refer to them as Lighting Normals and Outline Normals? I've been thinking of the lighting ones as the models actual normals as outline normals can be generated, the lighting ones will be artist supplied. So I'm having to read very carefully to follow you sometimes. I'm a little confused over your descriptions of lighting normals and shadow normals, do you mean there are three sets of normals in total? Aren't light and shadow pooling part of the same function? Are there some circumstances where controlling the light normals separately to the shadow ones makes sense? I can't picture how that would play out. Lighting Normals + threshold offset seems more intuitive to me.

I'm really thinking of a global palette in most cases. You could assign say 8 colours per character, and have a bunch of consecutive variations of those in the palette. Changing the entire character palette is then done just by adding multiples of 8 to the UV X lookup. It makes 8 bit style colour cycling easy too, just duplicate the section of the palette and cycle with a looped UV X offset.

Each data point in the palette is another texture read, I don't know how much I can push things there until I've decided where all the data goes. There will be a point where using a light warp texture would be a better idea but I've no idea where that is yet

I'll keep textured outlines in mind when I work on hatching features, there's similar issues in doing both of those (such as using screen space for UV co-ordinates).

I've probably misled you a bit by saying my palette was like a ramp, when it's really just data. I'm still throwing ideas on how the palette will be edited around in my head.. Mostly likely route will be 'Colour' scripted objects that are dropped into palette slots and a bake of the palette object will produce the palette texture. Palette management is my biggest ease of use concern. I'm a little wary of adding more shade levels (I prefer the look and flexibility of extra lights on a two tone base), but I'll have a think about ways the blending could be controlled - I was already thinking of optional dithering rather than a blend / step between shadow & lit, but there's curves and neighbour colour blending to consider too. It's another point where light warp textures start to look like a good option.

 

One of the big temptations to go for a scripted render loop is to set up a deferred style system with enough info for the styling and some of the 'post fx' to be done in the final pass. Then things like the oil painting effect could be set up per object and things like the palette could be set up once instead of per material. I should probably plan on doing a pro version after the enhanced one, I'm keeping half an eye on scripted render loops but wasn't planning on tackling them until Unity 2018 or out of necessity, or maybe clear coat envy, whichever comes first

 

I see, sorry for being unclear. I was referring to the pools of shadow as more of a tendency for light to *not* affect the surface (in the case of GGXrD, take the image of the guy with the long trench-coat or the inside of a jacket or hood where ambient light would very rarely reach -- except in circumstances of almost direct light. This would be possible to define with a single color channel, and would likely affect normal calculations somewhat independently, if not completely, so yes and no -- the difference between it and an AO texture is that this version of AO is *not* texture/image-based, but vertex-based instead.

I do like this idea -- brings me back to my spriting roots! -- Yes, in this case, I could see 8-10 being enough.

If you went this route, would characters such as the Link image above (where the clean shadows across painterly texturing would actually make some styles look really great) still be possible? If the artist could pick colors to match their texture better, that's really useful, but if it were also possible to implement some way to do a color/hue changes (akin to how Photoshop does its color/hue shift to replace, say, a blue with a red -- but in Unity) on the texture itself, that would be enormously helpful (again -- color control is great for artists!) -- though that may be beyond the scope of a shader, but at the same time, it could set this apart from other shaders to have a utility included that could do this sort of thing.

I'm not understanding or familiar with the light warp texture idea -- would you mind elaborating on the technique?

Sometimes just a quick but informative tutorial is enough (covering stuff like tools/techniques to get a desired effect) -- but if you could take a texture and convert it to, say, an R/G/B/A channel explicitly, the data would be pretty easy to make if you have a paint program. There's a tool by @jbooth (the guy behind MegaSplat) that lets you paint vertex data directly on a model. I'll have to look it up, but I think its name is simply "Vertex Painter" or something, and it allows you to paint channel data directly for use inside shaders. Showing users how to do this via video would enable them to make a lot of useful shader data with a free tool.

That being said, I'm glad you're taking into account ease-of-use -- it's stuff like that which can separate a good tool from a great one. However, don't underestimate your users. They'll require some introduction, but once they understand the concepts involved, they'll pick stuff up pretty fast. -- After all, they use Unity -- some level of fudge-factor learning is pretty much always required, so people will love you when you minimize this, but nobody expects you to eliminate all the work for them and their specific needs.



Last but not least, I like the idea that you might be looking to do a more realtime approach to this type of shader. I'm not terribly sure what that entails since toon shaders tend to be a formidable task on their own, but it's an exciting prospect nonetheless to be able to palette swap on the fly. So don't let me deter you from your goal -- it's one I'd really like to see come to fruition! -- If there's anything I can do to help, feel free to let me know!

 

It'd max out at 256 colours per material, the total colours in a palette is limited by the platforms maximum texture size with the offset used for switching banks in steps of 256. So the global palette could be up to 32 banks of 256 colours on a 8192 x 8 texture, or you could cram everything into a 256 x 8, or anything in between - artists choice

You could have more palette textures but I doubt lack of colours would be the reason for using them.

Yeah, I'm not sure yet what form it'll take, maybe an albedo texture that's read instead of base colour, or as I mentioned earlier, an additive detail texture so mid grey is no change, brighter or darker gives +/- variations on the base colour.

I kinda think anime walls would be a good test for this.. Between varying vertex colours (4 x 4 grid?), lighting and a tiled detail texture it kinda works in my mind

It's a ramp texture that affects the light colour by intensity. Team Fortress 2 will come up in searches for it, they used it to get the red tint around the transitions from lit to unlit.

Yeah, I have tested that and will probably use it to generate the first set of test data. I've got ideas for custom UI if needed though. Painting has a bit of a disconnect workflow wise for things like outlines and inlines (which side(s) of the edge do you want them on?), selecting vertices and dragging controls around seems more focused on the work at hand in my mind.

I wouldn't dream of trying to do that - it's just the more intuitive I make it, the less support work I'll have to do

Thanks, I will.

 

I've gone one better and uploaded a new version with support for light warps. It uses the texture format I've been calling additive, it maps unlit to fully lit left to right, with mid grey as no change and darker or lighter tones modifying the colour. Use clamp on its texture import to avoid odd behaviour.

 

Definitely going to check this out -- thanks!

What way are you looking at doing the artist-generated lighting pools atm? Textures or single R/G/B/A channels?



If you're not sure, here's some use-cases you could consider:

Potentially let the artist to define a softer or harder edge to the shadow, perhaps selectively? The mountains in your screenies look great with this automated, but what if the artist wants a bit more control?

For example, I could also see being able to lerp between 2-4 lighting scenarios being very useful for that 2D sprite look. Say, for example, I have 4-8 directions I want my character to generally face, and I want very specific views of that character (i.e. Zelda: A link between worlds or any topdown SNES/PSX-era RPG sprite), I would want to be able to manipulate the lighting normals to work based on the particular direction the character's facing (note how lots of SNES-era RPG sprites are lit -- the artist, in 2D, would swap out lighting pools to match the particular angle they need (for example, I want the back of the character to be lit differently when it's facing me than the front, as only the hair and back are important in that view, so I'd need 2-3 sets of lighting normals to lerp between, based upon which direction the character is facing and/or what action the character is performing) Does this make sense? The 3D "link between worlds" lighting is faked a bit by using hand-painted textures, but imagine how nice something like what I'm suggesting would look on more detailed character models? Can you see the image below as a 3D model?:



If you notice, the hair has a whole different look (i.e. lighting normals) depending on just which direction it faces from the camera. This allows the artist to put dabs of light on the most important features to give the illusion of texture and substance while maintaining a general light source. This also gives a bit of a "sprite" look, and is, arguably, much of what actually gives 2D sprites their charm when compared with 3D!

Just some food for thought.

 

At the moment, it's pencilled in for a single channel GGXrd style as a threshold offset. So basically a +/-1 modification to the light intensity which would affect where the shader responds to intensity change for that vertex.

Using light warp for the toon steps instead of the controls gives you more subtle control over the blending. The light warp scales the light color by 0 to 2 (0 to 255 on texture) for each channel, and higher resolution ramps give finer control. I've attached a ramp texture I made for the first toon shader I wrote for Unity. I used the V offset to change the lookup row on the ramp. So it had a slider to adjust how many lighting bands there were and how soft the edges were. I can add similar atlasing to the warp texture easily enough if needed.

The AO map, vertex shading and the threshold offset all give different ways to control the lighting effect. One of the reasons I was considering a Vector4 for the light controls was so they could easily be swapped to getting their data from a texture if I felt the threshold offset didn't offer enough control.

GGxrd handles this by having an animated per character light direction. For a forward only solution, this is probably what I'd do too. Use the directional light colour, but have a per material override on its direction on the main forward pass. Additional passes would work the same as this shader just using the light data.

The threshold offset + lighting normals + light data + camera direction + light warp + PBR stuff all contribute to the final pixel colour. The first two will come from mesh data so they aren't in a good position for lerping with alternate values.

I'd be looking into morph shapes for the hair for this type of thing. Or just using the model renders as a base for a paint over for the final sprites. To me, the light pooling is something that should be configured and then left alone, any changes should then come from adjusting the lights or camera angle.

As always my ideas may change once I get into implementation or learn something new.

Edit: Now I'm thinking about hair morphs for rotation & velocity so they could be weighted to match the characters actions.

 

New version attached in first post with a fix for light warps. I originally implemented them as a scaling factor on the current light color, but that didn't allow introducing new tints. It's now working correctly as an additive texture. I've also added two light warp textures, just quickly made ones for testing. Here's Unity chan using them.



Testing tinting..

 

Go Unity-chan! That nice 5-o'clock shadow might make a lesser man jealous.

 

Yeah, I need to practice making light warps. I was briefly tempted to make an editor based on animation curves for them, but it seemed a bit clunky compared to just using an image editor to make them.

 

Haha! No problem -- I'm sure with a proper editor it shouldn't be terribly hard.

I've been wondering how others do it myself, but I'll bet they have some sort of grayscale painting thing. It would be really neat to be able to do it realtime. Maybe something like that is possible in Blender to get a preview of what it might look like?

 

Yeah, I'm still hovering between writing tools in Blender or in Unity. Blender has a great gradient editor, but as I found out with my animation curve experiment, it's easier to do light warps in layers. The skin one is a good example, first half is shadow, 2nd half is light, but there's a red bump across the middle. I'd have to do math to figure out the right values for the step in the bump in a gradient editor, but it's simple to just paint an additive layer.

 

Also, the biggest issue with Unity Chan's face shadow is that she needs the artist to paint where there would be pools of light so that the normals can tend toward being lit in those places except in extreme angles -- for example, almost always the entire face would be lit except for just beneath the nose and potentially some places under the hair. You can soften these lighting transitions by softening the edges of the paint. I wonder how much better this would look on a normalmap texture vs vertex painting instead -- I feel like the pixels could be smoothed sufficiently to eliminate ugly blurring if they're large, but an optimal texture size would need to be established otherwise.

Painting lighting is necessary on any toon look in 3D, so getting the normals painted and lit properly could help you test what your lighting is doing better. This will also help a LOT in painting your light ramps because you could tweak them in Photoshop more easily as long as you know where your light pools are exactly meant to be. You could see the falloff much better that way too.

PS:

I suggest doing this on a more low-poly model first. Grab something from the models resource such as this: https://www.models-resource.com/gamecube/ssbm/model/5411/

That seems like a good blend between what you're wanting and what I'm wanting (i.e. a crossover between cartoony and realistic) -- plus it's simple enough to show off your lighting. You might want to paint some normals on some parts of the body first though and try to use them alone to give it a cartoony look.

 

Updated version in first post.

• Alpha works correctly on the outline colour now.
• Outlines fade with distance.
• Cull can be disabled (doubled sided)
• New experimental "Use Toon Data" features

The vertex color can now be used as toon data for the outline normal and a scale. The normal is stored in RGB and the scale in A. Future updates may change this data layout so just use this feature for testing for now.

The toon data can be added automatically - right click a mesh in the inspector and pick "Generate Toon Data". This will only work on a mesh without existing vertex colors and is retained until you exit Unity. To keep the data permanently, right click the mesh inspector again, select "Save As" and give it a new asset name.

You should then be able to selectively scale the outline size by painting into the vertex colour alpha channel with values between 1.0 (full size) and 0.0 (no outline).

 

I've updated again to add a control for the distance fade of outlines. This pretty boring picture is done with no post, 8xMSAA, single directional light. The objects have Toon Data and a fade threshold of 0.111, light range is 0.48 to 0.5

 

I love this. I've always been inspired by the GDC talk about GGXrd's art pipeline, but never got around to doing exactly what you are. I've been following your progress for a couple of weeks now and it's been working quite well.

It took some fiddling, but it works decently with Unity trees too! I needed to create custom Materials that pointed to your toon shader for both the bark and the leaves, and set the latter to Cutout, and play around with the outline until it looked alright.

Slight problem though - I tried updating to your latest release and the outlines don't seem to be working for me anymore - I'm not sure if it's related to the new toon data feature or not, but I can't get that to work in the inspector either. Not sure if I've installed it wrong or if there's a bug - it's a fresh Unity project, using built in spheres and cubes, and I installed it by dropping your folder into the Assets folder. Any ideas?

 

Thanks. This is a bit different than GGXrd's approach but it was influential in my ideas.

I did a quick test with a new scene and I briefly replicated the problem. My first thought was that the new distance fade was reducing the outline to zero, so I adjusted the outline width to check. Sure enough the outline appeared as I increased it.

I put it back to it's original setting and the outline was still visible.. Weird.. Try again with another new scene, outline is visible immediately, try with a new project, outline visible immediately.. I've not been able to replicate the problem again so can't track down what could be causing it. I'll double check the last update but I can't think of anything I did that could cause an intermittent issue like this.

If you can repeat the problem on a test cube scene and it's still there after reloading the project, could you let me have a copy to investigate?

 

Yeah, it's a 100% repro rate for me so far - no matter what I do or what settings I tweak I can't seem to get outlines to appear at all! Attached is test scene.

I've also run into a new usage scenario: if I try to use the effect (current or the prior version I've been using) on a model with intentionally flipped normals, the faces that should be invisible are black, and there is no outline. I can get the faces to properly disappear if I set the effect to Transparent mode instead of Opaque, but that messes with the depth sorting of other objects in the scene and there is still no outline.

 

On the missing outlines, you have the fade threshold set to 0, this scales the outlines to 0 width. Increasing that one setting should make them visible. Would relabeling it as "Draw Distance" make it more obvious to use higher values?

The second scenario I'll have to think about. The shader uses the back facing triangles as the outline, so faces with flipped normals appearing with the outline color is expected behaviour.