Comic Book Shader

As some might have noticed previously, I previously have worked on a shader that was based on Ark System Work's shader which was made for Guilty Gears Xrd and the new Dragon Ball FighterZ
You can view it here if you wonder about it.

While I was working on that shader, I was focused mostly on having something I can work with that gives out the result similar to Guilty Gears Xrd. I'm actually using the shader I made for one of my current mobile project as it's super light in computation (especially in comparison to a PBR shader) and it's requires relatively low resolution textures on top of that which saves a lot on the storage size of the application in the end.

Still, I was wondering how I could push the shader ever further. Making something new or original out of it, even if it's just something small. And as I was reading some good old Marvel comic book, it struck me! Why not making a shader that looks similar to a comic book. Not just the TV anime/cartoon flat color, but an actual shader that kinda simulate the halftone look of a comic book!

It's still a work in progress, but I have made it... Took me about 12 hours straight to find the right way to implement the change in the lightweight shader I previously made, but I made it!

For some, this might sound like something easy or simple to do, but I would like to tell you that I started on learning and working on shaders in the last week of last October. This gives me about 3 months of experience in scripting shaders. I only did some minor changes to existing shaders prior to that date and nothing like writing one from scratch by myself.

There's a preview of the shader:


As you can see, the shader allow 2 types of halftones.
One is a pixel look. You could call it "Manga Halftone style".


The other has lines which is similar to old american comic style :


It's far from perfect, but I think it's a great first step.
The halftone size is controllable and it screen-pixel based. You can make each "square" the size you want based on the screen pixel size. The example able uses a 1.0x size ratio and since there's not Anti-Aliasing, it might seems a bit hard edged a bit, but if you can just put it at less than a pixel and it will look less edgy.




With it, I'll be able to convey the comic-book style I wanted with one of my projects.
Here's another screenshot with different kinds of possible result (no change done in the shader's code.
There are 2 parameters that does all the "job".


I'm still not 100% done with it. I got a few more trick I want to try to add into it.
For example, in the next few days, I'll try to add some kind of way to controls the outline width so that it's possible to give a kind of relief through the outer line like you see in comic books. (It might be done today actually as I got an idea of how to implement it).

 

Quick Update
(I'm posting it as a new post since it's a new feature)

I have succeeded at adding an option that affect the outer shell width dynamically.
In the previous version of the shader, I was using the vertex color Blue and Green channel to allow a custom manipulation of the highlight and shadow thresholds.

The red channel is still being used for force-modifying the shadow actual value so that i can add or remove some shadow strength based on the artistic needs. (Think of it as a fake bumb map where pushing the vertex color's red channel to max remove all "black shadows" and vise versa.)

The Alpha channel is being used for customization purpose so that I can change colors of asset on the fly through scripts without changing any textures.

I simplified the threshold value modifier so that it only uses the Blue channel and, now, the green channel modify the outer shell's displacement, making the black line around the model thinner or bolder.


This will push the 2D style artistic capacity of the shader by a LOT as comic books (including manga) arts usually follow some specific kind of generic rules where specific part of the body has bolder lines and other has thinner.
For example, on a character, each part that can "bend" like the inner part of the arm (opposite side to the elbow) has thinner line while muscles and other part that aren't really bending visually are bolder.

Ironically as we all know, 3D models then to have vertex in area that makes this quite possible implement by vertex color.
Not only that, but it will also allow me to cheat my way whenever the outer shell doesn't look right by dropping its width to 0 at those specific parts!

Now, I can't wait to try it within an actual full fledged scene!

 

It took me a bit of time to get the hang of the workflow and to get from A to B in terms of modeling and setting up the models and then importing them into Unity.

Especially since there are a few "rules" in terms of modeling the meshes and level designs that are required.

But, here's a preview of the scene (main menu) I'm currently working on.



Due to the trick with the UVs, the current scene barely have anything in terms of textures. Except for the logo (which is a 1k texture), there are only 3 material with, each, 2 textures for the current content.

The Sword has 2 materials (The handle/hilt and the blade) as the project involve weapon customization outside of the battles and model switches during the battle so I have to cut the weapons into pieces (depending on its customization capacity) which might varies in material (and so color). The blade has 2x textures at 64x64. The hilt has 2 textures at 128x128.

The ground and rocks has 1 shared material with 2x texture at 32x32.

The only issue with the shader with an outer shell is the fact that it can't be batched because it involve 2 draw passes. For this reason, repetitive element in the background are, instead, using the same shader, but without an outline. As you can see, as I included some of the details into the geometry itself of the meshes, removing the outline can be possible with a bit of preparation (saving about 66% of the drawcalls with the batching).

I'll post an update with the complete menu once it's complete.
(I'm planning on having an animated sky with halftone clouds and stuff like that. Also an area in the background that includes a castle and a battlefield.)

= EDIT =

I added a kind of float selection to the parameters of the shader.



This allow to change the patterns of the mid-tones (halftones).

 

A comic book type shader, that tries to emulate the Guilty Gear Xrd look?
Yep. That caught my interest. Yippie!!

It's looking nice so far.

 

This looks awesome. Are you planning to post it to the Asset Store? I'd really like to get my hands on it.

 

A github of this would be great, looking at recreating this in shadergraph!

 

I'm trying to make my game in this style... I would pay money for this shader looks awesome.

 

Ouff! I haven't been active on neither the forum nor this project for a while.
Sorry for the lack of updates guys.

Due to some requirement I couldn't met with the shader I made in the ways I wanted, I went with a different approach.

The reason why I stopped working on this is due to 2 "issues" I ended up facing :

Issue A : A major change in how the faces are calculated have been implemented since Unity 2018.3x
Whenever there's new stuff that is awesome, there's always the backside of the medal and the backside of the medal, this time, is so hard to manage that even 1 year later nobody have found a viable handout "solution" without some major side-made fixes.

To put it in a simple way, with Unity 2018.3x came the initialization of a new way of handling the faces's in the rendering pipeline of the engine. This was in preparation with the Progressive Lightmapper implementation. As everyone (who follows Unity engine's updates) knows, that is one really great system that remove a tons of stress and waiting when working in any scenes since the lighting bakes itself pieces by pieces while you can keep working in the scene while the lighting is baking itself. While the Enlighten lighting system has great results, you usually have to turn off the Auto-Bake when building a scene since it keep redoing the baking of the whole scene each time something moves, is removed or added. With complex scenes, it's can take a long time during which the CPU is clogged and the PC run to a crawl.

Anyway, one thing that changed with the Progressive Lightmapper is how the Unity Engine handle the models' MATRIX.
The MATRIX is, to put it bluntly, a part of the system around which the model's normal are calculated. It's one of the key part of the Toon shaders with outline since they make uses of the UNITY_MATRIX_IT_MV and UNITY_MATRIX_P to displace the surfaces based on the vertex position and it's related faces' orientation.

It's all technical, but the results is astoundingly simple to see.



As you can see, some corners get's detached. This might sound simple to fix, but it's not. Originally, the way the MATRIX was handling the group of faces was based on the model's smoothed groups (also known as Hard Edges for 3D artists) and faces angles weren't taken into consideration.

So, previously, all the "groups" were handled as a whole and the "shell" effect was pushing the outline outward in a perfect and uniform way. With the change done to the way the MATRIX is stored in the engine, the "new" groups are now handled based on normal direction. This means that the previously Hard Edges are not used any more, but instead the engine decide itself what is "detached" or not based on the angle between each faces when rendered.

From my tests, it's around 60-75 degrees.
This is why the capsule above (which has like 15 degree between each faces) doesn't seem to have "cut corners" while the cylinder and cubes have their corners "cut". (The 2 cubes have different corner because I tried the 2 common methods of handling the MATRIX data when generating the outline. One had better results, but still was not a winner.)

The only viable solution to this problem has been to implement workaround within the shader that use another set of data that explicitly affect the normal post-MATRIX-calculated so that they are at the right size and right position.
To give an example, the famous JMO's Toony Color Pro asset had to implement a system in its asset (called Smoothed Normals Utility) that generate the relevant data and store it in the model's Vertex Color, Tangents or UV2 and even that doesn't fix everything since some view angle still break. (And you got to sacrifice that type of data for the fix. If you use Vertex Color for customization or, like I did, to affect things like the outline and highlights/Shadows, you got to use Tangent or UV2 and, sometimes, the effect isn't better since each store data differently.)

That's the first issue I faced and I did went into one hell of a stress & depression phase when I discovered it as I upgraded to Unity 2018.3x.

Issue B : It was incompatible with many part of the Engine and many useful tools.

The only kind of effects and tools that are compatible with it was things that do their effect post-rendering. This issue, which I was working on until I faced the Issue A previously explained, is highly apparent when I made an actual scene. The trick when you use a shader that is relatively unique is that you want to keep that style all around.

To give a good example, The Legend of Zelda: Wind Waker did it really great :

While you can see that there's a slight difference in details (like in the trees) between the character and the static environment, you don't really see a major change and it makes you feel like the character truly belong in that world.

If you want something with more details, Killer is Dead is one hell of a great example :



You can see that both the characters and the environment had a similar visual style. The only part of the environment that is the highlight around edges which, I guess, was kept to allow the player to better notice the characters in the environment. (Kinda like an outline)

To give a example of a bad implementation :


I could point toward every Sword Art Online games out there, but the one who truly stroke me was Fatal Bullet as it uses colorful characters with strong highlights and shadows (like the anime), but has an environment with 2x to 4x more "real-like" details and clearly not the same kind of highlight and shadows. It makes the character looks like they were Photoshopped into the scene.

So, all this to explain that I had quite some issue to manage the visual between the close/small assets and characters with the environment since I wanted to keep the look relatively constant between both.

For example, I couldn't bake the lighting because Unity's lighting systems (both Processive and Enlighten) doesn't take into consideration the shader's way of handling the resulting shadows when it involved screen space or anything like that. This means that I couldn't make use of static lighting at all. The only way of going around this would be to use a image post-rendering effect that store, during each frame, the lighting result (basically, a grayscale image of the screen view with the shadows and highlights results) and replace the rendering of the lighting on the screen (in deferred) with the one cooked up in the effect. So, in other word, a completely different approach and all my work in the shader is useless in this matter.

While I was able to implement the GPU Instancing within the shader, the optimization it gave me was sub-part.

If I was to use this shader in the current common ways when it comes to how most make their scene in Unity, I was ending up with a tons of drawcalls (way too many) for barely anything to show for it. For all the VRAM I was saving with small simple textures, I was using a lot of RAM in drawcalls and the rendering pipeline was truly becoming a mess with noticeable hold-up during specific parts of the rendering stages that was easily traceable within the Profiler.

Trying it out on an old Laptop I keep for the sake of testing gave me some good data about how "far" I could go with the shader. The shader could work well with game with a small line of sight such as an isometric Action RPG (Diablo style) or even tactical games. Even on mobile, it works relatively well for something like city simulator with a grid of 20x20 "slots" filled with city parts. The shader wasn't a total lost as long as I kept the rendering relatively short sighted.

Still, my 2 projects on which I work on that required this shader are made with, at some part, long range panoramic views of a detailed and animated backgrounds.

The only way of making things works properly was to make an heavy use of the LOD system so that the models keep changing based on the distance with the camera so that I can save as many lighted triangle as I could. Some asset had to have at least 6 LOD and, each, with different models that had to be modeled and UV unwrapped manually. It's literally like making the same 3D model 3-6 times from, almost, scratches since the "fake" lines modeled had to be modified each time. A lot more of time required to make the model and textures together than if I was to make use of a more conventional PBS shader that works with Unity lighting system.

So, what I did was simply revert back to a more "flat" anime-like look and, to save a lot of time and avoiding the Issue B explained above, I used one of the shaders that come with Toony Color Pro as a base and modified quite a bit of it to fill my own needs. Since it also come with a somewhat-fix for the Issue A, it's a win-(relatively)win for me to have switched gears.

 

Wait you can't bake the lighting THEN process offline that baking to get toon like features?

on issue A, since japan is very active, how does the unitychan shader fare with the update? I have seen they released a recent talks on unity japan for 2018.3

edit:

http://unity-chan.com/download/releaseNote.php?id=UTS2_0
Translation google chrome (english version don't have that snippet of info)

https://github.com/unity3d-jp/UnityChanToonShaderVer2_Project

 

I actually kinda like the broken edge look. Any word on getting your shader up and running again? This is probably one of the best toon/comic shader I have ever seen and would hate for it to disappear.