Character Kit - High end skin shaders and more for HDRP

I'm working on a package of shaders for character materials using the high definition rendering pipeline. Still a way to go before I release anything but I thought I'd share a couple of images. HDRP seems pretty stable now and allows for more impressive visual effects than previous versions of Unity. I've been able to create skin shaders that support subsurface scattering, transmission, bent normals, plus blending of up to four combined smoothness/normal maps controlled by a four channel mask map. Along with the base smoothness and normal maps this results in a very detailed skin material, even though the individual texture maps are quite low resolution. These material options allow for detailed art direction.

PLEASE SEE THE LATER POSTS FOR UP TO DATE NEWS. (You also get to laugh at me contradicting myself as I work through options and generally go in circles.)

Next on the list is wrinkles based on realtime tension maps. After that I'll look at eye shaders, then move onto textile shaders for character clothing. My plan is to build a package that handles most, if not all of your character material needs. (The main skin shader and bundled detail maps alone should be very useful.)

I'll consider the package to be in beta for as long as HDRP is. Once HDRP is finalised I'll create the first official release of Character Kit. However, I plan to keep everything up to date with the latest version of Unity and HDRP, so you can use it day to day. (I also hope to provide shaders based on the older Standard Shader. Obviously these won't support everything the HDRP versions can, but I'll do my best to make them look good.)

Please let me know if you think this would be worth purchasing. Obviously there's no point in me working on it if no one plans to buy it. Initial beta releases would be cheap, with the price increasing as features are added, so early adopters will get the best deal.

Blending detail smoothness/normal maps with the mask:

 

Hi Paul, that does indeed sound quite interesting.

As for wrinkles: I've only done some superficial reading/thinking on real-time tension maps but it seems like a rather involved and mostly performance intensive process - so if you do find a performant solution that just works™, that'd be pretty sweet. Not just for skin, but also clothes. Other than that, having a weight-based wrinkle system (probably with a map) would already be pretty nice as well.

It seems like unity is looking into creating its own set of character shaders, though, so you might want to look into features they probably won't cover. I'd assume wrinkles being one of them. I think I recall them working on skin, cloth and eye shading.

 

I haven’t decided on how to tackle wrinkles, but performance will be very important. It will be a compromise between that and visual appearance. I also want it to be easy to use.

Do you have any links for the character shaders Unity is planning? Definitely worth seeing what they have planned.

I’ll add more updates here as I work on the shaders. Comments and questions are always welcome.

 

Had to google a bit - I recall this from reading various posts all over the forum over the course of the last year or so.

Here's someone from Unity saying they work on a hair shader, here's a bit of (placeholder) documentation for a fabric shader. I could imagine they're calling it a day for the skin shader by providing a SSS profile tweaked for skin. I don't think they're looking at wrinkle maps, though - even though I'm sure some people (myself included) are very interested in that. Personally, I'd just ask the HDRP team which features they aren't/won't be working on (at least in the foreseeable future) and focus on that. They're quite active on the SRP forum.

 

Thanks. I noticed the mention of fabric shaders. Will be interesting to see what they come up with for hair. I think wrinkle maps are worth investigating. It's been mentioned on the forums in the last couple of years but nothing ever came of it.

I'll add some updates as soon as I can.

 

I haven't forgotten about this. Just busy on other work at the moment.

 

Sure thing. People here are used that a couple of weeks or even months can pass before seeing some more progress on projects.

 

Hi,
very interesting project and I used a similar skin normal detail controlling approach in a paper.
What is Detail Smootness Intensity going to accomplish? As there are no Smoothness Detail Maps?

Best wishes

 

Sorry about not responding. I've been busy with other work and didn't get updates about new messages. The shader allows up to four detail smoothness/normal textures to be blended with the main textures. The intensity controls allow you to control how strong the smoothness/normal effect is.
I'm going to see what Unity releases with The Heretic assets. I suspect they'll have a lot of skin shader stuff I can borrow.

 

Do you have anything we can subscribe to for Alpha access?

 

Nothing I'd feel happy with people using at the moment. Let's see what is released with the Heretic assets. I suspect the shaders in this will be very useful.

 

I looked, and was not able to find anywhere of them saying they are releasing The Heretic shaders, assets? or anything. Do they say that somewhere, I waited for the MegaCity the same way, but can't find where they say it about Heretic.

 

On the main Heretic page they said "the Demo team is preparing a package with the tech and shaders used for the digital human in the film so you will be able to use them in your own projects." Hopefully won't be too long.
https://unity3d.com/the-heretic

 

Still waiting for shaders from The Heretic. In the meantime I'm looking at raymarching for eye occlusion. Unfortunately this all has to fit around real work.

 

Just what I need..looking forward to its release. Sign m up for any alpha testing

 

It's all on hold, but I hope to revisit it when the Heretic shaders are released and I have more time.

 

I'll keep an eye out too. I can throw you some bones for early access, I like to get in early. Sounds like you are doing it right, if you build and sell tools around what you yourself use, I always find those the best is what I mean.

 

The Unity 2019.2 release has some shader stuff I can use. Will try to get an alpha version together for people to try out.

 

I tidied up the shader code and combined texture channels for efficiency. The main skin shader is working quite well now. Would anyone be interested in an early access asset? My plan is to add features and increase pricing until there's enough for a version 1 release. All going well the asset will cover:

Skin shaders for HDRP
Eye shaders for HDRP
Hair shaders for HDRP
Wrinkle maps and scripts for realtime blending

At the moment I have a skin shader for HDRP that uses a standard albedo, smoothness, AO and normal setup, but also allows you to blend 4 detail normal/smoothness textures on top of this using a mask. E.g. specific normal/smoothness for forehead, another for cheeks, another for nose, another for neck. These detail textures don't need to be high resolution because they're tiled. I'm currently using 512px detail textures and getting good results. The shader also uses SSS which is controlled by a mask in the albedo alpha, plus a thickness map from the combined smoothness/AO/thickness texture.

The asset will include pre-set detail normal/smoothness textures for forehead, nose, cheek etc, so you don't have to worry about creating these.

 

I'm looking at two techniques for implementing wrinkle maps. The first defines facial regions with a texture map, while the second uses the movement of vertices. Each is pretty complex and would require custom tools to help artists set things up. So, as well as implementing these techniques in the shader I need to look at what tools will be required for the user. E.g. a paint tool to set up facial regions, or a tool to record the initial 'relaxed' face vertex positions. As well as runtime performance, ease-of-use is key. I want the asset to be artist-friendly.

 

Thanks. I'd like to get an early release version of the asset out there, so check this forum for updates.

 

Looks pretty awesome, I'd like to test it also.

 

I’ll try to get an alpha release together for next week.

 

Yes, I never did figure out how to make the wrinkle maps for blacksmith demo for my own characters. So I'd say either find out how people are doing that...as that is already a workflow used in shops?? or yes your own tool/s if you don't mind the work. Timing is everything, as often the first in wins. So I recommend focusing on alpha / beta skin with a discount for asset store early adopters. With wrinkle maps coming. I don't recommend doing it all, or someone can beat you to it.

 

I'm reading through academic papers on the subject and circling round possible techniques. Papers by Oat and Jimenez describe techniques based on compositing multiple wrinkle maps using masks and data from triangle deformation. It's very clever stuff but only works with morph target / blend shape animation. I need to come up with something that works on standard skinned meshes. Luckily some clever people have been exploring this topic. I just need to make sense of their results and translate it into something us mere mortals can understand.

 

The little I read yielded similar results. Usually Alembic or morph target animation. I had always wondered what capturing a Alembic mesh animation would look like within the editor. So virtual camera's recording such animation so it is then easier to play from Timeline and such. But it all seemed not the right way, and limited...I think the future is in realtime dynamic characters not static 3d vr movies - i.e. adam tech demo (a great first step, but I think the future is dynamic characters.)

Certainly fun hearing about your research.

 

I think I have a technique for the wrinkles but it's a pretty big topic to dive into, so I'll sort out the eye shaders first. That way I can release an alpha asset with useable skin and eye shaders for HDRP. This will give early adopters something genuinely useful, with the promise of further features.

 

Have you decided on a price so I can begin saving

 

Not much for the early release version, around $20. Enough to help with development, but not so much I'm putting people off. The price will go up as features are added.

 

Working on the main single layer eye shader (for an eye model where there isn't an extra transparent layer for the lens). Some of this involves translating work I've done pre-HDRP. However, there are still some things missing from HDRP Shader Graph, such as accessing light attenuation, so I'll have to fake these for now. Luckily new nodes get added to Shader Graph with each release.

Here you can see the fake refraction, lens bulge effect, iris AO and highlighting, and the limbal ring darkening:



Still to come:

Eye edge blending / shadowing
Pupil control
Sclera reddening control
And more...

 

Got the pupil control working. Now to make the iris distort correctly as the pupil is resized. The single layer shader doesn't use vertex distortion as this would affect the lens shape.

 

great looking eye, will there be a sample scene to control pupil from code? I see it is shader control based, its not always obvious how … so just suggesting.

also curious how "fast" you plan to make everything. as in, quality vs render time. where do you favor?

 

There will be demo scenes and documentation. I’m trying to make everything as efficient as possible, but aim to have a range of shaders from fast/simple to complex/max quality.

 

Working on the eye edge blending. I'll look at the tear line next.

 

The tear line helps blend the eye into the main head geometry:

 

I also highly recommend faked ambient occlusion at the upper eye lid - this often makes a huge difference (and is IMO much more important than tear lines.)

 

I plan to do this too.

 

I'm working on the fake AO for the eye edge and have re-done the edge colour blending too. Previously I had a scene depth based technique which used the intersection between the eye and main geometry to create a blending mask. The main advantage of this was that it worked when the eye was rotated so the edge blend was always in the correct place, but there were drawbacks:

-Scene depth is really weird and unreliable in HDRP. (No grab-pass either so that doesn't work.)
-The eye material has to be made transparent so depth can be sampled, which can lead to all sorts of rendering issues.
-The depth is sampled against the main face geometry or, the better option, an eye socket mask so there are times when you get noticeable faceting. Also, the edge blend effect might not be visible at angles where there isn't actually geometry directly behind the eye to be sampled. We can also have rendering issues where the eye seems to take on transparency based on the calculated edge mask.

Instead I'm letting the shader generate an ellipse mask I can use to apply fake AO and colour blending to the edge of the eye. It's much simpler and always renders a smooth blurred edge. However, because it's simply part of the rendered material it will rotate with the eye, meaning the edge blend won't match up with the eyelids. This isn't as bad as it sounds because the shader parameters used to size and position this mask can be exposed to scripts. When animating a character we don't manipulate the bones/geometry directly. Instead we use a control rig or some other proxy. So, when I use the control to make my character look up something like the following happens:

Move the eye target up.
The eye controller script rotates the eye bones to point at the target.
The appropriate eye squint, eyelid up etc. animations are played.
The eye shader is given updated parameters for edge mask size and position so it matches the new pose.

This new edge blend mask will also be easier to integrate with other effects such as the fake iris caustics. I could create a shader variant that uses a texture for the edge blend mask too. (We could allow the shader to detect if there's a texture and fallback to the procedural ellipse if not, but that would add complexity and isn't very efficient.)

Another thing not currently available in HDRP is light attenuation / light direction, so I'm faking what I can. I'm using subgraphs for useful functions with a view to dropping newer ones into place as HDRP gains Shader Graph nodes. (For now I think I'll create an unexposed vector3 property in the shader for light direction and pass a value in via script. If it's an unexposed property it should be shared by all materials using the shader.)

An alternative approach for eyes is to have a close-fitting piece of geometry in the socket, just above the eye surface. This can have fake AO and edge blending applied to it. However, there can be problems when the eye is rotated because the bulging lens may poke through the mask geometry. A single piece of geometry for the eye is simpler.

 

I think I'll keep the different edge blending techniques and have three variants of the eye shader:

1 - Edge blending mask using scene depth.
2 - Procedural ellipse blending mask.
3 - Texture based blending mask.

Each has advantages and disadvantages so the user can choose what's best.

 

And that's why you keep quiet until the work is done. The eye shader is very complex and I'm not sure what's the best technique yet. At the moment I suspect I'll end up with a shader for the eyeball that has fake refraction and normal bulge for the lens. Blending with the eyelids will use a mask texture that is mapped onto the eyeball by the shader. The control rig will have to update the shader when the eyes move or face animates so that the blending mask is remapped to the correct place on the eyeball. E.g. the eye looks right so the edge blend mask has to be offset left to look correct. Alternatively, if I can get z-test to behave I may be able to avoid all this.

If HDRP shader graph ever gets grab-pass / scene colour I'll use that for edge blending instead. However, it doesn't look like this will ever be added as an HDRP shader graph node or exposed for custom code.

That said, I could be completely wrong. It would be good to see what shaders The Heretic demo was using.

 

I’m getting the impression The Heretic Demo used so much custom code it was no longer a standard version of Unity. This is no good for us. The Sherman demo has eye shaders and scripts, but I get errors when trying the project and there's no documentation.

I’m going to investigate raymarching techniques. Signed Distance Fields may be a viable technique for eye shading.

 

A super interesting development! We are currently working on an HDRP project trying to achieve the greatest possible realism in a human model. Very interested in trying your shaders!

 

I’m away for a few days but will be working on this when I get back.

 

I would be interested to test the skin and eye shader from a realistic and stylised perspective. We did try the sherman eye shader which was awesome but only worked in 2018.4.1f1 . Its not easy to port to a new project (we couldn't work it out)

 

I had no luck with the Sherman project either
I hope to have more time to work on the shaders next week and would like to publish an early release asset soon. I may pause the eye shader work until the skin shader has more features.

 

Conversation was accidentally moved to another thread. Will try to bring it back here.

 

Any updates on the eye shader?

 

Unity 2019.3 has new HDRP (v7.01) nodes which fill some gaps. I'm looking into incorporating these now. EDIT: Actually it doesn't help at all. The Heretic eye shader is still a mysterious custom solution the Unity demo team came up with.

 

Making progress on controllable eyelid shadowing for the eye. This is driven by marker objects / control rig. You'll be able to animate these or control them via scripts. E.g. when the character blinks the eye shadow markers are adjusted to suit. The shading is done in world space so works correctly when the eyeballs rotate within their sockets.

This example shows a single control point for the upper eyelid shadowing. The shadow intensity and softness can be adjusted in the shader. The white line is a debug ray from the eye to the shadow control point. This is the single layer, modular shader on a completely spherical mesh.

 

Eye edge shading using upper and lower control points: