Why can't Unity HDRP correctly render AAA looking character models?

Yes, thank you... This helped a lot and really opened up the path forward. Still tweaking, but 1000% improvement from wooden & plastic to something far more detailed and in the range of a Unreal MetaHuman.


Imported FBX of model (right) new workflow process (left).


I'm shocked that Unity hasn't done anything for character model imports to significantly improve this workflow process. I can't help but put the blame on Unity for the lack of a decent extensible importable FBX process for Digital Humans. The fact that unity points to asset store for everything and then says anything is possible! Yet, provides very little in the way of common things that game developers would like to do such as add a character that doesn't look like it was carved from melted legos.

 

Really glad to hear that.

I guess, I don't have knowledge about FBX process for Digital Humans. I will be happy to learn more about it, if you can shed some light on the questions like how it must be done, what it is lacking or how this effects other FBX imports rather than digital humans?

 

We are comparing different things here. We are comparing out-of-the-box basic setups (Unity Side) to highly customized and carefully authored assets (Unreal Side). Just throwing in something Unity themselves created to show the quality you usually see from Unreal.

This is a demo anyone can download and take a look at in the editor, a lot of stuff is packed into it. Also, keep in mind a lot of the custom work here was integrated into HDRP with shader graph same as Epic did when they first released their Believable Humans demo (They integrated things from that to their material editor). So at this point, only Unreal provide out-of-the-box characters fully setup to work with their system with proper materials, textures, etc (Metahumans). So for both, you can either create your custom setup from the provided nodes or just copy what they did already and input your own maps.

If you don't know what you are doing out the gate for Unity, you have to read Docs, watch talks they did on The Heretic, and reverse engineer the Digital Human sample project to get that quality. It's possible 100% to achieve that quality yourself as others have done so. You just have to author things properly.

Like
@OguOguzkagansahin in the post above
and @HIBIKI_entertainment with

You can copy the shader/material and input the maps for your character. At least the few basic maps you need to get the skin to look good. The extra maps you can do without sometimes if you don't have any.

To be honest, most of the work is done in texturing for the character, a simple lit shader with subsurface scattering will get you a good-looking skin. To get it to look the best possible then that's up to how much knowledge the artist has on creating realistic characters and on the technical side of things. You have to do some work to get the eyes and such to look nice. To get a believable character that's not still (speaking and such) requires a lot of work with both art and tech art (model, rigging, animation, shaders, scripts to help things out, etc)

 

Adding that the shader is compatible with any model you add it to it's not anything built to specifically work with just one model in one project and one version of HDRP. In those screenshots I took I'm using Unity 2022.1 beta 2 with HDRP 13.1.3. As I mentioned before all the new nodes we have for Skin, Hair, and eyes came from this demo (well derived from the original custom shaders made for this demo) Now they've released the demo with those reworked nodes and in the shaders provided and not the original custom shaders. Although these still retain some custom subgraphs/custom functions to run the 4D stuff. Nothing you need to worry about if you aren't using that tech yourself. You can still copy the shader and just remove those parts or just use it as is, it's not going to throw errors asking for that data since it's just a shader.

Just keep in mind it's using the Stack Lit Shading model which isn't suited for games but still works so you can convert that to the base lit shading model. The only thing you will lose is the second specular highlights, anisotropy for area lights, and the smoothness lobe (these are Stack Lit features only). As a matter of fact, I just went in and duplicated the shader and switch it to Lit just to show.


Stack Lit vs Lit - Left - Default | Right - Switched to Lit
Barely any difference but you can see minor differences at the side of their jaws. It's a little lighter with the Lit Shader. You might have not have seen that if I didn't mention it. I wouldn't really have noticed but since I sat here and switch materials back and forth to see I can tell.





Shaders Master Stack Nodes



HDRP Default Lit Shader
Here's with just a default Lit shader, it still looks good. I just added the maps that the default Lit shader supports and match their settings to the original material, so It's missing the Cavity, detail normal. For the detail texture on the default lit shader, you have to pack your texture in a specific way to have normal and smooth detail in each channel. So instead I just used the tiling detail for the smoothness and left out the detail normal. I was too lazy so I didn't do that. The textures for this were authored for a specific workflow so they weren't packed to match the default Lit shader. It's all about how your character was made more than where it's rendered as long as it supports decent subsurface scattering.

Proper skin textures and subsurface scattering will get you 95% the rest is all technical and lighting.






Fia Render
Here's also a render for Fia I did. It looks a little off because it's missing a few things/tweaks, for the scalp and the eyes.
Tear line mesh with AO and blur could be better, a scalp cap to blend the head with the hair/fake some AO. The model might have been updated with the second release of Look Dev Studio - The screenshot below is using the first version since this was a while back.

 

I think it would be a good idea from unity devs to make a project fully packed with everything on and higher quality of all settings and make it a presset on tthe hub, because, sometimes i want to make a render with everything at max but i do not know where is all the graphic settings, because sometimes there are setting in one menu and other settings in other places, or sometimes you need to activate the same setting in 2 different menus to work (very useless in my humble and noob opinon) right now i know Unity can achieve at least 90% of the unreal's quality.

BTW: the second image in your post looks amazing, is that unity or unreal?

 

I mean it's still up to us to setup whatever we want to render how we want to render it, Unity can't do too much on that end. Unreal doesn't provide anything like this and everything is fine. But they do provide solid documentation, docs that you can follow and do basically the same in Unity as long as you have the base knowledge to understand what's being talked about. It would be nice for Unity to provide better documentation on character rendering and such. similar to Unreal. Other than docs all both provide outside of that is their Digital Human Demo projects.


Every image posted is rendered in Unity

Here's some shot with another character same lighting
Credits 3D Scan Store - https://www.3dscanstore.com/blog/Free-3D-Head-Model

 

You can always use lookdev or the newer lookdev studio for this. It's generally the stage you want to do something like that anyway.

This process is called staging
It's a rendering and lighting stage where you test the models and materials in your projects lighting scenarios.

In larger productions this is critical to display characters in lighting the best way for their colours details and hit the moods of the design docs.

You could say it's 'not as critical' for smaller entities, however, whenever you browse artstation or Instagram and see octane, stager, unity, marmoset, Clarisse etcetc render, that's exactly what the artists are doing.

Showcasing their models and work in the best rendering and lighting they can muster with skill set and renderer of choice, wether offline or realtime.

So it's quite a critical stage and artists in general, it's usually and end point for them and where they will iterate over and over and over again, feedbacking and collaborating with teams and the public.

So that makes it a fantastic place to learn as well.

The newer lookdev studio aids this a little more by showing the relevant engine tools and resources first, it's still not perfect, however it's a good place to start to learn about the stage rendering side of things, where you can really just push your system a little more to the max and work with the render pipeline assets more.

As Grim said, it's up to us to set up. That's mostly because every project has some many different needs, hardware too, so there isn't always the same available " low medium high" look settings.
In look Dev though, you probably could create presets for that as you learn more about it.

 

@Grimreaper358 appreciated

 

Yup, your examples look good, but I'm still of the mindset that Unity has essentially made the overall process of using a useable cinematic AAA character in a unity game so complex, that only a small percent of the gamedevs out there can actually bring a model into Unity and have it look and work (speech, emotion, targeted IK, etc) like it should in a modern game.

For years, we've been putting characters into unity games that look like melted legos carved to resemble people, this shouldn't be the case. Unity made this more difficult than it should be.

 

https://twitter.com/unity/status/1485629019827712000?s=20

 

this looks really interesting! Less on the rendering side, more on the simulation aspects. In this regard, this is way ahead of any other "meta-human" tech I have seen. But seems to be a Maya plugin so far, wonder if it will be possible to bring it to Unity, I'd guess a proper integration might not be so easy, as it needs to integrate into the Physics system somehow.

 

I noticed that the result in URP is quite decent too. Not state of the art, but still decent.
This example uses ONLY the standard shader. It doesn't have SSS, IoR, layers or anything. The mesh is around 20K~, and the textures are 2K or below. I did use the pore tip shared above but the guys that really know about this subject.

 

Really looking forward to see if there's going to be Ziva models and integration in the future. It would be great if Unity could focus on character pipelines for a bit. They just bought Ziva but used Digital Humans 2.0 in the Enemies tech demo. Sadly, Unity has nothing for character design at all other than shaders for Hair, Eyes, and Skin. There's another thread in the forums that Unity is looking at upgrading Animator to "something else" that can allow realtime hair without relying on the physics engine.

 

https://portal.productboard.com/3iz...ion?utm_medium=social&utm_source=portal_share
Here is the roadmap...(maybe i am wrong but feels like this will be directly integrated inside the engine for free because I found this under their main roadmap, but even though it's free won't it require you to buy the 4D facial capture equipment????

 

Nice find! I'm sort of hoping that Unity (or they partner up with someone) to provide a bunch of precomputed base models. Stick a pre-generated head (from a wide selection) that has some customization options, then place that on a rigged body and you're good to go. The alternative wouldn't be ideal, since a 4d rig is about 50k, but some smaller ones like faceware can use a gopro camera for $5k.

I feel like there's an interesting intersection here of the technology where you can have AI generate faces of people that don't exist for this kind of thing, versus doing an expensive and tedious process to use someone to generate a full 4D digital human / ziva character. It's going to be interesting to see where Unity takes this technology, but the Ziva tech is really impressive.

 

Having been using ZRT face trainer for the last few months, I can see why the acquisition was made relatively fast.

I certainly recommend dropping the feedback on the product board, Chinny is particular good at framing some great questions for these ideas to aid in giving good feedback for the team

 

This looks like you havent even tried to match the scenes
Where's the ambient lighting on the right side
Also these materials are noto setup the same

If you try to do something as complex as a realistic character you gotta put a little bit of effort into it

 

The partial solution was to use the tips posted by Oguzkagansahin above. The same PBR materials were used for both sets, the underlying issue is that highly adjusted normals are required for Unity. Unity also released a HDRP Skin and Hair shader that was greatly improved in 2020.3, after the initial posting.

All that being said, up until that time, and still today, Unity has no tutorials or stock AAA quality character models for use in Unity. What they do have are "tech demos" showing off digital human packages with some blog information and a couple GDC videos. While some of that information is okay, it's not a good solution for an indie game developer to put a AAA style character into a Unity game. Prior to my post, I had put some extensive time into a character pipeline workflow, which was frankly, often ruined by HDRP the standard Lit shader.

Where Unity should improve:

• Unity has no character pipeline workflow tutorials with integrated editor tools. Lookdev Studio doesn't count.
• The editor tools and packages lack things like motion capture w/ speech synchronization.
• Pre-defined open extensible character models with generative attributes (i.e. unreal metahumans)
• Digital Human frrameworks v1 & v2 are also too complex to try to use natively without a large team.

What does work:

• Using a complex workflow from 3rd party toolsets that is "figure it out as you go".
  
• Cinemachine w/ Timeline & Animator with layers & avatar masking.
• Newer Eye, Hair, and Skin shaders in HDRP 10.x that add some realism.

Where I think unity is heading, is a weird internal showdown between using Digital Humans or Ziva. What's frustrating is that Unity seems to be going for film production teams with Weta and perhaps Ziva without a lot of information on what it's doing for character pipelines for the game side of Unity. I'd love to see Ziva adopted with multiple generative options into the editor as a package. Probably won't see it, so until then, it's an extensive labor intensive character pipeline from design tools into Unity HDRP.

 

Unity's said they're going to incorporate Ziva at some point. It's not a weird showdown, just the fact this stuff takes time / work to do.

 

Hey, guys. Decided to toss my hat into this conversation and say that, Unity HDRP is very capable of a high level of quality when it comes to digital characters. The screen shots below are an example of how much you can push the details if the lighting is done with care. .. These were done in 2021 as I'm still looking to push the details even further. This is by no means quick.

All of the shaders are custom made with Amplify. Waiting to add support for raytracing and animated wrinkle maps.

My updated skin shader now has dual roughness and support for cavity maps. To get that skin look, you need to properly set up your diffusion profiles, set up proper transmission falloffs, and properly author your maps. My current skin shader uses three separate textures (Dermis, Epidermal, and Subdermal) properly calculated in a similar process to programs like Marmoset and Arnold

 

I'm actually designing my own custom hardware for facial performance capture (gotta love 3d printing), but one issue with Ziva is unless your DCC pipeline consists of Maya, you're SOL.

On my character models, I'm even designing my own custom rigs to handle emotion and nuanced performances, which requires a dense head mesh (the one I just posted is about 30K).

 

Epic games have really made some great improvements to Metahumans in their new update!!! The new mesh to Metahuman tool can really reduce the time of creating realistic human models to just few minutes and not just that they have also added more customisation options as well!!! I really want unity to do something similar with their acquisitions

 

I agree. Though all of my character creations will be done from scratch, if it wasn't for it being locked to Unreal, I'd use it for background characters. Same issues arise with the Daz and CC4 character assets. Licencing. And after a while, they'll start looking too similar.

That's why building a very good base mesh that you can quickly tweak for any character not only speeds up production, but keeps the quality consistent

 

Awesome journey as always @KRGraphics - thanks for sharing.

 

You know me. I've always been pushing the quality of my work in Unity and it's not gonna stop. Still working on faking dual roughness without StackLit by adding two roughness values with one of them being slightly higher

 

The Unity one is pretty damn good. What is the diffusion profile set to? In my custom SSS shader, I have a scatter mask that allows me to scatter the skin properly.

Also, I have some fresnel on the roughness added to the roughness maps using Schlick Approximation. If you're interested, I can do a full shader demo. My texture setup is similar to Marmoset Toolbag and Arnold

 

That's our result after many changes, using DAZ Genesis 8.1 models.
Any feedback and ideas for improvements are very welcome. Right now the part we are most unsatisfied with is the hair, which hopefully will change once the mystic Unity hair package will eventually be released.

 

It's possible some kind of reflection for env lighting on the eyes would bring the whole thing to life more. Plus the mouth / lip AO stuff they used in heretic and enemies - that's available somewhere too.

Other than this you're rocking it aren't you?

 

Skin is looking good. Needs more colour detail for the epidermis and subdermis... Also overall roughness is a bit too uniform... Does this novel come with cavity maps? You can generate them from normal maps and blend it into the roughness map

And I second using an environment map and working the eyes. Hair is a challenge but I was able to get it looking really good with a custom shader. Turn down the scatter a bit on the skin...it makes the character feel hollow

 

Good idea to try the AO attachments, nearly forgot about those. But I am waiting for the Enemies demo, as they also have GPU based attachment handling in there.
On environment map: you mean using a reflection probe to show environment in the eyes? It's in, but tuned down, as it tends to look unrealistic under some lighting conditions, I guess because Reflection Probes are baked / not real time. But will try again, as things change a lot if you tune some parameters, so it's a constant re-evaluation of options to get it right.

 

I use cavity maps, but rather subtle, also the scattering is very high, as you noticed, this makes things look more uniform. This is a bit question of taste, but I might have overdone it a bit with SSS, will try to tune down again and post results.
With color detail you mean albedo maps? We have a variety of skin colors in game, so each one is different if it comes down to albedo coloring. Some have more details, some less (as the example I have chosen above).

 

I would try to get away from the doll look as much as possible. Punch up the cavity map a bit more and add more skin imperfections, like freckles. And overlay the cavity map into the roughness. Skin pores SHOULD NOT be reflecting the environment. And speaking of environment, set the reflection probe strength to 1 and leave it there. Any reflectivity strength on your model should be handled in the shader. Keep up the good work and post your next results. We can get very close to unreal quality. The next step will be post processing.

EDIT: I zoomed in on my phone and the skin looks VERY GOOD. Just turn down the scattering of the skin (scale should be 1 in your diffusion profile) and work on the environment some more

 

Here's an update with 2 changes based on your suggestions @hippocoder and @KRGraphics (big thanks for this!); using the scattering values from Heretic project, and setting the reflection probe to 1 and having full reflections on the model. I think it looks better, but I somehow dislike the strong reflections on the skin, but have not found a way yet to change / minimize the reflection probe influence on shader level without impacting the general smoothness / reaction to lights.

 

Yeah, you should share your shader info, I think a lot of people here in this thread would be interested to see it.
Since my original post starting this thread, a lot of tips and work here has basically allowed me to meet my production quality target window of a AAA style realistic character from commercial studios in the 2018-2020 range. A lot of the work boiled down to more normal layers, specifically micro normals, a lot AO tweaking. Advancements in HDRP helped as well, the lighting is much better in v12. The Unity HDRP eye and the static hair shader is also decent as well.




I think where Unity "breaks down" still at this point is in different types and styles of hair with the physics aspects to match. Unity has dropped some hints there's a new hair solution in the works, but for now there doesn't appear to be a robust solution.

The workflow is also insane as well, I'd just prefer to chose from a gallery of already completed models inside Unity that are already set with hair, eyes, etc. Maybe different styles of clothing for genre (modern, fantasy, high fantasy, sci-fi. etc). I hope Unity takes a big clue from Metahumans and puts something like that into the engine.

Would like to see what people are doing for hair physics. I checked out an asset called magica cloth but there's no easy tutorials and seems to be overkill and complicated. So for now I'm just using cloth physics on a hair mesh for characters with long hair and some bone-attached capsule colliders.

 

Excellent work and knowledge sharing in this thread.

this is beautiful. especially since it's a daz character but you made her look so soft and non plastic like. What were the many changes you had to do to get to this point? Are these the same imported daz textures with just some tweaks done? Specifically the depth of the eyes look amazing. Also this is just my opinion of course but i'm a fan of the overdone sss look to the skin, the second picture looks very textured, which is more realistic but less pleasing.

 

I am using mostly DAZ texture, some extra micro skin textures which are tiled to give more details.
Shader is basically the digital human / Hertetic Shader. Eye textures are again DAZ, but using multiple layers of shaders, as provided by DAZ model.
Normals / normal maps are the most important part, I also do mesh normal calculation on my own.

 

Do you have a image of your roughness map? The lips look a little dry .. And maybe turn up the scattering just a smidge

 

That's fair enough. I would need some time to compile some notes and explain my shader details

 

thanks for feedback! I redid everything, adjusted SSS and smoothness map so it's better distributed now: lips more wet and smooth, some areas a bit more dry and less scattered.
In the end it's and endless tweaking process. As we support many different skins and shades and also lighting conditions, there is at some point no more "better" or "worse", just compromises so that it looks ok in as many situations as possible. I think next important steps for more realism lies in hair and indirect lighting, which both is not yet the strength of Unity, although the new hair solution looks really cool and seems to be useable for vellus hair and eyebrows, too, and for lighting APV improved things a good bit.

 

Hey, all. So while I'm waiting for my presentation to upload to YouTube, here is some more HDRP goodness from my LookDev scene. I spent the morning, improving and adding details to my character.

"Grey Render" is active and if you zoom in really close, you can see the cloth details, all done in the shader. Until I get my PC built, I can't test it thoroughly for performance. In gameplay, you'll never notice the detail, but up close, oooh boy.

Video should be up in a few hours for those interested.

 

Wh

What is the youtube channel your presentation will be on? I'd love to check it out when it's uploaded.

 

This is my channel page. I was going to upload an unlisted video, but I may as well make it public

https://youtube.com/c/KashifRileyCG

 

I was lucky enough to get hair working correctly in my own custom shaders using HDRP and Amplify. Currently, it doesnt use StackLit but i've managed to get it looking soft and natural

 

Alright, guys. The video is live and I hope you enjoy it. Any questions you may have, you can ask them here or on the video (would prefer asking on the video).

 

I'm going to try this tonight and see if I can get a better roughness map. I'm using Amplify Shader and it doesn't support StackLit so trying to do it additively might help, though I'm using the inverse smoothness, like in Unreal 4

 

Morning. Some more HDRP awesomeness...tweaked my shader some more and now I've added a Skin Weights parameter which now automatically averages the skin to 1

 

Not to disrespect anyone, but if we are showing models in blacked out environment with pin lights, then pretty much anything will look good. I think we should start posting character models in directional light scenes to actually see if Unity can render AAA looking human characters...

Just my 2 cents.

 

My example was one directional light.
This thread is about characters, more specifically about rendering of skin, eyes, teeth, hair. Not sure how the scene would help here. If it is to proof that scene rendering is it is not AAA worthy this would IMO be another thread.
also on all papers and GDC presentations about this topic you rarely see a scene.

 

In my screenshot, I do have an HDR sky lighting my character, but I chose a flat background colour, because I can see the character better, and with all of the post processing, my HDR image would overexpose the game view. And in this particular shot, I DO have a directional light acting as the sun, since most of my game levels do take place outdoors

The focus is the character and I think I've gotten very close to having a very high quality character model. My characters all have detail maps

 

Yup, a 3 point light rig (fill, back, key) are usually what I use in my cinematic scenes. I also found using 2 different character setups/configs per model depending on the use (cut scene vs gameplay) is helpful as well, especially for directional lights where the camera is going to be more than 10m away from the character.

Overall lighting in Unity is well, it is what it is, and Directional Light with HDRP 12, is pretty decent. As far as character setup goes, regardless of light type, I find that trying to get within the ballpark of AAA without a custom shader (2021.3) is not possible, as HDRP\Lit doesn't have all the necessary settings.


Over the course of 4 years in going from the "block of wood" look to a character within the ballpark of AAA, I feel the following things are mandatory:

- Semi highpoly mesh (80k-150k tris)
- Micro normal maps
- Thickness maps
- Multiple ranges for smoothness
- Iris/Sclera/Limbus adjustments for eyes.
-Low poly hair design

I now feel like solo indiedevs can get a AAA-looking character into Unity, but the pipeline workflow is complex and requires a lot of rough and fine tuning, and you have to spend a decent amount of money on some design tools.

Where I'd like to see Unity go:
- Something akin to UE's Metahumans to cut the pipeline workflow down from days to minutes. (I believe they're considering this in trying to adapt Digital Humans / Ziva into a future version after seeing feedback requests for Ziva).