Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Why can't Unity HDRP correctly render AAA looking character models?

Discussion in 'High Definition Render Pipeline' started by cloverme, Sep 9, 2021.

  1. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591
    If you're using Reallusion's pipeline victors shader plugin is a must since the auto set up for unity went open source and regarding facial mocap you have a few paid choices you just need to do the one that suits you the best. Really I have no idea why AccuFace is a separate product for RGB cam support and video files, but hey ho.

    You can find out more specs on the plug-ins here
    https://kb.reallusion.com/Product/53172/Which-facial-mocap-plugin-should-I-choose
     
    cloverme likes this.
  2. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    The Mocap from video solution sounds interesting; can you use the result outside of Reallusion? E.g. in Unity to drive blendshapes?

    I got some improvements to eye shader and skin, it's a lot of work and sometimes a bit frustrating if you switch to Unreal and everything is already there.... skin-shader.PNG Anyway, took some clues from the Meta Human skin shader to improve skin shading in Unity:
     
  3. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591
    You can use expressions driven by unity blendshapes yeah, just not character morphs.
     
  4. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    Morphs and blendshapes are the same; currently we drive expression blendshapes (facs based) with Unity's Face Live Capture, recording real life actors.
    I was curious if the Reallusion software allows to record face mocap from videos AND export as a file usable inside Unity (e.g. human readable .json or .csv).
     
  5. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591
    Yes for sure, you'd have to convert the key frames created, I recommend looking at Reallusion's python API for that and create a script. https://wiki.reallusion.com/IC_Python_API
     
  6. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Those eyes are kinda bothering me. Do you have an HDR setup for reflections? This was a difficult challenge for me as well
     
  7. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    Reflections are from a static reflection probe only so far. I need to keep everything realtime and 60+ fps even in VR.
     
    tmonestudio likes this.
  8. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    I now came to a point where I basically tried everything Unity has to offer in HDRP / Digital Human package, including now also peach fuzz hair and attached brows / lashes. I'd appreciate if anyone has ideas how to improve further.

    Capture-fuzz4.PNG
     
    DavidMiranda, KRGraphics and blueivy like this.
  9. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591
    Modelling light and studying portrait lighting in particular. Provided your textures and shaders follow PBR principles you'll be able to provide lighting that showcases the face, shape, skin pores etcetc. It may even show problems or limitations that were unknown, you can find a typical flat lighting that works well as a base too and go from there.

    Fact of the matter too, objects don't look good in all lighting, we largely ignore these as observations in day to day life, so it's still worth studying.
     

    Attached Files:

    KRGraphics and nehvaleem like this.
  10. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I often do HDR only lighting to see how the skin would REALLY look before adding lights in the scene. I have environments in my game that are usually overcast or night lighting, and getting this correct is paramount.


    In my custom skin shader, I have set it up to where you will need three albedo textures to make up the skin, and a lot of my tests, the skin looks so much more natural, especially simulating blood flow and hair follicles. I haven't be able to use Unity on my old laptop, but I am looking forward to doing more tests like this. I love the eyes a lot more. One thing I've done is to NOT use specular mapping for the sclera and stick to either a roughness of .15 or using a Clearcoat texture.
     
    hopeful likes this.
  11. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    hopeful likes this.
  12. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    I have seen this, same guy seems to work on SSS shader for URP and also HDRP. There also seem to be some assets of the guy already on asset store, but I guess those are for built in pipeline. Interesting stuff, hope it will hold up to the previews in different lighting conditions.
     
    blueivy likes this.
  13. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    This is possible because URP supports custom lighting (I had an eye shader like this a long time ago) and it looks fantastic.
     
    blueivy likes this.
  14. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
  15. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Was just about to post that as well. Dev is posting his work on twitter... looks to be for URP only though.

    https://twitter.com/_david_miranda

    Hopefully the dev decides to make both
     
  16. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    He wrote on some tweet that he considers to do a HDRP version, too. Probably until he finds out that it’s more difficult to do custom stuff in HDRP.
     
  17. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Why Unity has 3 pipelines is dumb anyway, but that's another topic lol

    What really drives me nuts is anytime anyone posts anything new about character design on social media, someone always replies with the Heretic screenshots and says Unity has a digital human package. Unity has damaged character design for Unity and doesn't even realize it.
     
  18. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Yep. Though I got my skin shader looking dead on good and my eyes look natural with Subsurface Scattering on them. The highlights in the iris you'll RARELY see in reality unless the characters are in very bright environments and you're looking at them in the eyes.
     
  19. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    The digital human package is bullcrap... Your average creator doesn't have the patience (or the knowledge) to do characters on the level of The Heretic or Enemies, especially with skin shaders and complex face rigs.
     
  20. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591
    Absolutely, a good lookdev is a great cornerstone, ambient lighting can really set up a look

    HDRIs can help with software consistency as well, we can end up so easily misdirected at times it helps to have cross software consistencies.

    Most of the time I transfer a light rig as well just a basic 3 point, but IES in unity HDRP helps a bunch of I'm doing real world comparisons.
     
    KRGraphics likes this.
  21. methusalah999

    methusalah999

    Joined:
    May 22, 2017
    Posts:
    637
    Could you give more information about attached brows and lashes? plus some detail views? Are there strand based or in volume somehow? What kind of texture resolution do you use for them?
     
  22. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I also build my own HDR probes and skies for rendering, and the new Adaptive Probe Volumes help immensely
     
    HIBIKI_entertainment likes this.
  23. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591
    The anchor points is part of the digital humans workflow unity created ( The Heretic) the anchors can be used to position for any character, adjusted and reused and switched out, this is also the case for Enemies. What you use is entirely up to you.

    In many cases Queenie's perhaps too, an underlying texture brow and then alpha card brows using the attachment. You could switch card for strand and Instancing if you wanted.

    https://unity.com/demos/enemies
     
    methusalah999 and Qleenie like this.
  24. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    Exactly as @HIBIKI_entertainment said; I use the attachment system from digital human package to attach meshes, no textures on the attached meshes, but still using eyebrow textures on the skin below the attachment, as attached strands might look too thin.
    Unfortunately, I needed to rewrite most parts of the attachment system, as the way it works does not really work well with prefabs, as both the attachment and the target need to be in the scene; which is okay for Unity demo purposes, as long as you don't need to switch attachments a lot.
    I'll try to do close ups when I find some time!
     
  25. methusalah999

    methusalah999

    Joined:
    May 22, 2017
    Posts:
    637
    I don't know about this feature. Is this a way to attach an object to a skinned mesh renderer vertices, so it follows the skin deformations? If so I need to check on that and find doc.
     
  26. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    methusalah999 likes this.
  27. methusalah999

    methusalah999

    Joined:
    May 22, 2017
    Posts:
    637
    So you're telling me that something Unity did in their tech demo does not work in a real world? ^^

    I understand that the strand base eyebrows are not thick enough. Or they just disappear, flick, or become stupidly thick when the camera is far away. Anyway you still have to draw the eyebrows on all the maps on the skin for each characters (because eyebrows shape is always different), which seems like a hassle. If you need a color base underneath the strands, maybe you could have a second skinned mesh renderer on top of the skin with the drawn eyebrow.

    Someday we will have a good line renderer for strands, with actual super sampling AA ^^
     
    Qleenie likes this.
  28. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    exactly, to our all surprise most of the digital human package does not work in real projects ;)
    for the attached hair I don't use the HQ line renderer which can do all the things you described, because it's way too expensive; instead I use the simple shader from the Heretic project. And it only looks okay with DLSS or at least TAA activated.
     
  29. methusalah999

    methusalah999

    Joined:
    May 22, 2017
    Posts:
    637
    I understand. I personally struggle a lot with the hair shaders in situations where there is little to no direct light (like an indoor scene lighted by the sun from the outside, with only baked GI). Both Kajiya-Kay and Marschner models fail in this situation and make flat hair with strange transcluent parts.
    I tried using a probe proxy volume, which works very well for the skin, but the hair node does not seem to catch any directionality from it.
     
  30. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    Unity improved the hair shader for strand based hair a lot in 2023.2; but unfortunately they forgot to update the documentation, so it's a lot try and error to understand how it works. And probably it only works with the hair package, which I think is a marvel in the Unity eco system. It's experimental, but much more stable than many non experimental Unity features.
     
  31. methusalah999

    methusalah999

    Joined:
    May 22, 2017
    Posts:
    637
    Did Unity forget to update a documentation?? ^^

    I agree. I switched to Unity Hair in my game too, which is a shame since I'm the author of HairStudio ^^ I never understood the restriction in the collider count, though, it's very annoying to me because I interact with hair a lot in my game. But overall a very good tool, as you said. The hair node can be used without Unity Hair (I used it with HairStudio) but some features like the hair/hair shadowing requires that we make the same computations as they do in Hair (which I didn't). I suppose it was hard to avoid.

    It's still a shame that they didn't add an efficient line renderer, especially since they take almost all the tech from AMD TressFX, which mastered the line rendering for hair back in 2013 with the Tomb Raider reboot.
     
  32. HIBIKI_entertainment

    HIBIKI_entertainment

    Joined:
    Dec 4, 2018
    Posts:
    591

    Nailed it! :D
     
  33. DavidMiranda

    DavidMiranda

    Joined:
    Nov 30, 2012
    Posts:
    617
    That hair caught my attention :) Soft alpha blend and no sorting issues? How :D
    Is it the standard HDRP Lit shader?
     
  34. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    831
    This is HDRP Hair shader set to strand based (Physical Cinematic in 2023.2+); hair itself is strand based simulation from Unity hair package. It's also using the new HQ Line rendering feature from HDRP, which basically makes the lines /hairs thinner, but tries to keep the volume and does AA.
    Also the hair package calculates the probes used by the shader for ambient lighting.
    Shader is Opaque, so no sorting issues.
     
  35. DavidMiranda

    DavidMiranda

    Joined:
    Nov 30, 2012
    Posts:
    617
    Okay!, thanks for the info
     
    Qleenie likes this.
  36. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    220

    I thought I would follow up on my earlier post to talk again about how a simple light to eliminate sharpie black shadows is a simple fix, even in Unreal. I am just now getting into shadows and lighting (what a trip).

    The first pic is with only the sun at 10 lux. the second is with a non shadow light 180 degrees to the sun at .4 lux. It eliminates the sharpie black. It's a simple fix. I am sure that there are simple things I can do further where this may not be necessary, but it is so simple I can use it if I had to and it makes the image much easier to see. That is the point. To make the character or the models easier to see, so that the human perception can "immediately" recognize and go on to process other elements of the scene, not get bogged down trying to figure out what is being looked at.

    upload_2024-1-20_21-19-56.png



    upload_2024-1-20_21-20-21.png
     
    blueivy likes this.
  37. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    I also like the specularity it adds.
     
  38. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    220
    They actually have a component for the scene called the SkyLight which can do some of what I want, but not as strong, and no specularity.
     
  39. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    621
    Chatavatar which allows someone to create faces from photos or a text prompt is getting a Unity plugin! Super cool
     
    hugokostic, marcoantap and Qleenie like this.