Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Why can't Unity HDRP correctly render AAA looking character models?

Discussion in 'High Definition Render Pipeline' started by cloverme, Sep 9, 2021.

  1. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    This is good for those who need characters quickly and NPCs. But I'm wary of potential license issues and also loss of creativity. Character Creator suffers the same issue as Metahuman: the character start looking similar and bland
     
    blueivy likes this.
  2. impheris

    impheris

    Joined:
    Dec 30, 2009
    Posts:
    1,580
    "Indie" in most cases does not mean "we have money" xD, so, no, is not for indie devs XD
     
  3. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I'm an indie dev, and I would never shell out near a grand to generate human characters that are poorly optimized and I'll have to redress them in my custom garments. Also their textures would have to be redone to work with my custom shaders
     
  4. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    622
    I guess different strokes for different folks, there's not really any alternative for developers that want realistic character models besides paying an artist themselves, or going to unreal.
     
  5. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221

    Also, while I agree with you about the effort needed to make a AAA production quality from the raw model, the fact is the raw model is far ahead in terms of hours of modeling goes. The extra work you lament doesn't seem to be anything compared to the modeling itself for most people who are not modelers (like myself). So from an Indie perspective, I would use it, even if my production wasn't AAA production in the end. Reallusion brings a more accurate model within my grasp. Period.
     
    Last edited: Jun 23, 2023
  6. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Sad but true. And now they have Metahuman Animator, which is undoubtedly amazing. And there is nothing for us like that in Unity, aside from authoring your own rig
     
    blueivy likes this.
  7. impheris

    impheris

    Joined:
    Dec 30, 2009
    Posts:
    1,580
    Why not MakeHuman? Is totally free (i think is even open source) is used to create realistic humans, it has lots of settings for head, torso, legs, arms, genitalia and details for mouth, eyes all that stuffs, i mean, yeah is not metahuman, but you can create a really good base shape with really good topology on makehuman to start, you can also install plugins from the community...
    1.jpg 2.jpg
     
    MrBigly likes this.
  8. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I remember MakeHuman... I could use this for NPCs and build base meshes from it to create a new character. Would have to make the topology animation ready
     
  9. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Hmmm I think it's less about meshes and more about the workflow pipeline and quality of the character skin & overall looks. I think "realistic characters" boils down to multiple high resolution textures and advanced shaders that take skin, eyes, and hair into account to produce a AAA quality style character.

    There's lot of free characters out there or characters from the asset store etc, but they tend to not look AAA or they're not fully rigged for cinematics (facial mocap) like MetaHumans or Reallusion. Unity's Digital Humans looks amazing, but they've done little to help the community, there's no one from the wider community either A) using it or B) making tutorials for it, which is kind of the kiss of death for a Unity feature.
     
    KRGraphics likes this.
  10. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    This right here. And for most indies, the amount of work needed to do high end characters is astronomical, especially in the rendering front. And most younger artists don't want to put in that level of work because they want something fast with minimal stress. Also, unless you have some 3d printing and engineering knowledge to build your own HMC, Metahuman Animator gets your facial animation ready to go.

    Speaking of quality work, Check out SakuraRabbit on Instagram. The models in Unity look amazing
     
    Last edited: Jun 23, 2023
  11. impheris

    impheris

    Joined:
    Dec 30, 2009
    Posts:
    1,580
    mmm that is not correct IMO, while textures, shaders, hair and all that stuffs are really important, the meshes is the most important thing in realistic character creation, in fact, the topology and the details of the mesh will be what defines your workflow if you do not do it right (even the rig is more important)
    BTW The screenshot i posted is from a basic mesh with almost zero customization.

    Now this is a personal opinion but, IMO and i know this is a very unpopular opinion, a good indie does not care about hyper-realistic humans, just imagine the amount of work to create an enviroment that match the character, good indies will show their ideas in a simple way, so, i get it, hyper-realistic humans are not for indies.
    I insist, if you need realistic humans but you are an indie a very good solution is makehuman, you can create your own skin later, that is easy

    This is just one tab:
    1.jpg
     
    KRGraphics likes this.
  12. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I've used MakeHuman in the past for NPCs, and I would optimize it with a lower polygon counts and tweak it in ZBrush. Now my hero characters are done by my hand.

    I just wish Unity had some kind of tools for rigging and facial animation in engine. I'm sure there is an asset that will do this, but I have to search for it
     
  13. impheris

    impheris

    Joined:
    Dec 30, 2009
    Posts:
    1,580
    can you explain to me why that would be useful?
     
  14. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221
    Not only is this a reasonable opinion, it is unrealistic for an indie to expect to produce AAA quality without a new generation of tools available, like MetaHuman. Since MetaHuman is not available for this audience, I would agree with you; and I would even say that Halo Reach level quality is good enough for an indie game. What would make an indie game sky rocket is not the quality of model, but the art being clean and the story being unforgettable. These are my targets.

    A studio haz gots to know its limitations.
     
  15. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Mainly to get things working in engine straight away. Imagine being able to Livestream motion capture data onto your character in Unity with a rig that can be quickly processed in real time
     
    Last edited: Jun 25, 2023
    cloverme likes this.
  16. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Halo: Reach is still a great looking game
     
    MrBigly likes this.
  17. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    622
  18. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Yeah, I don't disagree with your point. There's certainly layers of depth that goes into making a game. Devs make games for all sorts of reasons, telling a story they feel they need to tell, recreating a style of gameplay they enjoyed for others, making a game for profit, etc.

    There's definitely a segment of indie devs out there that feel you don't need realistic character designs to make a good game and they're not wrong. There's a number of games that are from AAA studios that have been successful that don't have any cinematics and feature low fidelity representations of character designs.
     
  19. ARealiti

    ARealiti

    Joined:
    Oct 4, 2013
    Posts:
    132
    Thank you. This will probably make some people flip out but I got the Daz3D characters to work pretty good in real time. The hairs are CGTrader assets.
     
    blueivy likes this.
  20. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221

    What is temporarily impressive are graphics (e.g., modern day hair textures and effects) because everyone knows these are dependent upon technology which is ever increasing. These present the initial WOW factor, not much else.

    What lasts a life time are amazing stories, iconic music, clean art work, and memories of playing well designed game experiences.

    But on the flip side, none of this means you can go back to the original Half Life graphics and animations and expect anyone to take you seriously.
     
    KRGraphics likes this.
  21. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Nice... Daz has always been somewhat of a difficult import, more so because the characters often have a lot of issues with hidden meshes and geoshells. Mostly I have gone to importing Daz models to CC4 then Unity with decent results. They've also been ultra slow to update import tools, do they have anything updated yet for HDRP?
     
  22. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    832
    There's a good working Daz to Unity bridge, which also allows to export with subdivision, has HDRP shaders already etc.
    I guess DAZ is currently best option from ratio of cost to quality.
     
    cloverme likes this.
  23. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    520
    I was at a Unity Developer Day recently where Mark Schoennagel gave a presentation very similar to the one in this video. However, there was another part of the presentation that I don't notice when quickly scrubbing through this version I just found on YouTube. He also had Maya open (I believe). He was making changes to the face rig in Maya and the changes were all updating live in the Unity editor. I'm not 100% sure if this is the same kind of thing you're looking for, but maybe it's a clue that can point you in the right direction.



    I think it might also have something to do with Ziva, and a Metahuman face rig in Maya. I'm sorry that my knowledge is fuzzy in this area. However, Mark Schoennagel knows how to make live changes to the Metahuman face rig in Maya that show up immediately in Unity Editor. (If I'm remembering his recent presentation correctly.)



    Sorry if any of this was already addressed previously in this thread. I just noticed the comment about face rigging tools within Unity, and I remembered seeing something sort of like that. It doesn't seem like any of the current YouTube versions of Mark's talk have the part that I saw though, I'm not sure why.
     
    Last edited: Jun 26, 2023
  24. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Unity did make a tool called Live Capture a few years ago, which has a Take Record system. However, the face & head have to be rigged to use it. There's also some tutorials on Youtube for it as well:
    https://docs.unity3d.com/Packages/com.unity.live-capture@3.0/manual/index.html

    Overall, Digital Human and Ziva are well mentioned in the thread, but no worries on mentioning it. The situation with both of those is that the workflow pipeline is too exhaustive for the larger community to use. Unity seems to couch Digital Human as a proof of concept rather than a system that's extensible enough to work with model formats that most of the community uses, like FBX. Ziva is something that very, very few people are using, it also costs $1800 per face to train. There's definitely a lack of a community step-by-step tutorial from taking a model to a complete Digital Human or Ziva trained model all the way into Unity. When I pressed Unity on that (before the layoffs) they were aware and acknowledged that DH was very much "garage shop work" and Ziva was more geared for larger film studios like Warner Bros.

    They also had a Pulse feedback survey and were hinting around building a "library of ready use characters" with the some of the same features as MetaHumans, without mentioning the word MetaHumans.
     
    KRGraphics and CodeRonnie like this.
  25. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I've been watching the Enemies demo EXTENSIVELY and trying to update my pipeline to allow this kind of visual fidelity, especially for facial performance capture (I'm building my own HMC for this very reason). Luckily, I've been able to reverse engineer some advanced facial rigs and break it down into easy to build components (such as FaceIt in Blender which is AMAZING). It's just sad that we are left to build it ourselves, while UE5 has a built in tool for this.

    It's a shame that Unity stopped working on Live Capture, because they could easily keep updating it to use ML to REALLY push it and make it easy to use. The best I can do to get even close to clean facial animation is hand authoring my character faces.

    These looks pretty damn cool. The eyes look a little funky on the Braveheart dude, but everything else look great. I spent many weeks building shaders for Skin, Hair, and Eyes using Amplify, and the results are CRAZY!!
     
    cloverme and CodeRonnie like this.
  26. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221
    Does anyone know how MetaHuman generates their facial models? Does an AI generate them? I mean, true AI generation from scratch?

    I have a feeling we will see some amazing AI generated full body, full rig, full texture, full everything for any game engine soon enough.

    I came across the idea that AI generated textures for assets alone are pretty nice today, and I will be looking for an AI service to generate most or all of my asset textures for my game moving forward.
     
  27. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Metahuman faces are based on real life scans of people and they can be blended together to create different faces. And now you can bring in your own character heads and it will smoothly integrate it
     
  28. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    520
    One big theme I noticed from Developer Day was a push toward AI integrations in the Unity editor. They are looking to add the option to just type requests into a text field and have Unity create assets for you with AI. As a non-artist I was particularly interested in some of the AI generated textures and models in the demos. It would be nice for me to be able to just request a cobblestone texture right in Unity and not have to search elsewhere.
     
  29. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    You could also create such textures in Substance Designer :D
     
    cloverme and CodeRonnie like this.
  30. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221
    This was what I was seeing in a video and they generated a brick texture using the ChatGPT. I wouldn't mind if the artwork was more AI than photo realistic, because it would be unique in and of itself. Just so long as all textures are the same style, it would work.
     
  31. impheris

    impheris

    Joined:
    Dec 30, 2009
    Posts:
    1,580
  32. ARealiti

    ARealiti

    Joined:
    Oct 4, 2013
    Posts:
    132
    Thank you, I am still using the Live Capture as DAZ has an export option for the export of the ARKit blend shapes that works pretty good, I do all my own face acting and get the voices from a voice generator company. Here's something I'm currently working on running in the Unity Editor, talking starts about 3 seconds in (this is on an old 1060 NVidia machine and still runs at nearly 60fps built in 1920x1080, about 40fps in the editor at 1920x1080)...

     
    Last edited: Jun 27, 2023
    blueivy, KRGraphics and CodeRonnie like this.
  33. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    I hope to take this even further with motion capture and a full facial performance
     
  34. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Unity just put this out today, looks like generative AI characters in a new tool called Unity Sentis. Very few details available, looks very similar tech to D-ID. Could be of use in certain gameplay situations:



    There's a forum for it, but it's got permission errors today:
    https://forum.unity.com/threads/int...d-unity-sentis-ai-powered-creativity.1453384/

    A few months ago I was experimenting with midjourney, d-id, and replica for voice:
    https://i.imgur.com/7NoNx9g.mp4


    Speaking of AI... It's also worth mentioning that Replica (Voice AI) has a plugin for Unity, but also has a plug-in for iClone 7 (v8 wasn't working a few months ago, but they may have updated it) which does the lipsync with the generated audio. The Unity plugin is audio only and doesn't do any of the 3D work.
     
    CodeRonnie and Qleenie like this.
  35. Qleenie

    Qleenie

    Joined:
    Jan 27, 2019
    Posts:
    832
    Some update on the long journey to AAA looking characters; now including special AO solution of Unity for Teeth / Mouth, partly simulated lashes and brows, and some self made AO using SDFs. I really hope Unity manages to repair the new Line Renderer in 2023.1 in order to have really good looking hair; this will be a huge step into more realistic looking characters:

    shot3.PNG shot2.PNG shot1.PNG
     
    KRGraphics, blueivy and cloverme like this.
  36. ARealiti

    ARealiti

    Joined:
    Oct 4, 2013
    Posts:
    132
    One of the interesting things is that once you get past making characters light / look better, it is then harder still to get the lighting in the whole environment to work with the character lighting so the character still looks good and the sun / other lighting on other things / characters also looks good together
     
    Last edited: Jun 27, 2023
    cloverme likes this.
  37. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Very true, due to Unity splitting the pipelines, they lost years of effort on making significant improvements on realtime global illumination. Ideally they'd move to SDF GI, which would really improve lighting to be more realistic. This would drastically reduce the complexity in unity's current system of a mish-mash of configurations to balance lighting, exposure, shadow, and ambient.
     
    Qleenie likes this.
  38. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221
    This right here is why I have come to the conclusion if I want to make a photo realistic game experience, I should just use Unreal. Unity is more for games that are focused on and rely upon the game and not the graphics. I am not saying everyone should adopt my view, but this is my view.
     
    Last edited: Jun 27, 2023
  39. blueivy

    blueivy

    Joined:
    Mar 4, 2013
    Posts:
    622
    one solution to that would be per character lighting, or hero lighting like is used in the Horizon game. It'll make your character stick out from the environment but at least you know their lighting looks good all the time.
     
    ARealiti likes this.
  40. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    Now let's see if this works on my own custom characters
     
  41. ARealiti

    ARealiti

    Joined:
    Oct 4, 2013
    Posts:
    132
    Thank you, yes you can just darken/fade the background on character talking / close ups and concentrate on the character lighting. When not up close on characters on full scenes it takes a lot of trial and error to get everything looking reasonable, I'm almost there but never quite satisfied ;)
     
    blueivy likes this.
  42. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Here's a good example, but from an unreal MetaHuman example. This is a 3D model with a face and body rig. The iPhone being used is taking that mocap data and translating it right into the editor. The new MetaHuman 5.2 updates make this easy to do with relatively low cost equipment.

    This is also a good example of how far Unity has slipped in the character pipeline, not only in terms of having ready made and customizable characters that you can use for...free... but also the high end quality as well for doing cinematic close-up work.


    (The 1:30 mark shows a an iphone being used to translate the mocap in realtime)

    I do something very similar, but in Reallusion iClone, which is just a couple of extra steps to export that to Unity. However, the model quality is something on par with Digital Human/Ziva without a massive team or extensive workflow to make it a reality.

    It's something a single developer can do... something Unity has obviously (and unfortunately) lost sight of.
     
  43. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    (Which by the way, I'm not advocating Unreal as a gamedev platform. I prefer Unity still for it's ease of use to get started and do something very quickly).

    Here's an updated workflow from Reallusion iClone... probably the best indie solution for getting "AAA style" character into Unity (presently), while not free, it's certainly the easiest process from a Next, Next, Finish standpoint.

     
    Last edited: Jun 28, 2023
    Rewaken, ARealiti, Kreshi and 2 others like this.
  44. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    520
    That's awesome! I can definitely see what you mean.

    I'm starting to get quoted now, and I have to be clear that I am a programmer and this artist wizardry is out of my wheel house! :D I have my own opinions on what parts of the engine they should focus on, and what they should put less focus on, but they are unrelated to the art and rendering pipelines as those are not my areas of true expertise. It seems the others in this thread already have a good grasp on the state of things without my musings. :p
     
    cloverme likes this.
  45. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    This is awesome. And most character artists in Unity don't build their own rigs, especially for the face. I'm planning on using my new prototype to do EXACTLY this. Not using an iPhone but taking it way further with more facial performance detail
     
    cloverme likes this.
  46. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    It's sad.
     
  47. ARealiti

    ARealiti

    Joined:
    Oct 4, 2013
    Posts:
    132
    Thank you, yes I've been thinking about Daz characters into CC4 then to Unity rather than just Daz to Unity. But, the price for this is quite high IMHO, but may be worth it, the full face and motion capture with Rokoko is a full interesting option but again quite expensive, but cheapest full option on the market.
    https://www.reallusion.com/store/product.html?l=1&p=3d
    And have to pay for the Rokoko with a 10% discount.
    https://store.rokoko.com/
    https://manual.reallusion.com/Chara...l-Conversion-for-DAZ-Genesis-8-Characters.htm

    The biggest problem here is that spending $1,000 for the facial / model import part or $7,000 which would be a really nice full solution however, it still comes back to having to get good characters, costumes and scenes that work together for your game idea and doing the face and body acting (which some of the store animations may have done better) and getting lighting right and a game that is playable, in my case I'm just going with this el cheapie option with Daz to Unity with ARKit for free and store body animations and concentrating on what I can do and see if anyone is interested in my game. Sort of the same thing for going from Unity to Unreal, it would cost me a lot of time to do that and Unreal is a fair bit harder to use than Unity, no offence, it may be slightly better but my indie game is never going to be perfect it is a trade of money/effort to likely return, i.e. nothing :D I will try the free trial version of iClone / CC4 to see what results I can get in 30 days;) maybe just $1,000 for Daz to CC4 to Unity with iClone facial stuff would be worth it if it looks better than just Daz to Unity with ARKit.
     
    Last edited: Jun 29, 2023
  48. MrBigly

    MrBigly

    Joined:
    Oct 30, 2017
    Posts:
    221
    I think that people who play games with hyper realistic facial textures, shapes, and animation will pick out the flaws more readily than lower quality faces that were never intended to be hyper realistic and therefore no attempt to look for flaws is made since it is accepted as is by design.

    I honestly don't see any reason to try to develop hyper realistic faces in today's games. The bar is too high for the benefit it promises to provide, period.
     
  49. KRGraphics

    KRGraphics

    Joined:
    Jan 5, 2010
    Posts:
    4,453
    This could also be mitigated if you create characters that are non human and animate them convincingly, such as aliens
     
    cloverme likes this.
  50. cloverme

    cloverme

    Joined:
    Apr 6, 2018
    Posts:
    189
    Daz has updated their bridges in the past few days including the Unity one. It's still hit and miss though, one character model imported very nicely (probably the best I've seen from Daz), the other character model test, not so much and was a mess, so results will vary. It did seem to import props/static meshes much better, but all will still be largely unusable in unity due to high poly meshes.