Search Unity

[WIP][UMA] Master thesis player avatar creator

Discussion in 'Works In Progress' started by Mikael-H, Jul 24, 2015.

  1. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    2,014
    We're using UMA. :)
     
    Mikael-H likes this.
  2. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Would you be interested in working a bit with us in alpha/beta then perhaps? Free of charge of course, this is still R&D! It would be great to work with someone using it "for real". This is still a bit in the future though, there is still a lot of work that needs to be done :)
     
  3. derkoi

    derkoi

    Joined:
    Jul 3, 2012
    Posts:
    2,014
    Yeah that sounds very interesting. Hit me up when you're ready! :) Thanks!
     
  4. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Will do!
     
  5. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    717
    Where the hell is it man i want to buy it!!!!
     
    Mikael-H likes this.
  6. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,026
  7. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Hehe nice to see someone is very interested :)

    The work has had to be put on hold for a little while due to the team having IRL jobs that need to be taken care of. There is a very very early preview here that you could try:
    http://vikingcrew.net/Selfie_to_Avatar/index.html
    This is just intended for some internal testing so the usability might.. hrm... leave a bit to be desired :)

    What would be you use case for this system?

    Actually just so there are no misunderstandings, that asset is just a collection of face textures generated by this system!
     
  8. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    717
    I was going to allow user to upload pics of their face and attach it to the UMA avatar.
     
  9. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,026
    Oh? Oops! :)
     
  10. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Then you'd be a perfect candidate for a user :) I'll post here when development of this system starts up again!
     
  11. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    717
    Awesome, when do you think that will be?

    Do you think its possible to just apply the picture to the UMA face, then let the user slide the picture around on the face until it fits without doing any of the fancy math that you are doing?
     
  12. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Not sure about timelines at all, but the more people are interested in this the more I feel like getting going with it again :) I am currently working ~10hrs per day coding pretty advanced stuff for unity at work so my brain is a bit fried when I get home, that's why I put this on hold for a bit... As soon as things calm down a bit I hope to be able to get going again. It is definitely a long term plan of mine to make this a system easily available for other developers!

    Well you could I guess just do an orthogonal projection of a cut out face in a photo from the front and get something that will have similarity to their looks. Or maybe have them set a few vertices themselves and map the uv-coordinates from that. But it will still be quite a bit of work to get it to fly :) Did you read my paper on the subject? (linked above) It might give you a few ideas.
     
    Last edited: Sep 28, 2016
  13. daschatten

    daschatten

    Joined:
    Jul 16, 2015
    Posts:
    191
    Using UMA and very interested in this! Got some first results with the WebGL version, not perfect though :)

    Any news about further development?
     
  14. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    I got a bit too much on my plate right now to work on this but this is really something I want to keep working on in the future! There would be a lot of work to do before it is practically usable by other though :)
     
    daschatten likes this.
  15. radiantboy

    radiantboy

    Joined:
    Nov 21, 2012
    Posts:
    941
    Wow I am very interested in using this, if its available ?
     
    Mikael-H likes this.
  16. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Not exactly available, you CAN generate an avatar yourself and export the texture using the web gl app linked above but it is not available as a plugin. It is my hiope that it will be one day, but right now I unfortunately do not have the time to finish it...
     
  17. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    So... I have been working on cloud solutions in a different project and realized that microsoft azure would be a great fit for this solution. Anyone has any ideas or objections? I'm thinking something like azure blob storage to store source images and final textures and maybe some backend stuff as well. There are some excellent youtube tutorials on azure + unity here:


    if anyone is interested.
     
  18. lucasmd

    lucasmd

    Joined:
    Apr 3, 2016
    Posts:
    6
    Hey Mikeal,

    Hope you are all well !

    I was looking for some cool plugins/projects for facial landmark detection, and 3D face reconstruction, because I wanted to create a quick demo, mainly for VR/AR, with Unity3D/Hololens. I was wondering if you have planned to release a version of your project, at least a light version: It could be a really interesting and valuable plugin for Unity3D.

    So far, you have something really cool for developers with your advanced "editor" options, for the facial landmark detection, and to correct landmark points, and many other features could help to complete your project (eg. machine learning, tensorflow, third-party apis) and, so far, I am really curious to know if you planned to release some local and not only cloud based version of your plugin.

    In any case, I would be really curious if you could help the Unity3D community to bootstrap such feature with some advices or insights you gathered from your master thesis.

    Current scope of references:

    Cheers,
    Luc
     
  19. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Well I do not have any plans right now to build anything for local, as I think cloud is the way to go. If you just want to try some stuff out then you could use the web gl app above.

    If someone else wants to build something then there are (I hope) many insights in the thesis linked above :D

    I think the basic approach will work but there are probably better segmentation algorithms that could be used.
     
  20. norbertots

    norbertots

    Joined:
    Jul 4, 2014
    Posts:
    15
    Hi Mikael.. any news about the project ?? when it will be in Asset Store? my email norbs@uol.com.br - i very interest in it
     
  21. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Oh... no news unfortunately I just haven't had time to work on it. I think about it almost every day though :) I have been contemplating some kind of cloud based solution but I have some learning to do first...
     
  22. Brian-Santos

    Brian-Santos

    Joined:
    Apr 10, 2014
    Posts:
    1
    Seems like facebook blocked your app, it says: "Facebook has detected that Selfie to Avatar is not using a secure connection to download information.
    Until Selfie to Avatar update their respective security settings, you will not be able to use Facebook's login to enter it."
     
  23. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    oh... I didn't think anyone was interested in it :) It's been quite some time since I gave this any thought.
     
  24. nsmith1024

    nsmith1024

    Joined:
    Mar 18, 2014
    Posts:
    717
    im interested!!
     
  25. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    I didn't even remember using Facebook login.. This is a project that I really want to do one day but I think my naive approach needs to be done with a higher level of AI as well as using photogrammetry rather than only two pictures. Getting a face texture worked well but there was a way much errors in the estimation of 3D model of the face. Also, all this should run in the cloud and allow a user to reuse their avatar in several different applications. Not to mention the low code quality in what was essentially a research project.

    For now this will have to be a dead project but whenever I do the reboot this project deserves I will post here I promise :)
     
  26. rcthompsonuk

    rcthompsonuk

    Joined:
    Aug 2, 2019
    Posts:
    3
    Hey Mike I know this project is dead for now but just wanted to thank you for making your tool available http://vikingcrew.net/Selfie_to_Avatar/index.html I've had a lot of fun messing around with it. How you've managed to do it is way over my head but just wanted to say its pretty amazing. Its shame you haven't had the time to do more with it because I would pay money for it with just a few tweaks! Thanks again!
     
    Mikael-H likes this.
  27. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    Hey thanks for the kind words!
    I had forgot I actually had it up online :D
    Who knows, with enough encouragement maybe I will find the energy and will to make a new implementation, worthy of the asset store. I know for sure I could make something a lot better today than I could at the time...

    I am very happy to hear it was of some use to you!
     
  28. Firlefanz73

    Firlefanz73

    Joined:
    Apr 2, 2015
    Posts:
    1,129
    Not bad. If it would be just a Little more improved and would Export the textures and the whole uma face overlay, it would be really interesting in the asset store I think, if not too expensive :)
     
  29. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    I guess that just creating textures may have a good value... Creating a 3d model is really hard, textures is easy. I'll think about this once more :)
     
  30. Firlefanz73

    Firlefanz73

    Joined:
    Apr 2, 2015
    Posts:
    1,129
    The user or game developer can Change the 3d model himself using the uma parameters like nose length etc :)
    Or you could try to use them to get closer to a result instead of working on the mesh yourself.

    I am in for some testing, this is interesting :)
     
  31. Mikael-H

    Mikael-H

    Joined:
    Apr 26, 2013
    Posts:
    281
    No promises that this will be made at all, but it does actually sound quite interesting to make :) I think UMA has a bit small user base, so I need to think about a way to make it worthwhile for other systems as well...
     
unityunity