Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Let's bring machine learning to Unity - Artistic Style Transfer

Discussion in 'Assets and Asset Store' started by MyIsaak, Oct 12, 2018.

  1. MyIsaak

    MyIsaak

    Joined:
    May 23, 2013
    Posts:
    49
    Hello serious Unity devs,

    You've probably used Unity before for creating games, but have you ever considered creating art with machine learning? I wouldn't be surprised if that was a nope.


    There's a new category in the Asset Store labeled 'machine learning'.. but it's lame.


    That's why I am working hard on a Unity plugin that can repaint an image into an art style using pre-trained models per style (for now).

    All computation is done locally on the CPU (GPU support on its way), no need to connect to a cloud provider, and is done parallel in a background thread.



    It would look great as an asset next to all these Google Cloud providers, however, all I need right now is your feedback.
    Leave a like if you think this is cool idea!

    Thanks for reading,
    Isaak
     
    Last edited: Oct 12, 2018
    Gekigengar, 313b, wetcircuit and 4 others like this.
  2. kdgalla

    kdgalla

    Joined:
    Mar 15, 2013
    Posts:
    4,612
    If you could create such a thing, I'd love to play with it.

    Lately I've been really thinking about ways to make the art style in my game more abstract and textural. Even though it's a 3D game, I want it to somehow evoke the tone of gritty urban abstract art (here's a google image search if you don't know what I mean). I'd really be interested in experimenting with your product.
     
  3. mgrugl

    mgrugl

    Joined:
    Mar 1, 2017
    Posts:
    23
    This is a brilliant idea! I am actually currently working on applying fast neural style transfer on a stereoscopic video. Doing it live, ideally rendered for each eye in a VR headset would e.g. allow people to walk inside art-pieces. I hope you can pull this off!
    Edit: I'd also be happy to beta test your asset, or help computing style models. Have two PC's with 1080Ti"s, if that's any help.
     
    Last edited: Nov 3, 2018
  4. MyIsaak

    MyIsaak

    Joined:
    May 23, 2013
    Posts:
    49
    Okay, sure thing, it's just that for now I'm only using CPU with my own implementation but I'm working on integrating an external library (perhaps tensorflow for it's extensive GPU support) in Unity. Then I'm sure your 1080Ti's will fly ;)

    VR + style transfer would be a very sensational experience haha ;), however the specs are expensive, but I believe, in due time, it should be possible if more people invest in neural network research. Machine learning is becoming a buzzword now, with everybody trying to get into 'artificial intelligence' but know nothing about it..
     
  5. FlightyFelon

    FlightyFelon

    Joined:
    Jan 3, 2016
    Posts:
    4
    Hey Mylsaak, I'm working with a studio called Gallium Studios on a game called Proxi, which stars a quirky AI that learns about Humanity though the Memories you share with it.
    Our website: https://www.proxiai.com/

    We're looking into applying Style Transfer technology to our game and are currently looking for artists who can help us achieve the abstract look we're searching for. Even if you're not interested in a job, we'd love to talk to you to learn more about your work with Unity and Style Transfer.
     
  6. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,521
    Its great!
     
  7. jmehrabanz

    jmehrabanz

    Joined:
    Feb 7, 2019
    Posts:
    1
    hey!
    I love your idea its very genius and artistic, how can I test it in a unity project? ;), where Is it available?:)
     
  8. drudiverse

    drudiverse

    Joined:
    May 16, 2013
    Posts:
    218
    Hey mylsaak, well dont for your efforts on style transfer, it's a very fun and interesting project. There is an online version called https://deepart.io/pricing/ which is limited at 500 pixel free and it's AWESOME, although it just shows that a desktop/gpu version would sell like hotcakes if it had unlimited 9megapixel file outputs :D funky!!! well dont good initative best luck to you, you have not been on the site since nov.18... boo sniflly :)
     
    wetcircuit likes this.
  9. lumeneo

    lumeneo

    Joined:
    Mar 3, 2014
    Posts:
    60
    Style transfer would be amazing for my real-time music visualization app. Ideally, I could use the Webcam to create pulsating art in the style of the masters!

    Ping me!
    grig@lumeneo.com
     
  10. SpaceRay

    SpaceRay

    Joined:
    Feb 26, 2014
    Posts:
    455
    Hello, Sorry for the bump of this thread, But I am really very interested and would be really awesome to be able to make AI art, Style transfer, or Machine learning art inside Unity.

    Please is there is any news about your interesting project? Is still in development and you continue working on it?

    I found this thread because I thought also as you that it could be maybe possible to make computer generated art or AI art using Unity and this is why I found this thread.

    It would be really great and I would buy surely an asset that is able to make art using Unity.

    Thanks very much for any possible news or help

    Best regards

    Merry Xmas and happy New year
     
  11. SpaceRay

    SpaceRay

    Joined:
    Feb 26, 2014
    Posts:
    455
  12. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,350