Nvidia Gaugan programmer's art future

Couldn't let @Arowx have all the fun, but in the upscaler thread I touch a bit about inpainting AI, turns out nvidia publicize one recently called GauGan

Even though the tech is at least 3 years old and first appeared with neural doodle:

It's only now fast enough to be real time-ish

 

Everything the do... dude in the colorful shirt said made my skin crawl.

 

Wait till you see that:
https://www.theverge.com/2019/2/1/1...tudio-colorize-ai-artificial-intelligence-art
https://paintschainer.preferred.tech/index_en.html

https://make.girls.moe/#/
https://www.thiswaifudoesnotexist.net/
https://kotaku.com/artificial-intelligence-gets-good-at-creating-anime-gir-1829488226
https://www.animenewsnetwork.com/in...site-can-colorize-your-manga-drawings/.111637

from
https://github.com/deeppomf/DeepLearningAnimePapers

 

What a neat toy.

 

That's what oil painters said when presented with photography then the computer and photoshop, it's technically true.

 

Stop acting like this is even remotely similar to that. That equivalence is so false that you may as well have said that the sky is made of wood and that chicken have a mouth full of teeth.

 

I act like this is already in use in production. Remember the animation spiderverse? It use machine learning to help bridge hand painted and cg animation together (helped with automating the hand traced line). In fact sony want to patent that.

https://www.reddit.com/r/MachineLearning/comments/a698fg/n_sony_machine_learning_animation_patent/

Also I was more prudent 3 years ago when it was still experimental, it's not so much anymore. It's old news now. The whole movement is 5years old by now, and we have already crossed some instance of the well of desillusion (chatbot craze in the tech industry) and startint plateau of productivity.

There is still some inflated expectation people fall into, but anything I'm pointing into is already useful. What people dream is that the machine is fuly replacing an artist. What I'm saying it doesn't need to be perfect to be useful or convinient.

Think about it, skill you had as a oil painter (such composition, color selection, etc...) translate directly into photoshop, photoshop only allow you to work faster, you don't really need undo, layer, etc ... all of this is gadget, but they are handy and save times.

In the Gaugan example, for an artist, what it does is a very quick rough, it also is useful for concept art to visualize idea quickly, and while it's not perfect, it faster than doing anything from scratch and you only have to add the human touch.

In fact neural doodle was created by a guy who actually work in the game industry (Alex J.Champaniard, who worked on killzone and also had the aigamedev site). He is championning those techniques and Ubisoft has embrace them for basing their pipeline on top of similar idea.

So no the equivalence is spot on and you have to demonstrate it is not

 

No, I don't, because the tech does it itself. Photoshop was not viable for digital painting until more than a full decade after it originally launched, and even then there are massive aesthetic differences between it and oil painting anyway. On top of that, this isn't nearly the same anyway because it's just generating content based on extremely loose input with very little direct control.

 

The thing you miss is that i'm thinking about production, not aesthetics. Digital painting (what I mean really with photoshop) had to fight the inertia of traditional workflow, as computer was too new. In the end it didn't win because of aesthetics, but because it was a net improvement in term of productivity, and since that movement was led by newcomer it took a while before they integrate proper art lesson from traditional technique like not using garish color (ie direct from the tube to use traditional painting lingo) that seemed like the signature of digital painting and that traditional painter reduce it to. As someone who use photoshop since version 1 I did live through this whole discussion. We had a similar schism with web design, with traditional page editing seemingly nervous about the fluid nature of the browser windows and cool kids™ leadin the charge of the tech by sheer ignorance of design principle, eventually we did converge toward mix of old and new practice that dictate stuff like responsive design or framework like bootstrap (and his 12 column and gutter grid template).

The new thing is that machine learning is bringing new stuff with the hindsiht of these events, and it show in discussion with modern artist exploring the technique, and the main difference too is that tech, in generam is more mature and accepted, and change is now routine.

So to loop back to this argument: "it's just generating content based on extremely loose input with very little direct control." Is the same concern people had with the indtroduction of filter with photoshop. It seems like that magical thing that did stuff for you, and we had overuse of ready made lens flare halo on image (remember that?). Ultimately it was a non starter, filter, when motivated, save you time, they aren't aesthetics, they are tools. And that's teh gist of it.

This gaugan stuff is a tools, it allow you to get content to a ready state at a faster rate. I compare it to photoshop because you don't need photoshop to make art at all, you don't need gaugan, it just a productivity tool. And while the application shown here is full composition of image, it's probablty going to be the next content aware fill/liquid rescale/tampon, which is use everywhere. Except this time it has some understandng of the semantic of the image and have a global reasoning.

So in some way yes it's different, which is teh point and teh usefulness, photoshop is not aware of content, this is, still, you still have to had input, you tell by "drawing" what you want, you give the mountain shapes, not the neural network, it just adjust the "texturing" smartly and harmonize the whole, which you can easily override anyway, it's just a picture. So it's different but it's the same, it's a way to compose image the way you want, but fast. Just like digital painting was different (you aren't manipulating physical media) but the same (you are composing a picture).

 

He should have reserved that fabric for the contrast fabric only. I like colorfull fabrics for the contrast fabric, but keep the main fabric strict and single colored.

 

No. As said above, it is a toy. Regardless of what the dipstick in the video claims, it won’t turn people into artists, and it won’t be useful for self driving cars. (???). Someone invested tons of developer hours into a toy, and is trying justify their departments budget. The underlying code may be repurposed or used for some practical in the future, but the product is a toy. And a limited one at that.

Sipderverse employed one of the largest amount of animators ever on a single film. It has some neat tech but is the product of artists. It was driven and built by artists. There is zero comparison to the toy in the op.

 

Wait I never said it replace any artist, calm down lol, I never said it will turn people into artist either. Spiderverse did AUGMENT their artist with machine learning, it wouldn't have been possible other wise according to them.

On the diving car stuff, that's what nividia is using it for, to synthesize scene for car, see the real time version of it using semantic map. It's actually just the pair down version of that, repurpose for "artist".

I take no responsability for what the guy said in the video, but anyone following the deeper aspect of the technique for a long time, know it's already there anyway, it's not even about the future, it's raising awareness.

 

Neat.

 

Here is a quote directly from spiderverse's people (from houdini thread):

Emphasis mine

Tools don't replace artist until they actually become socially and culturally aware to participate in the social discourse. We aren't talking about that level of ai now, I hope not

 

These tools are not like the Spider-Verse tools. You're desperately trying to cling to this idea when the only things they share is that both use applications of machine learning, which is an incredibly broad term that encompasses thousands of different ideas.

 

Probably 50 years from now this could have evolved into something useful. But if art is so easy that a dog can bark and the contents of its imagination are beautifully rendered in stunning realism.... then what?

Even if the thing worked flawlessly, it would still have limited use. Maybe concept artist don't have to grind out so much work... but it's in grinding out the work that they learn through trial and error what makes good art good. So in the end I think you'll just see more bad art, while the bar for good art gets higher and higher. In other words, a few really talented people get richer, and everybody else suffers.

I don't know why humans are so obsessed with eliminating work. Life really sucks when you have no work to do.

 

the video said that. Not you. The video shirt doof also said it will be useful for self-driving cars. he is a tool. (or more likely... a business major)

 

So much this. It's like "cloud" was or "AI" or 'procedural'. Super broad categories. Not all entries in those categories are related. Bowling and Free Climbing are both referred to as sports. Other than the fact that they both have special shoes, the are completely unrelated.

 

Exactly. Moreover a good concept artist can do something like very quickly, and with a lot more variety and skill. And they don't have to actually do that many as they have the ability to make the right choices already, they aren't arting via guess and check and not limited by how many trees the ml knows.

--

Besides, if those tools work, it won't make people into artists, it just eliminates the need for people at all. If it can make "anyone" an artist, then no one is an artist. I mean, really. Come on. If that tool can make a nature scene out of simple shapes... having people make those simple shapes is patronizing. I could write a tool fairly quickly that could generate those simple shapes based on minimal information or even just randomly. There is no practical use for that tool except for unskilled people to think they are being "creative". Granted, sometime weird experimental tools turn into something grand (z-brush), but even in the early days of those there was a clear idea of what it could become. This, at best, is destined to be a phone app.

 

It's going to have that "Photoshop filter" effect where people are going to see the little effort put into it very quickly and perceive it as being cheap-looking, regardless of how closely it approximates an art style.

 

I would be a lot more impressed if it took a complex painting of amateur or intermediate quality and made it photorealistic or simply better (and conformed very closely to everything the artist originally designed). Like zombiegorilla said, the artist is doing practically nothing, which means that it's likely the output would be very regular and difficult to control. It looks nifty, but I'm suspicious this is a one-trick algorithm marketed as an artistic tool.

 

The second video in OP is not about photorealism anyway.

But the entire end goal of that tech is to do it 60fps or more from a semantic map (ie the vague input)

Here is the former experiment they branched out to make that api

Also it has understanding of semantic content unlike previous alogorythm. It's also bidirectional, you can generate semantic map from a picture, or generate a picture from a semantic map, and you can do simple operation using those semantic:

This will make vfx cheaper, deepfake is already an application of that. And if they optimize the algorithm at least for HD 30fps, it will simply replace shader.

 

No it won't. Stop arowx posting.

 

I always find it fun to try and break these tools. Figure out with what kind of input they have trouble with, and then try to hone in on that. Sometimes cool stuff gets produced.

 

I can't really agree with that.

The thing that sounds cool to me from the tool chain you described is that, using that approach, rather than creating one image an artist could make a class of images. The demos in the video seem to have one set of inputs and rules, but those are things which, in my mind, the artist should have control over. Make or select input textures, define rules for combining them, set up generators for the shape drawing part. If I had artists working with those tools that's the bit I'd be interested in having them do for me, not just the shape drawing part at the end.

As a concrete example, every "news report" that pops up in a game could have a different background behind it, and an artist would only have to make a handful of configurations to describe each style that we wanted - country, city, seaside.

 

Any scrap of junk laying around in the yard could one day save your butt in a pinch. If you're resourceful.

What I got from ZG's post was that, basically, no matter you are gonna go to work for 8 hours right? Well, if the artist job has been reduced to drawing simple shapes and waiting for the computer to spit something pretty back, that's a pretty lousy job. For me, the joy of art is being in total control. It's not like gambling at all. It's my own personal world, my own personal characters, and I place all the stinking buttons exactly how I want them to be.

Personally, I think technology like this is kind of useless or extremely limited right now -- in the future it could save some menial labor and multiply niche opportunities like angrypenguin described.... but at the end of the day, I am firmly against automation in general. Idle minds are the devils workshop -- and honestly if people cannot cope with the cushy existence we currently have then we are just f'd as a species.

 

IMHO the trajectory of any new (visual) tech is like this:

1. Tech people who don't understand (the process) of art, say it will democratize art and replace artist, by doing same thing artist does but faster and automated.

2. Pionner avant garde artists AND cool kid™ people take hold of the art and push it to breaking point, to do stuff you couldn't do before, and promote the breaking point as the aesthetics of the new "medium", to the dismay of old guard that call it all crap "get ofr my lawn"

3. Hindsight era of maturity, everybody does whatever the tools allow to do, and mix it with old media and process anyway.

It's always fun to see the agitation new tech bring, especially with the hindsight of art history, it happenned when animation happen (gimmicky!), when movie happen (gimmicky!), when video games happen (gimmicky, especially after 84) when photography happen (come on that's not art, gimmicky), when web design happen (fluid layout has no consistency GIMMICKY), when VR happen (oh wait they still struggle too bad ) ...

How else we would have fun during coffee break now let's get back to work

 

I want the 3D version so I can draw a picture in MS paint and have the software pop-out the next Elder Scrolls game. We're not quite there with "make MMO" button yet, but maybe we'll have the art portion down in the next few years. Sure it's not "real" art, but that only matters if you want to make a "real" game.

 

Given the length of some of your posts, you're spending far too much time on coffee breaks.

 

That's the secret I don't drink coffee at all

 

Art Direction Bootcamp: Building a Creative Future with Artificial Intelligence

Naughty dog is leading the charge

 

Coffee is best serve in a cup on the desk. Maybe take a sip and the rest goes cold. Get a new cup and repeat

Edit: and coffee here in the EU comes out of real machines like these

 

Real men chew raw beans into a fine dust then pour scalding water into mouth, gargle, and swallow.

Pffft, europeans

 

You cowards. Doesn't anyone speedball anymore?

 

They all died doing what they loved. God bless em.

 

Now that, is extreme coffee making skills.
That's the tough guys way of making coffee.

 

Haha, reminds me of evil dead when ash kills little versions of himself by drinking boiling water

 

I think this time they might be kinda right and it could strongly and negatively impact artists like myself. I'm fairly certain I will live to see the time where I will get replaced by AI. I don't think anything good is going to come from using AI in artistic fields.

I just hope I can either adapt in time to still make a living from skills that I have, or they introduce a working universal basic income system where I live.

I partly share your fascination for AI related research, but not your optimism.

Thanks for the GDC talk link! I've added it to my list for future viewing.

 

I'm definitely not optimist neither pessimist, paradigm shift tend to produce their own new system of value, we had similar crisis with photography and computer, we will just adapt to a new normal. One potential thing is that art will be even less about execution but about the vision. What you have to say and share will be more important, form of art will be wilder and less formal with a more fragmented niche market, because there would be saturation of well formed automated media.

Also I don't believe in basic income in an automated world, that's a solution for a human labor oriented world, ie it's supposed to give you time to build bargaining power by being able to sustained yourself, it's basically a maintenance allowance for the system to keep labor potential, and increase adaptation and flexibility of labor unit known as human. Ai economy (supposedly an idealized world with perfect robo replacement) bypass all of that, except the notion of social. IMHO an idealized robo world is one with the worse of social media with a world fighting over endless cliques.

 

A full circle back to tribalism, only this time much dumber.

 

We never left tribalism, and we never really understood what it is really (collectively).

 

Have you read the comment thread between Charlos Huante and Scott Eaton under this video? You have to scroll way down:

It's quite possible I got this video from one of your posts in the first place, I can't remember. I'm leaning towards Carlos' point of view that nothing good will come from this.


Can't comment on the second part of your post, I have no grasp on (macro) economics.

 

I think the preview pic for that video is one of the creepiest things I have seen in a long time

 

Reminds me of this one I saw the other day on facebook

 

It definitely made my "adult me" skin crawl, for sure. But then I thought about 12 year old me in the early eighties, and how I was fascinated with Koala pads and CIRCLE (128, 50) and getting more than 4 colors on the screen, and I think wow that's wicked cool, I can't wait!

• Adult me: Everything will kill us.
• 12 year old me: This is sooooo awesome.

Not really sure who to listen to anymore, but I'm beginning to realize I can definitely use a little more '12 year old me' in my life. Think I'll go make a baloney and chips sandwich and some Kool-Aid. It's Summer, after all.

ps. yes indeed, I refuse to spell bologna correctly, unless I am singing the Oscar Mayer commercial song.

 

I still have my Koala pad.

 

For sure... in terms of the tech itself. Though things like Koala pads and Mouse Pens (my fave) were (for lack of a better analogy) 'brushes' whereas this is a sticker book.

Regardless of the product, I was just commenting the massive amount of marketing bullcrap coming out that guy's mouth, he was clearly trying to take something fun and entertaining and make suggest it is the foundation of a coming of the next level technology. (there are others already ahead of this and doing more interesting things in this direction).

If my 12 year old self met this guy, I would suspect he would be offering the most amazing candy and evolution in van technology.

 

I know this isn't exactly the topic announced in teh title, BUt I thought sharing this here would help contain useless thread that are similar in spirit.

By searching ebsynth in youtube you find very insteresting experiment.
The interesting thing is that it's not AI, it's a inspired by style transfer technique born from neural network experiment, but it's actually closer to technique use in mpeg encoding, which lead to glitch art, where the motion data was applyed to the wrong color buffer.

@Martin_H
I read it, but yeah It's an OLD debate, it feels like get off my lawn, I do feel it from time to time, but then I had to remember where I was when I started and video games where not true art and the same thing where told back to me... and my lecture lead me to see it's a sentiment that transcend place and time as I have read a novel where a tea master (and another with a martial art master) in feudal japan expressing the exact same sentiment. We old that's it! lol

Also there is also always a tension between the average, the people who specialized in a very deep skillset, the people who have a deep visual culture, and the experiment. Those are tribes of artist who value very different things.





Basically the discussion you see is typical of a traditionalist argument (in the above sense, focusing on skills and sensibility) vs a revolution argument (more people can express their world through democratization, new form of art and doing art)

 

Thanks for the explanation! I had half seen that video, but didn't know anything about the approach the implementation took.Mpeg makes sense, now that you say it. Kinda looks like the datamosh stuff too at the points where it fails.

I really really hope you're right about this one, but I also really think you aren't, and it's gonna be different this time .
I recently talked to someone who specifically studied evolutionary and adaptive systems already a decade ago and left the AI research field completely because of ethical concerns. When I see people who released research papers drop everything and walk away from it because they don't want to be a part of the change the tech is gonna bring, and they tell me "Yeah, the part of your job where you just create artworks based on client input, will eventually be able to be done by computers" (which is the bread and butter part of my and most creative freelance jobs), you'll probably understand when that has more impact on me than anyone else telling me "don't worry it's gonna be fine" :-/

I'd rather be wrong than right on this one, but if I'm right, I at least wanna be a bit prepared and see it coming soon enough.

 

Oh i'm definitely not think it will be fine for the people, at least those who don't adapt and define the new form. One reason why I share this here is because I see it becoming something to wrestle in the future, so you are warned, you can try to find these new form.

Its' like when photography came and now all the portrait painter got out of jobs, while there is still portrait painter, it's not the prestigious job it used to be. But we still use the skills, it's the same to do the concept art and the head uv textures of high def character, except with no oil painting. Those job are still lucrative. And they had to be explored and created too.

The painters that got famous where the one who redefine what it mean to paint. A lot can be say about cubism, as it was trying to represent object from all angle, but they are basically precursor in spirit of the UV map.

Also more people can afford to experience the skills of portrait painter than ever, and even try it on oil painting, it's more of a hobby though, not a lucrative work.

When I was a child I remember CG reception before toy story, people told you cg can't have any warmth, as most visual where driven by tech and academics with no training (color straight out of the tube and abused of shiny specular), it wasn't up until artist actually applied their skills we started to see good stuff and new job. Now it's everywhere.

Start using the tools and define the job, the core of being an artist will still say the same. Artistic literacy will still be a currency. It's up to us to define the use case.