Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Text2Mesh is already technically here in 2022 using AI

Discussion in 'General Discussion' started by cyanspark, Oct 22, 2022.

Thread Status:
Not open for further replies.
  1. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    DreamFusion can render 3D video creatures from a 2D image. Currently it takes 20 minutes on PC to have a 3D video that can mesh. When the process has been optimized in 12 months, it will take only 5 minutes to generate a colored 3d mesh.

    The app would need a collection of images or prompts that you want to mesh, leave it running all night, and the PC would greet you with 80 meshes when you wake up.

    I thought that it would use point-cloud/poisson for meshing, although the google reasearchers already have marching-cubes mesh exports from prompts on the project page.

    The race is on to do the first text2mesh on a home PC using stabilityAI, we could see it happening this week.
     
    Last edited: Oct 22, 2022
    DragonCoder likes this.
  2. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    5,533
    We've had that discussion already regarding stable diffusion.

    I wonder how you come up with these numbers? In 12 months, in 5 minutes, this week.
    Btw you mean next week, right? Since we're on a weekend and AI doesn't work on sundays.

    I'd say, give it 5 to 12 more years and then it might have some limited application in games.
     
  3. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    Literally every demonstration I've seen of this tech has generated models with topology so terrible that I wouldn't even use them for LOD3. On top of that, there's already growing sentiments among artists (both 2D and 3D) around AI art because the training data is taken from their work without any licensing whatsoever, which is absolutely going to lead to legal battles down the line. This will probably start from the top, with major corporations like Disney/Sony taking legal action for how training data is used.

    Placing this tech as the Next Big Thing is foolish both technically and legally.
     
    MadeFromPolygons likes this.
  4. CodeSmile

    CodeSmile

    Joined:
    Apr 10, 2014
    Posts:
    5,533
    But ... what if you were to combine THAT with NFT? :eek:

    upload_2022-10-22_20-22-55.png
     
    ippdev and Kiwasi like this.
  5. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    I'd argue that's dramatically less ugly than the vast majority of NFTs out there.
     
  6. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    It's about Dream-Fusion saving .obj wavefront files, not SD:
    https://youtu.be/dIgDbBTztUM?t=306
     
  7. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    For a 20 day old github release I think those meshes are pretty good!?
    dreamfusion 3D mesh.jpg
    Do you think the above models can be copyright challenged? I don't think so. They are from google's own libraries.... you'd be taking google to court.

    For SD, they have not broken any laws : it's legal to use any images you want in an AI system.

    The laws would have to be invented after the act of AI-training occurred. Those laws would be challenged by google and the AI community.

    As a digital artist I would save a lot of time by using the above mesh as a starting point, it increases work-rate because it saves time to dedicate to refinements.
     
    Last edited: Oct 24, 2022
    DragonCoder likes this.
  8. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    This topology is nightmarish and there's no practical use for any of these.

    Uh citation F***ing needed, my dude. This is the equivalent of saying "if I found it on Google Image Search it's free to use"
     
    MadeFromPolygons likes this.
  9. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    Erm, no. Counter-citation F***ing needed if you come up with a citation of your own, my dude, which you didn't. Can't expect me to fetch references while you don't.

    Good luck bringing a court case for the frog in a sweater mesh! :)

    Who claims copyright to Dall-E? nobody. nobody has yet tried to claim copyright ownership of an AI generated media, why not? because they can't, else they can copyright your brain for having seen the images too, and using them as inspiration.
     
    DragonCoder likes this.
  10. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    You are literally the one making the claim that images, under copyright, can be freely used without being licensed. You're the one who has to back that up.

    Because it's barely been around for a year and you know full well that there is a difference between a person observing and interpreting art and a box you throw all the art you can grab into to produce new outputs.
     
  11. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    It's futuristic mesh generation technology, it's a technology post, nothing personal.

    You claimed the technology can perhaps be copyrighted, that it's images can't be copied.

    There are no documented legal challenges to diffusion AI, has even a single lawyer complained?

    Sure, corporate media have run scare-stories about "dubious legality", without references. Factual websites documenting these technologies cite "ethical concerns".
     
  12. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    No, I'm claiming the training data can be copyrighted.

    AI art? Not yet, but there's already multiple github copilot lawsuits in the works.

    https://visualstudiomagazine.com/articles/2022/10/21/copilot-suit.aspx?m=1

    It's not just "corporate media" but individual artists like Simon Stålenhag and Greg Rutkowski. If you were actually paying attention to any of the discussion around AI art, AI art provenance, and the use of artist works in training data, you'd understand this already.
     
  13. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    This is a technology thread. If you want to talk about the legality of a specific implementation of the technology, perhaps we can start a thread about it, rather than hijacking this thread with tangential references to copilot which is text-copy-paste technology.

    If the artists you mention mount a legal lawsuit against SD, Dall-E, Google, which they haven't, then it will be even less strong than Napster 2.0, for which there was clear evidence and legal precedent. No amount of legal challenges let Britney Spears/EMI prevent the mp3 format's popularity.

    Don't you feel like you are fighting the tide for the uncontestable future of technology: AI. It's a fight against a tide of technology involving 200 million+ users. Your legal futurology topic deserves a thread that perhaps you can start?
     
  14. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    Legal provenance and technology are directly interconnected in the modern era and you're also ignoring that I did bring up technological points. Those meshes have terrible topology and polycounts to the point where they wouldn't be useful in any project, you posited and improbable future where they tech just magically works when AI art can't even put the right number of thumbs on a hand in 2D, and I even specified there'd be legal challenges down the line, something I backed up with the link about that same thing being applied to Github Copilot.

    But since you're deliberately sticking your fingers in your ears and going "NA NA NA NA NA" at the slightest challenge it's pretty clear none of this matters to you.
     
  15. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    A technology has let millions of users to create billions of images which are, according to you, illegal on their computers and on the web?

    You aren't interested in the technology. You seem to hate it. I could take a picture of my hand and 3D it and that's a problem now. You write expletives first, references second: Copilot is a copy-paste field for a different AI technology that involves no images or meshes.

    You said it was very bad quality, so I posted images that look very nice, especially for a 20 day old code release, it's fantastic quality.

    AI imagery has 100'ds of millions of users in 5 months of existence, I'm wrong to suggest it's the next big thing?
     
    Last edited: Oct 24, 2022
  16. MadeFromPolygons

    MadeFromPolygons

    Joined:
    Oct 5, 2013
    Posts:
    3,967
    Actually you are kinda putting words in their mouth and ignoring/misunderstanding what they say. They did not say the quality is bad, but that the topology is bad. Anybody with even a tiny bit of understanding of topology and 3d meshes can see from the picture you posted that the topology is awful, like even worse than VR sculting apps outputs. Making a nice looking model, and making a usable model with proper topology are 2 very different things. Do not conflate one with the other - as you say this is a technology thread so lets actually look at the technology instead of "the picture looks nice so it must be good" type approach you seem to be employing in your "arguments".

    Because you think it "looks nice" doesnt stop that from being a fact that the topology is terrible.

    Second, everything said about the potential for this tech to be considered illegal or heavily restricted by laws in the near future are also true. Stating this is not "hating technology". Its just paying attention to the news surrounding this technology.

    And finally this is a forum, do not post on here if you are not willing to have discussions about things. Just because you dont like someones points doesnt mean you can shut down the conversation, it doesnt work that way. If thats what you want go on facebook messenger or a chatroom.
     
    Ryiah, stain2319 and angrypenguin like this.
  17. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,754
    I am curious, why topology is bad?
    Is anybody have seen/opened these models?
     
  18. MadeFromPolygons

    MadeFromPolygons

    Joined:
    Oct 5, 2013
    Posts:
    3,967
    I have, and yes they are bad. There are lots of people on twitter using these with animations and pointing out the wide variety of artifacts that are visible, due to bad topology.

    This is not to say the tech is not impressive, its just a long way from usable meshes for realtime 3D purposes unless the creator really does not care at all about performannce, memory and disk usage.

    Obviously tech like Nanite in UE5 would make the performance etc issues go away, but the artifacts from bad topology will remain.

    Right now to make something properly usable a trained 3d artist has to retopologise the model - so its not all that useful right now in production / within industry except for rough concepting.
     
    Ryiah, angrypenguin and Antypodish like this.
  19. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    Beyond that, you can tell just by looking at the silhouette in known problem areas. Most notably is the way the edges at the bottom of the ghost eating a cheeseburger are crimping and the way the texture and lighting applies. We can also take this and look at other implementations. For instance, here's some models Nvidia showed of their own tech:



    Looking at them untextured like this, it's easy to see the amount of distortion that happens because it can't properly identify edges or surfaces relative to the objects themselves. These models are tens of thousands of polygons when they should be maybe a couple thousand tops, but because the topology is handled so poorly and because it just doesn't understand the subject, they're a mess.

    These are not viable tools.
     
    Ryiah and MadeFromPolygons like this.
  20. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    He is dismissing the fact that it is a 20 days old code release, without upscaling, processing, using low res marching cubes.

    Mesh is synonymous with topology, and he didn't post images of the mesh for considaration. Those marching-cubes artefacts have a very low resolution. You can count how many cubes are used, it looks like it's only 128x128 grid. That's fabulous for a prompt-based 3D shape using 128x128 marching cubes.

    Of course, computers have no problem meshing 1024x1024 topologies, so it will only be a matter of time until the resolution is 900% finer.

    Marching cubes is known to be a sub-optimal process for meshing scans, point cloud poisson process is far less aliased.

    This technology is going to upgrade to 1024 images, the videos will be upscaled to 4k, It's going to spawn fan sites like lexica.ai with millions of 4k meshes mapped properly using point clouds and post-processing.

    It's already acceptable quality to add crickets, bugs, snails, rocks, objects on tables, flowerpots, frying pans, apples, onions, bottles, teapots, and thousands of other thing, at least there will be free galleries of these things in 1-2 years.
     
    Last edited: Oct 24, 2022
  21. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    Heavens to mugitroid. Bad topology? I can fix that in ZBrush in 5 minutes. Pretending this is an end to end solution? Nope. It will be a part of some folks toolchain. Boogie man lawsuits? Drop all the DRM lawyers in the ocean. It will fix glow-bull warming from the massive reduction in hot air. Training data from public sources? Most would be glad to contribute. Training data stolen from someone who hand assembled their sets and licensed them under non-open source? Whole other can of worms and not specific to this discussion. WETA or ILM running their libraries thru this AI and releasing the results. Galactic!
     
    cyanspark likes this.
  22. UhOhItsMoving

    UhOhItsMoving

    Joined:
    May 25, 2022
    Posts:
    104
    According to the U.S. Copyright Office, an article from The Northwestern Journal of Technology and Intellectual Property, this OpenAI docket, and other places, using copyrighted works as AI training data falls under fair use.
    Copyright doesn't protect ideas, so trying to copyright a prompt isn't possible because it's just an idea (or a list of contents), and, since the work needs to be created by a human author, trying to copyright the work isn't possible because it isn't authored by the user.

    Honestly, that's not really a problem. Not being able to copyright AI generated works doesn't mean you can't use them, it just means the usage rights of them are not exclusive to you. You still have copyright ownership of the works you make with them, so unless you're just trying to display or sell it as is, you shouldn't need to worry about copyright.
     
    Last edited: Oct 25, 2022
    cyanspark likes this.
  23. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    None of the things you posted say this. The first link you posted that actually says anything says that courts are currently divided on the issue and that no ruling has been officially made.

    The second thing you posted was an OpenAI docket which is a submission by a group that has a vested interest in making it fair use.
     
  24. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,614
    Even aside from that, there's rather a lot of misunderstanding / wishful thinking going on in here.

    You don't "try to copyright" something, it is an automatic right. And the risk in question here isn't that someone will copyright what the AI made*. The risk is that the AI was trained on some dataset, the items in that dataset were indeed made humans, who usually either hold the copyright or sold it to some organisation, and the AI then outputs stuff based on this data that could possibly be considered "derived works".

    Indeed, @cyanspark referred to Copilot as "text-copy-paste technology". This is acknowledging that the inputs there come from somewhere and are being copied to end up where they're being used. Somewhere within the magic black boxes the exact same thing is happening with AI generated visual content. The only difference is that it's less obvious** to human observers.

    The whole thing comes down to arguing the difference between "learning" and "copying". And while that might be a fun debate to have, it doesn't matter what you or I think the answer is. Our opinions won't change anything. It's up to what gets decided when people test this in court. Until then everything is speculation.

    To be clear, I'm not denying that the tech is cool. To the contrary, it's very cool. But if you've ever professionally created creative works then you know how important licensing is, and that messing it up (link) can be bad, and that for relevant stuff it's pretty important to track "chain of title" (i.e. who is allowed to do what, with what). This AI stuff absolutely wrecks anyone's ability to track chain of title, because we've got no idea what something may be derived from or who might knock on our door (link) as a result.

    * Though that's certainly a risk elsewhere.

    **Except when it isn't, such as when Shutterstock watermarks were recreated in AI-generated images. Care to guess where they got their training data? Note that licensed versions of those images aren't watermarked. Also note that even when licensed you're usually not allowed to let other people make their own derivations.
     
    MadeFromPolygons, Moonjump and Ryiah like this.
  25. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    There is, in fact, an ongoing discussion when it comes to AI art and AI code generation that is about whether or not this just amounts to a twist on license laundering for this very reason.
     
  26. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    Do you want to land from your cloud and tell me: How many years will it take for me to be prosecuted? If I use a million AI frogs and squirrels in sweaters in a game?

    The law is caught behind the technology by about 10-20 years. Every time it catches up the technology will be 10-20 years ahead. AI tech industry by then is worth 20 billion, counter claiming indefinitely, 50 different AI models will be available, completely untraceable art, lack of specificity, lack of traceability, prior no-copyright status under US copyright law.

    Do you think I will end up living in a barrel?

    Until OpenAI+Google VS ArtistsLobby goes to Artist's lobby (i.e. 2033) I wouldn't be liable for anything because of the US copyright office ruling that AI art has no human owner or copyright. By then my game would be abandonware.

    If the artist lobbies won against Google and a judge ruled that some AI neurons were entirely owned by the artists because they neuronified their work, they would also have to reverse the US copyright ruling retrospectively for the previous 10-20 years, trace untraceable artwork and claim it, it's as realistic as controlling the stock exchange.

    Sure there are ethical concerns, discussions, and nearly everyone loves art technology, and your warnings sound totally incompatible with technological reality.
     
  27. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    Keep moving those goalposts, buddy. Also, it doesn't have to just be prosecuted, one of the primary things that will enforce copyright laws will be lawsuits, which is already in the works with Github Copilot, as already explained. You are basically going "well it's dubiously legal right now which means it's fine," which is foolish. Angrypenguin already pointed out how keeping track of proper licensing in commercial projects especially is a big deal, but you're just ignoring that.

    You're saying this stuff is incompatible with technological reality, but the only reality here is that you've not actually done your homework on the discussions here or legal aspects and precedent outside of AI license laundering.
     
  28. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57

    No lawsuits whatsoever regarding AI art. Unrelated reference. Art made by a computer can't be copyright. So you change the subject to unrelated technology which has nothing to do with art and mesh.

    Ultimately, the only thing that artists can do is request reference to their site when someone uses their name to generate art, and request payment from any companies that make money from art while quoting their name. Unfortunately, that will take at least 12-15 years to persue legally.
     
  29. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    It's not an unrelated reference, it's fundamentally the same premise and artists are looking into these things because their livelihoods are threatened by it. That you're desperate to ignore this is on you, not anyone else.
     
  30. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    AAA games companies contain lots of mesh which has been created by tracing over Pixar and Disney photos, it's perfectly legal. When they are on the internet browsing for a photo to copy into mesh, you won't be standing over their shoulder saying "Oooh that's illegal you'll get sued, that's not fair use it's copying Disneyland! ummm your not allowed". Only if you live in legal "lala land". If they copy lines of code from github it's illegal, not that anyone is fussed.
     
  31. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    You're literally just making S*** up now.
     
    MadeFromPolygons likes this.
  32. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    Argument lost. You can menace this guy too for his copying:


    Contact the asset store artists who have modelled Ferraris and m-16's and tell them: "There's a high chance that you traced that mesh from a google image result that you copied without consent, you will get sued"
     
    DragonCoder likes this.
  33. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,614
    There are indeed limits to what can and can't be done with models of F16s and Ferraris and other things where someone else owns the design. This is why Grand Theft Auto uses made up car brands with look-alike designs, where racing simulators sometimes make a big deal about having "officially licensed vehicles".

    As for content store artists selling stuff they shouldn't... yeah. See my earlier link re: 7 Days To Die and the trouble that got them in.

    Under some circumstances, and not under others. Knowing the difference is pretty important.

    Then they need to learn about the difference between "copying" and "referencing".

    Among other things, this shows a clear misunderstanding of what is being discussed.

    If we could see the future then nobody would need seatbelts. ;)

    Good day / evening to you all. :)
     
    MadeFromPolygons and Ryiah like this.
  34. Moonjump

    Moonjump

    Joined:
    Apr 15, 2010
    Posts:
    2,572
    The only AAA games companies tracing over Pixar and Disney photos are those with licences to do so. And there are people checking if it is possible to get sued because something is too close to someone else's IP. I have seen it happen first hand at games companies I have worked at, and most projects involve discussions to avoid it happening from the beginning.
     
    Ryiah likes this.
  35. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    Keep moving them goalposts buddy. We are talking AI art, not code.
     
  36. UhOhItsMoving

    UhOhItsMoving

    Joined:
    May 25, 2022
    Posts:
    104
    It's not explicitly stated, nor is there an official ruling, but based on the actual rules of what constitutes fair use, how courts actually decide on the matter currently, and how courts have actually decided on the matter in the past, if someone was to take you to court for using their work as training data, the court is likely to find it is fair use.

    Also, your first quote just states the main issue and doesn't actually answer the question, and your second quote conveniently (& shamelessly) cuts the answer short. Here's the entire paragraph:
    Only for works that are "independently created by a human author," so yes, people are "trying" to copyright AI generated art (or, as worded in this case, "seeking to register [a] computer-generated work as a work-for-hire").

    It could be considered transformative, though. A work can be considered transformative if it "uses the source work in completely new or unexpected ways" even when "all four of the statutory factors would traditionally weigh against fair use." Parody is a well known type of transformative use.

    When it comes to AI reproducing near replicas of specific images, its ability to do that is not based simply on if that image was used, but how many times it was used, or in other words, how many duplicates there were. It learns patterns, so the more times it sees the same pattern, the more it learns it, and the more it can accurately recreate it. For example, in my post where I showed that you can reproduce images good enough to recognize the sources, the Call of Duty logo was nearly an exact copy. If you look at the dataset, you'll see the logo appears many, many, many, times! So, if every training image is only used once (which would be the case in a manually acquired dataset), reproducing any of those images may not actually be possible.

    For the LAION-5B dataset, because it was acquired by crawling the public web and is "uncurated," there's going to be more duplicates of well known things and less duplicates of less known things. So the concern that it will reproduce, for example, an ArtStation artist's works is very unlikely compared to well known or famous works (e.g. The Mona Lisa). You can test it by simply using the actual title of the work as the prompt. For example, the prompt "The Mona Lisa" produces accurate recreations of the Mona Lisa instantly. If an image can't be reproduced using the actual text it was paired with, how could it be with any other text?

    Actually, come to think of it, whose to say it won't reproduce an existing image that it wasn't trained on? Just because it can generate images not in its training data doesn't mean that those images don't already exist. Is a generated image that's the same as an existing image not a problem if the existing image wasn't in the training data?

    Speaking of tracing over something, what do guys think about this, this, and this? Would these be infringing on the rights of the owners of the source images?
     
    Last edited: Oct 26, 2022
  37. Murgilod

    Murgilod

    Joined:
    Nov 12, 2013
    Posts:
    10,083
    I cut it short because not only is the rest speculation, but uses information from a completely different context. The use in a search function with limited viewable results is transformative both in output and use and had different market use. Does using an AI art generator for concept art that is trained on other pieces of concept art fall under that? That's highly debatable. It's especially debatable when you consider that this directly affects the ability of existing concept artists to find work, especially when it's trained on that work. Concept art is just a single example, as this can also extent to things like illustration and, depending on how things end up going, meshes and technical diagrams.

    No. People are trying to enforce copyright on their own personally created works that are being used for training data.

     
    Martin_H likes this.
  38. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    This is illegal today:https://www.deviantart.com/shop/search?q=mario So hop on over there and threaten them. ... how come this mathematical modelling thread has been ruined by obnoxious dream talk about suing for mesh of frogs in sweaters. Nobody owns frogs in sweaters.

    For AAA games, please tell me why "Is copying a photo to a mesh" is asked zero times on the billions of pages of the web? Can you find a single human, games creator, forum post discussing "legality of copying a mesh from an image" I can't find a single person who gives a hoot about that form of "mesh-tracing-licencing-scam" or whatever a copyright-ultra-fascist would call it. Please find me a web article on the subject, it seems that nobody has any legal concern with it at all.

    Guaranteed that mesh happens in every AAA game with photo-realistic objects: a seagull from national geographic, a car from a journalism photo, AK47, tank, rabbit, because that is not "breach of copyright" because it is generic, honest art process using photos. That's the whole point in AI, to understand the generic graphical equivalent of words.
     
    Last edited: Oct 27, 2022
  39. spiney199

    spiney199

    Joined:
    Feb 11, 2021
    Posts:
    7,610
    Something like an AK-47 isn't 'generic'. AAA studios will be paying license fees to use their likeness. This is true for vehicle and other things where the rights are held by a particular company. It a company holds the right to it, above board studios will be getting rights to use it first.

    Big studios regularly get called out for using assets they don't have the appropriate rights to. Off the top of my head I remember it stinging Mass Effect 3 a bit for using a few stock photos they didn't buy rights to.

    And just because people are doing and getting away with it doesn't make it legal either. Part of maintaining copyright is enforcing it, and legal teams will generally go after big players rather than random small fries no one's heard of.
     
    Moonjump likes this.
  40. cyanspark

    cyanspark

    Joined:
    Mar 12, 2022
    Posts:
    57
    Do you realize you are encourage people to serve Unity a cease and desist infringement notice? You are all being rediculous:
    https://assetstore.unity.com/packages/3d/props/guns/ak47-206704
    a frog in a sweater is copyright too? so is a seagull?

    You can sue shapeways too: https://www.shapeways.com/product/G3LE7JFYC/grogu-star-wars-legion-baby-yoda-the-asset

    you can sue artstation:
    https://www.artstation.com/marketplace/p/2dyqp/baby-yoda

    shutterstock:
    https://www.shutterstock.com/image-photo/sumperk-dec-19-2009-scuderia-ferrari-265507190

    These are big companies with industry best practices, they could not give a F____ about what Murgilod is saying.

    The frog in a sweater? Are you Biff 2015?
     
    Last edited: Oct 27, 2022
  41. spiney199

    spiney199

    Joined:
    Feb 11, 2021
    Posts:
    7,610
    I think you're the making this into an argy bargy when it doesn't need to be.

    I don't think you understand how the law works either. Currently, Unity/Shapeways/Art Station aren't at fault for users posting copyright material on their platforms, but can choose to enforce it, usually by giving actual IP holders a means to file their grievances.

    The individual users can of course be targeted by said IP holders, but like I mentioned, they will seldom go after small players as it's not worth the resources.

    It's a losing battle, of course, given the prevalence of the internet. But it doesn't change how the law works.

    This is all beside the point as to the complex legal melodrama associated with this "AI" stuff.
     
  42. Moonjump

    Moonjump

    Joined:
    Apr 15, 2010
    Posts:
    2,572
    The AK47 asset is something that nobody should use. It is an item designed and made by a company, they have IP rights that are being infringed. You may get away with using it if the game is obscure because it doesn't sell, but who wants to make games that deliberately don't sell. Weapons companies do protect their IP (a company I was working at ended up paying out because of a weapon in a game after they were pursued by the IP holder's legal team, and had to remove it from the game to avoid bigger trouble), but go after those first who are easier to spot because the games are doing well.

    A frog in a sweater or a seagull is not copyright unless copying someone else's creation of them, which would be hard to enforce unless stylised in some way.

    Those are big companies, but they are not creating those assets. Not cracking down on IP infringing assets on their marketplaces may become an issue, but it is more likely to be a problem for the infringing creators.

    Note also that the Ferrari image you linked to is for editorial use only, so it couldn't be used in a game for example. That store at least is doing something to protect IP.
     
  43. UhOhItsMoving

    UhOhItsMoving

    Joined:
    May 25, 2022
    Posts:
    104
    I decided to try and fill in the factors of fair use without including any thoughts, opinions, arguments, or conclusions. These don't respond to anyone in the thread, they just fill in the fair use factors with factual information, and are true both in and out of the context of fair use. This is only for the use of copyrighted images as training data, not anything about using the generated works.

    1. The purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes:
    An image is a visual representation of something that is used for viewing, and an algorithm is a set of instructions that is used for solving a problem.

    An AI model is an algorithm that can perform "complex tasks in a way that is similar to how humans solve problems." A machine learning model is a type of AI model that learns to perform tasks by training on data. The input of the training process is a set of data that is that represents the goal of a certain task, and the output of the training process is an algorithm that can perform that task to achieve that goal.

    For Stable Diffusion, the input of the training process is a set of image-text pairs, and the output of the training process is an algorithm that generates images from text. The output of using the images to train a model is not another image, but an algorithm. The images are not used anywhere else outside of the training process. (In fact, actually seeing the images is not even necessary for the training process to work, as the images are only used for training the model, not for actual viewing.)

    The Stable Diffusion model is freely & publicly available. Its use is for "research purposes only," and using it to share "copyrighted or licensed material" or "content that is an alteration of copyrighted or licensed material" is not allowed, as well as other misuses. In addition to that, the license specifies that, while "CompVis claim no rights on the outputs [users] generate" and that "[users] are free to use them," users are also "accountable for their use which must not go against the provisions set in the license." And, while users are also allowed to redistribute the model, "[users] have to include the same use restrictions as the ones in the license and share a copy of the CreativeML OpenRAIL-M to all [their] users."
    2. The nature of the copyrighted work:
    The LAION-2B (en) dataset (a subset of the LAION-5B dataset that Stable Diffusion was trained on), contains 2.32 billion "high-quality image-text pairs" in the English language that were "crawled from publically available internet."

    Some types of images in the dataset are (but are not limited to) photographs, screenshots, diagrams, illustrations, paintings, portraits, posters, propaganda, advertisements, and signs. They range from factual images (like photographs of places or diagrams of objects) to creative images (like illustrations of characters or posters for movies), and anything in between (like illustrations of historical events or art made in the style of an era).
    3. The amount and substantiality of the portion used in relation to the copyrighted work as a whole:
    An image can be used in its entirety. However, in relation to the entire dataset, its contribution can be extremely small. For example, if all 2.32 billion images from the LAION-2B (en) dataset were used as training data, a single image (with no duplicates) would make up approximately 0.000000043% of the training data. If a person had 1000 of their images in the dataset, the images would make up approximately 0.000043% of the training data.

    In addition, the images are not used anywhere else outside of the training process. The model's ability to reproduce training images is caused by duplicates in the dataset. For example, with Stable Diffusion, "no additional measures were used to deduplicate the dataset. As a result, we observe some degree of memorization for images that are duplicated in the training data."
    4. The effect of the use upon the potential market for or value of the copyrighted work:
    An image is a visual representation of something that is used for viewing, and an algorithm is a set of instructions that is used for solving a problem.

    The output of using an image to train a model is not another image, but an algorithm (a machine learning model). Because an image is only used to train a model, and the output of that training is an algorithm, not another image, the use of that image to train a model does not add another image to the world.

    Even when not trained on a specific image, the model can still generate images. As well, a model that was trained only on a dataset of images that were supplied by the owner, within the public domain, or made by the AI developer themselves can still generate images. Even an algorithm made by hand can generate images (ex. Procedural Swords, Procedural Staffs, Pixel Bow Generator).
    Additional:
    Machine learning models can only generate what they've been trained to generate. Based on the data they're trained on, models can acquire a bias and generate biased outputs. For Stable Diffusion:
    In addition to that, the data also needs to be high-quality in order to get a high-quality output (and the LAION-5B dataset describes itself as "high-quality").

    Also, I would recommend reading the OpenAI docket, as it does actually make things clearer.
     
    Last edited: Oct 27, 2022
  44. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    Some gunsmith's make miniature replica's of trademarked firearms designs. They do not license the design. It is not a duplicate nor removes the original's market share. I think the issue with games using brand names is that they do not want their autos associated with the pure mayhem criminal clownage that games like GTA inspire in their gameplay. It don't look good on their brand.

    That being said. How is someone going to prove that those 4200 pixels out of several million is killing their brand, defaming them, stealing their work outright, libeling them or harm,ing them or their business? Impossible. They would have to digitally prove via forensics that the vectors assigned to the training data arose only from their unique dataset amongst the multitudinous others. Now, some folks will argue with a tree stump and most any lawyer will file a brief to make that mortgage payment. That don't mean they have a leg to stand on in a civil court, and it most certainly is not criminal matter so the "legality" of it is moot.
     
  45. Moonjump

    Moonjump

    Joined:
    Apr 15, 2010
    Posts:
    2,572
    In most countries it would come under civil law, not criminal, but it is still law, so legality is relevant.

    Whatever the IP, be it copyright, trade mark, registered/unregistered design, or patent, the legal description often includes wording about work not used in a way that negatively affects the reputation of the author. A legal case won't need to identify which pixels, just that the IP is being used, and if damages for defamation are included, how it harms the reputation of the author.
     
  46. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    The amount of forensics needed to derive the fact that some vectors in a particular configuration and filtered by many exotic mathematical techniques were responsible for a ripoff/defamation/infringement would allow any defense lawyer to claim undue burden in meeting the standard of evidence required by the complainant. The judge would consider the courts time being used frivolously. That is from someone who prepared litigation documents for court for over a decade.
     
  47. Moonjump

    Moonjump

    Joined:
    Apr 15, 2010
    Posts:
    2,572
    That may be true in USA, but the laws around IP vary worldwide (USA is notably different to most other countries in some aspects of IP law), and why different opinions are shown here. Inclusion in the data set will mean the burden is on proving the data was not used in some jurisdictions, or even just the presence is sufficient.
     
  48. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    Good luck with that. If I got a summons from a Euro, African or Asian country to appear in their courts I would laugh, make a paper airplane out of the summons, set the tip on fire and see how far it flew off the rear balcony.
     
  49. Moonjump

    Moonjump

    Joined:
    Apr 15, 2010
    Posts:
    2,572
    People need to follow the laws of where they are trading. It can get very messy when not doing so. Most games are sold in many countries.
     
  50. ippdev

    ippdev

    Joined:
    Feb 7, 2010
    Posts:
    3,850
    This technology is not game centric. There is no law against using AI to generate assets so this is just pure bogey-man-ism. For conjecture let's say Belgium wanted to haul my ass to court because i used AI to generate some asset that got someone's short and curlies in a knot. Good luck Belgium. I will crumple your summons up and toss it. Absolutely no jurisdiction over me. They won't get China to remove my hypothetical game that has some asset that tweaked some lawfare freak into attempting to scare me with an offishul looking document.
     
Thread Status:
Not open for further replies.