Search Unity

Official Introducing Unity Muse and Unity Sentis, AI-powered creativity

Discussion in 'Announcements' started by LeonhardP, Jun 26, 2023.

  1. ttermeer-reboundcg

    ttermeer-reboundcg

    Joined:
    Jul 12, 2017
    Posts:
    62
    There are lawsuit every day on AI. This is just sensational headline. Even if US rules against it, it will not apply on an international level. LLM are just too widespread at this point - there is so many viable open source LLM.
     
    DragonCoder likes this.
  2. I doubt this. The US is super-strong on copyright laws even around the globe. Pesky trade deals.
     
    atomicjoe likes this.
  3. atomicjoe

    atomicjoe

    Joined:
    Apr 10, 2013
    Posts:
    1,869
    The problem is not AI but the sources to train it. And nobody is going to forbid you from using open source software, but using it on commercial projects can put you in a lot of legal trouble that indie devs don't have the money to face.
     
  4. ttermeer-reboundcg

    ttermeer-reboundcg

    Joined:
    Jul 12, 2017
    Posts:
    62
    And how are you going to prove that AI was used in the development process ? While I can see a point for image generation, AI code assist is no different than browsing internet for answer. For us End-Users, we have no risk. The one in legal trouble are those who train the AI. As indie dev, it is the least of your worries.
     
    DragonCoder likes this.
  5. chilton

    chilton

    Joined:
    May 6, 2008
    Posts:
    564
    So hey this facial animation thing--when do we get access to this?
     
  6. bart_artanim

    bart_artanim

    Joined:
    Jul 18, 2018
    Posts:
    19
    I'm somewhat disappointed this thread has gone silent from the Unity side after some early participation. There are plenty of unresolved questions here that are worth addressing.
     
  7. ThynkTekStudio

    ThynkTekStudio

    Joined:
    Sep 4, 2021
    Posts:
    59
    hi there ,

    been using muse for a couple of days now,

    but i think its generating code with text format errors. i dont know if im allowed to share the code its generating or a screenahot so if someone could take a look at it it would be appreciated

    by the way i didnt know where else to ask for help so i came here
     
  8. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    21,128
  9. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,716
    Yes, but also if a ruling comes out that you can't just scrape random stuff off the internet, or you need to disclose everything you used, etc etc, it means big trouble for all current models.

    Not saying it's super likely to happen, but still.
     
  10. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    21,128
    And if they don't just settle out of court which is very common for class action lawsuits in the US as once again it's largely about the money.

    "Integrated" sounds like lawyer speak. Has it been trained solely off of that data? Or does it include third party data?
     
  11. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,086

    These are precedent setting cases.

    If you think judges don't understand that simple fact, you're making assumptions on the wrong side of history.
     
  12. Demkeys

    Demkeys

    Joined:
    Nov 14, 2014
    Posts:
    32
    Hey I'm a bit confused. Not sure if the info's been mentioned somewhere and I just missed it. How can I get access to Muse (and maybe Sentis)? I signed up for the beta program hoping to be able to test them out but haven't seen or heard anything since then.
     
  13. OmarVector

    OmarVector

    Joined:
    Apr 18, 2018
    Posts:
    130
    My wildest fear is Webgl , webgl is single threaded, do really sentis offer better performance than barrcuda 3.0? , at least in term of garbage collection? ... possible to support GPU inference on webgl?:D

    I've applied for beta, please, someone let me put my hand on it, I will give you juicy feedbacks trust me
     
    Last edited: Jul 25, 2023
  14. tsanyqudsi

    tsanyqudsi

    Joined:
    Jul 17, 2016
    Posts:
    10
    Yea, tbh it's rather dissapointing. It seems like Unity came up with many updates but fail to bring documentation or communications. It's like chaotic jungle out there.

    I used muse using my work account, Muse failed to get things quickly. I need to fine tune my prompts every time. Plus, the longer the chat means the laggy it will be on my PC.

    It's easier to use free version of ChatGPT at this point. Muse also failed to get the knowledge that exists in the documentation. Claiming that it's not even there. Even when I brought the link, it's just saying thanks but still giving the wrong example.

    Unity only said it's close beta, but fail to give the feedback link at the interface and/or giving recurring updates on what to expect on this close beta.
     
    ThynkTekStudio and Unifikation like this.
  15. ThynkTekStudio

    ThynkTekStudio

    Joined:
    Sep 4, 2021
    Posts:
    59


    that is exactly what i was going to say, the lack of a quick way to post feedback and how somewhat out of context some answers seem to be generated
     
  16. OmarVector

    OmarVector

    Joined:
    Apr 18, 2018
    Posts:
    130
    Its not the dev problem, its the management and decisions problem... no one can deny that they have one of best software engineers in the world, better than google if you ask my opinion.... but again, its not the dev decisions to continue dev something or kill a package or any thing that their top management ask them to do
     
    ThynkTekStudio likes this.
  17. tsanyqudsi

    tsanyqudsi

    Joined:
    Jul 17, 2016
    Posts:
    10
    I'm curious why you think I would blame only the devs. Yes, the dev has problems... However, which company doesn't have dev problems?

    That being said, it doesn't change the fact that the documentation and things is quite chaotic.
     
    ThynkTekStudio and Unifikation like this.
  18. Unifikation

    Unifikation

    Joined:
    Jan 4, 2023
    Posts:
    1,086
    I'd be more inclined to believe any claims Unity might make about "AI" when they can get Search to provide some semblance of accurate responses in this forum.
     
  19. skaenz

    skaenz

    Joined:
    Feb 11, 2021
    Posts:
    1
    There is a question. The video shows the creation of 3D animation.
    - Is it possible to create animations for 2D characters?
    - Will it be possible to link AI to work with Spine2D, for example?
     
    DragonCoder likes this.
  20. Caliena

    Caliena

    Joined:
    May 10, 2023
    Posts:
    1
    Can you give us a release date for the closed beta for MUSE? I signed up the week it was announced, and cannot wait.
     
  21. MnogoZnaet

    MnogoZnaet

    Joined:
    Jan 23, 2018
    Posts:
    8
    Is it event possible for a small no-name startup to get just Sentis acces, to migrate from Barracuda ?
     
  22. InquisitorTR

    InquisitorTR

    Joined:
    Nov 28, 2022
    Posts:
    5
    I think it is closed to public right now. When it will open? And, is animation updates valid for 2D or it is just for 3D?
     
  23. alsharefeeee

    alsharefeeee

    Joined:
    Jul 6, 2013
    Posts:
    80
    So the silence on this question from Unity would confirm it was trained on copyrighted data and that using it would mean we won't be able to release on Steam, right?

    Nice.
     
  24. VEU-Developer

    VEU-Developer

    Joined:
    Nov 10, 2021
    Posts:
    6
    I was wondering, in the promotional material for Muse, its advertised as being able to help create animations and materials to be used in engine. It seemed like from that, that you could add Muse directly to your game engine (almost like a traditional package).


    I was wondering is this currently possible, and if not, is this a planned feature for the future?


    Additionally, I wanted to get a more detailed idea of Sentis as well, as its a project that greatly interests me. Is the purpose of this AI Neural Network, primarily to give us the ability to add ChatGPT like NPC's into our projects and games? Also, other than doing something like that, what other potential use-cases could we have for this kind of technology
     
  25. Andy-Touch

    Andy-Touch

    A Moon Shaped Bool Unity Legend

    Joined:
    May 5, 2014
    Posts:
    1,483
    Currently, it is not possible to create animations and materials with Muse. It only answers questions with some google-able info and can generate some basic code/shader code (with varying results of accuracy).
     
    Unifikation likes this.
  26. pokelocos

    pokelocos

    Joined:
    Nov 9, 2015
    Posts:
    54
    Hello, I'm fascinated with the Muse tool as it brings documentation closer to users like me who aren't experts in this field. However, I have a series of comments that could enhance the user experience on the website.

    1. On the web, the image layout has two black fades at the top and bottom of the text area. These elements look nice and are clearly aesthetic, but the issue is that they take up a significant amount of screen pixels, reducing the available space for the chat. This wouldn't be a problem if I had a high-resolution screen, but unfortunately, my screen is old and budget-friendly, so it has low resolution. These elements are more of an annoyance than anything else.

    2. Other pages like Chat-GPT allow you to stop the scrolling of the screen while the text is being generated. This allows for comfortable reading of what has been written while the rest is being typed. This functionality is not available in Muse, so I have to wait for the text to finish typing to read it comfortably or get used to the text occasionally jumping. It's not a drastic issue, but having this solution would improve the overall experience.
    I hope these comments are helpful. If this isn't the right place to share them, please let me know where I should post them. If any Unity Muse web developer happens to see this, I hope this feedback proves useful!
     
    ThynkTekStudio likes this.
  27. Deleted User

    Deleted User

    Guest

    Well hope not that they also abandon this project in few years.
     
  28. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,316
    Any practical example of Sentis, showing for example creatures learning to steal boxes from one another?
     
  29. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,769
    To be honest, I am not too of hope, if seeing ML package not being touched for a while. While the announcement sounds exciting, I am yet to see some exciting ral use case examples myself.
     
    laurentlavigne likes this.
  30. Hikiko66

    Hikiko66

    Joined:
    May 5, 2013
    Posts:
    1,304
    If the US chokes it, then non western countries take the lead on THE money making AI tech of today.
    Difficult to imagine the US letting that happen.
     
    Last edited: Sep 29, 2023
    DragonCoder likes this.
  31. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,769
    UK doesn't care, and puts effort, to stop blocking AI development. No matter what will happen, UK seems that will proceed with various AI tech, while rest of world will be fighting with arbitrary problems of AI existence.
     
  32. Bgustin53

    Bgustin53

    Joined:
    Aug 11, 2022
    Posts:
    1
    I have been using Muse since its inception, but now, all of a sudden I get this message:
    "Join the closed beta program to create with Muse, and accelerate development of your real-time 3D experiences in ways you never imagined! If you were recently accepted from the waitlist and received this message, please note that it may take up to 2 hours to process your account for access. When signing up for the Muse beta, please use the same email address associated with your Unity ID"

    WHY?
     
  33. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,316
    Even not exciting would be nice, to get a sense of cost/benefits without spending time on it. Their GOAP had meaningful examples and could easily be changed to check for scalability.

    I have a feeling that many teams are being jettisoned.
     
    Antypodish likes this.
  34. jimmyd14

    jimmyd14

    Joined:
    Jun 10, 2017
    Posts:
    6
    I changed my account email and now I lost the beta invitation because it's not longer the same email... Any solution to that problem?
     
  35. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,769
    Use your previous email.
     
  36. adamgolden

    adamgolden

    Joined:
    Jun 17, 2019
    Posts:
    1,555
    A promo email today had some interesting stuff,

    muse.jpg

    The videos are:
    Muse Sprite:

    Muse Texture:
     
    DragonCoder likes this.
  37. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,316
    Did you folks train your own model? What did you feed it?
     
    adamgolden likes this.
  38. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,769
    An internet :p
    I would be really surprised, if Unity would go through all Asset Store database effort and try to feed them in.
    I can imagine handling these packages and whats not.
    Where much easier (and controversial) way is to grab internet data.
     
  39. adamgolden

    adamgolden

    Joined:
    Jun 17, 2019
    Posts:
    1,555
    I'm hoping it's ethically sourced. I don't currently have any use for this stuff (as my active/unannounced/commercial project is 100% AI-free, and I'll be busy with that until at least the end of 2024), but in the future I'd definitely like the option to leverage technology like this with a clear conscience.. and related tools being integrated directly within Unity Editor would be ideal. I'm certainly interested [continually] to know what I can and keep myself up to date on progress.
     
    laurentlavigne likes this.
  40. Simon-O

    Simon-O

    Joined:
    Jan 22, 2014
    Posts:
    51
    Un-huh. And how are you planning to monetise this feature?
     
  41. PanthenEye

    PanthenEye

    Joined:
    Oct 14, 2013
    Posts:
    2,063
    Judging by Riccitiello's narrative in the last earnings call, they want to charge by usage. So probably similarly to OpenAI API access, but likely more expensive since this is just GPT 3.5 or 4.0 with some added Unity data on top.

    The image gen will be probably similarly priced per X number of generations.
     
  42. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,316
    Makes sense, I wouldn't use it, my use of generative AI has dropped to zero.
     
  43. PanthenEye

    PanthenEye

    Joined:
    Oct 14, 2013
    Posts:
    2,063
    It's $30+VAT tax:


    Also, there seems to be a behavior tree functionality that's locked behind AI paywall?

     
  44. Andy-Touch

    Andy-Touch

    A Moon Shaped Bool Unity Legend

    Joined:
    May 5, 2014
    Posts:
    1,483
    I use ChatGPT every day to 'pair program' and bounce how to solve some challenges.

    Gave Muse Preview a spin asking the same Qs ive given ChatGPT and its giving me worse answers. Eh, ill keep hounding it for the trial and see how it goes...
     
    Ryiah and Antypodish like this.
  45. PanthenEye

    PanthenEye

    Joined:
    Oct 14, 2013
    Posts:
    2,063
    ChatGPT is also general purpose. I can shoot S*** about anything beyond just programming in Unity and I pay 10+ euro less than Unity is asking. Additionally ChatGPT can also search the web, and offer various additional functionality via plugins such as youtube video summarizer, etc. I can also hook into GPT API directly via IDE plugins and some months I'd be way below what Unity is asking.

    I do pay another $10/month for Midjourney, but it's not really applicable to gamedev besides maybe some concept art. Steam still bans any use of AI unless you can prove dataset ownership, Unity tools fail this check by default since we have no clue from where the data is sourced. Will be interesting to see how Steam reacts to games with Muse generated sprite art still in the final build or at least part of early access builds.

    Overall, I don't see the added value right now but I'll check out the trial unless it's another 12 month contract.
     
    Ryiah and Andy-Touch like this.
  46. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    612
    Looks like it is generally available now: https://unity.com/products/muse
    They sent me a free month for participating in the beta program.

    I don't think the current state of Muse is worth the $30/month. The main features right now (chat and sprite) are slower/worse than competitors. In many instances ChatGPT3.5 is better and always faster.. GPT4 is much better especially for coding. MidJourney also produced better images with much less effort. Muse chat performance feels like what you get from Llama2 7B model that was fine tuned on Unity documentation.

    The Muse sprite feels like Stable Diffusion 1.5 without any fine tuning or additional settings.. it needs to be able to have different model selection. See how Leonardo.ai allows you to select different models for different styles. Anyways running your own Stable Diffusion plus plugging your own LoRA gives much better results, especially for a game since you have much more control over the style similarity between each generated image.

    Lastly Muse chat needs better integration with editor and IDE. See Github copilot chat for an example of great integration. Copilot is integrated in the IDE as auto complete, in-line chat, and also a full chat. This also allows it to have much more context of your code base and doesn't require you to "copy and paste" code to ask questions.

    It's a crowded space and $30 is too much for Muse right now. Copilot is $8.33/m, Midjourney is $10/m, GPTPlus is $20/m (which can generate images and textures too). You can also run Stable Diffusion or a Local LLM for FREE and in many cases get better results than with Muse.. especially when it comes to image generation.

    I hope this surfaces to someone in the Muse team as this product has potential as it has the opportunity to be tightly integrated with Unity. I would post this in the Muse feedback forum but I can't seem to post there and can only read discussions.
     
    Shizola, WolveX and Ryiah like this.
  47. ShaneeNishry

    ShaneeNishry

    Unity Technologies

    Joined:
    Jul 11, 2019
    Posts:
    2
    Hi PanthenEye! I'm the lead dev on Muse Behavior. You can get it for free on the package manager by installing the package by name: com.unity.muse.behavior

    You can use all of the Behavior Tree tools without a subscription, only the generative features are locked behind the subscription. I'd love to see what people think of our tool! :)
     
  48. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,404
    tested 4 texture generations and 1 pbr preview,
    that costs 203 points(!) for very low quality textures..

    also would be quite important to clearly see how much each click costs..
    upload_2023-11-17_15-17-11.png
     
    laurentlavigne and Ryiah like this.
  49. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,404
    Surely that AI has seen Mario images in some training data? (perhaps wikipedia?)

    upload_2023-11-17_15-42-9.png
     
  50. xjjon

    xjjon

    Joined:
    Apr 15, 2016
    Posts:
    612
    I think this is because they only trained the model on images licensed by Unity
     
    DragonCoder likes this.