Showcase Aify - Text to Image Utility [Editor Extension]

This looks great! Thank you for sharing. You could also use this in conjunction with the ESRGAN upscaler with produces a crisper image. As an example, we used match image on your creation at 0.6 of the strength and here is what we generated upscaled to 2x.

 

Ai.Fy - text To Image for Unity is based on LoRAs of SD models. Stable Diffusion, just like DALLE-2, is a newer deep learning, text-to-image framework released in 2022. It is primarily used to generate detailed images conditioned on text descriptions. LoRA stands for Low-Rank Adaptation, a mathematical technique to reduce the number of parameters that are trained. You can think of it like creating a part of the model, instead of saving the whole model. LoRA was developed by researchers at Microsoft, and has been applied to Stable Diffusion for specific tasks. You can obtain more information about stable diffusion by visiting stability.ai.

 

[Updated]

2.1
- Concept Art generation limit increased to 50
- Retrieve Prompts from the Concept Art Browser
- Info tool tips added
- Editor Enhancements
- Removed and fixed EU Decimal Comma patch

 

Thanks. Below is the image made with ESRGAN upscaler

 

Another image made with ESRGAN upscaler

 

Are you changing something?
Just got error: "HTTP/1.1 500 Internal Server Error"

 

We were running a few maintenance checks for the new update we are going to release next week.

It should be working now.

 

Now I got error: "HTTP/1.1 400 Bad Request"
Any idea, how to fix it?

 

Hi, again.
Now, I get it to work in Unity 2020.3.46f1
Generate Depth Map still does not work, though.

 

Please could you tell us which Unity version you were using and the duration of the issue? Was this issue only with depth maps or the main image generator? We will check for server status during that hour.

 

I just purchase this asset, then followed the documentation correctly, but keep getting this error: "InvalidOperationException: Insecure connection not allowed"

As mentioned earlier the workaround is to enable non-secure HTTP downloads:
Go to: Edit->Project Settings->Player
Scroll down until you find: "Allow downloads over HTTP*"
Set it to: "Allowed in development builds". or the less secure option: "always allowed"

I think non-HTTP request are now blocked by default in newer versions of unity. Note I'm using unity 2022.1.7f1.

I think the documentation should be updated to reflect that this setting might need to be changed.

Or better yet, the developer should switch over to making secure requests using HTTPS, that way we don't have to change the player setting. I know it would take a lot of work switching your API over to HTTPS, but it would make this asset, and our projects more secure. HTTPS is the new security standard nowadays, that's probably why Unity now blocks insecure HTTP requests by default.

Note that this slight security risk is not stopping me from using this asset in my project, the benefits greatly outweigh the risk.

Keep up the great work on this amazing asset! I'm really excited to start using.

Here is a screenshot of the workaround.

 

Hi,
as an AI-tool noob I find it hard to understand what ”model”-setting to use and what difference they make. I can’t find anything about it in the documentation (although sometimes my brain plays tricks on me )..

it would be really nice if you coulf add a short description of intended use cases on which setting to use for what result.. I have been playing around with it and I can’t say I even notice a difference in the generated results

Other than that I really love the asset

 

Hello Rorep,

Thank you for your feedback and enthusiasm about the asset!

We understand your concern about the "InvalidOperationException: Insecure connection not allowed" error and appreciate the workaround you provided with the screenshot. Migrating an API to HTTPS involves obtaining SSL certificates, updating URLs, and ensuring that all dependencies and connections support HTTPS. It might require significant effort and careful testing to ensure the transition goes smoothly. We are in the process of doing so.

Also, with the protocol that we use (HTTP), the user passes a prompt with a few json settings and not the API key itself. This poses less of a threat because there is no personal information being transmitted. We've also mentioned in our privacy statements that the data transmitted is not stored or used. However, we will still strive to upgrade to the latest security protocol being used by the industry.

Again, thank you for your feedback and support. If you have any further questions or need assistance with anything else, feel free to ask!

 

Hello thefreik,

We appreciate your feedback and are glad to hear that you love the asset! If by "model" setting, you mean the engine enum, we agree, it can be a bit confusing, especially if you're new to diffusion models. The different engines offered are trained on different datasets. These datasets are vast, and often different models are great at producing different results based on training of the models. For example, the v2 and v2_1 are good at producing photorealistic pictures of people and XL is overfitted with aestheticism compared to v1.

We've include tool-tips for the editor tool as requested by @DeamyV in the latest version of the asset. If you hover over the "i" symbol before the setting heading, you can see a tool tip drop down that tells you more about the feature.

 

[Announcement]

Please update the asset to version 2.0 or above for optimal functionality.

 

Hello, Awesome asset! Im running into an error suddenly when generating:

"Curl error 28: Failed to connect to 3.219.85.134 port 5000 after 21210 ms: Timed out"

Any idea as to whats going on?

 

Hello pandemik,

Please update the asset to version 2.0 or above. It may be likely that you have a version prior to 2.0. If this is not the case, please send us an email at info@aikodex.com

 

Thank you! I was able to get it up and running again in my projects. It seems as if it doesn't work during runtime builds anymore. I removed the unnecessary code that prevents it from building and it just hangs after I initiate a generation from an android device. Has that functionality been removed?

 

It is possible that the code used for the build does not point to the correct server for inference, hence the halt. Again, we may need to test the issue in builds at our end. The asset was designed for developmental purposes in projects rather than builds. The asset is error-prone for builds due to a number of reasons, the biggest one being that this is a server-based asset and the endpoints change with every update. This unintentionally helps protect the asset against piracy.

 

To ensure we avoid any copyright infringement issues related to the AI-generated images, could you please clarify the source of your models? Did you use downloaded models or were they trained by you? If they were trained by you, have you obtained the proper rights to use the images for training? This clarification is important so that we can use the generated images without facing any legal troubles. Thank you.

 

Hello, can this asset create consistent 2D animations ? (the art style doesn't matter).

 

Nice work, how much time it takes on image creation on runtime on mobile?

 

\

Hello Hugopok,

The tool cannot create a video file, however, animations can be made by stitching together multiple images outputted by the AI model. By using the match image strength parameter and using prompts, you can generate a lot of images that are similar to each other. The images can then be made into a video file.

if you’re looking to create 2D pixel art, even that is possible. You can create stylised sprites of various objects and characters for your game by specifying “pixel art” in your prompt.

An elaborate sprite animation sheet, however, may not be possible. The model may not understand how spritesheets are arranged much less the intricate animation and movement of the sprite.

 

Hello jGate99,

The inference on the model takes ~7 to 12 seconds. The generation time is independent of the hardware or platform, reason being, the inference is performed on the server and pinged back to the device.

 

Hi, I think even you do not answer the question, your documentation at least should have all the related license and term of uses URL. Other generative ai tools in asset store have similar terms of uses. Am I right I should look at "stable diffusion"'s Term of uses? Stable diffusion now with lawsuit with Getty, are you using a totally different model that is without getty images? Or I should create a support ticket with Unity for follow up? Thanks.

 

Hello jc8888,

We apologies for the delay in addressing your question. Your concerns regarding the legality of AI and ethics of usage in your game are valid and we will try and address this by revising the terms of usage documentation. Ai.Fy is based on Stable Diffusion (SD), which utilizes pairs of images and captions from LAION-5B, a publicly available dataset derived from Common Crawl data scraped from the web. LAION-5B consisted of 5 billion image-text pairs that were classified by language and sorted into separate datasets based on resolution. SD specifically utilizes a dataset from this collection.

Regarding the copyright implications of generative AI, it is important to note that there have been no legal rulings or decisions passed as of date on this matter due to the nascent nature of the technology. While there are compelling arguments suggesting that copyright's fair use doctrine allows generative AI to use the images, there are equally strong counterarguments. From a reasonable legal perspective, there appears to be no significant deviation from the current situation.

All human creative output involves some degree of inspiration and derivation. For instance, if an individual were to create a painting closely resembling "Mona Lisa," it would be difficult to maintain copyright protection. However, there exists a fuzzy line where the work can be considered an original creation inspired by a famous painting. The only discernible difference lies in the likelihood of exposure. It has been argued that creators were unaware of prior works, which contributed to the originality of their own creations. In the case of an AI, such information can be more easily verified, making this scenario less likely. Generative AI can be seen as transformative in nature, as it generates new content inspired by existing works, while incorporating unique elements and adaptations. This transformative aspect can enhance the creativity and originality of video game experiences.

The use of generative AI in video games holds significant potential for enhancing visual creativity, expanding immersive experiences, and fostering player engagement. By considering the fair use doctrine, the transformative nature of generative AI, and the longstanding practice of drawing inspiration from existing works, developers can responsibly incorporate generative AI to create unique and captivating gaming experiences.

We have provided our terms of use and legal terms within the documentation we have provided. Additionally, Stable Diffusion's legal terms are also available for your perusal.

 

Thanks for your reply. Great!

 

how to generate icon with transparent background in 2.1.1 version?

 

Hello IrrationalNumberStudio,

As of now, we do not support generation of transparent backgrounds.

 

[Update]

Ai.Fy - Text to Image 3.0
- New AI Model capable of generating 1024x1024 introduced
- Server Improvements
- Editors for 2020, 2021 and 2022 added

 

wow on 1024x1024

 

Hi ..
Really like working with AiFy ..
One small issue though ..
Updated tonight to latest version 3.0
and the Browse Concept Art button just throws errors.

(new project, installed on 2021.3.27 and 2022.3.3)



Previous project still works fine ..
thanks for any assistance

 

The concept art browser is currently overwhelmed with requests due to the asset being on sale, surpassing the server's handling capacity. However, once the sale is concluded, we anticipate that its functionality will promptly return to normal.

 

is there any way to make ther backgroudn transparent? OR how am i supposed to use for example a knight sprite in my game?

 

Hello NotYou99,

We would like to point out that Ai.Fy, as of now, does not support transparency.

The way the latent tensors (the raw data that the AI creates) get turned into images is by scaling the number values in each dimension from 0-255 and assigning them to RGB values.

Adding an alpha channel (transparency) would require adding another dimension to the tensors, which would make all the models that were trained with three color dimensions incompatible. Basically, it could work, but it would require re-training the model from scratch. This would result in a bigger and less accurate model which is not the best trade off. Also we'd require a dataset having transparency to train on. So unfortunately, we do not support transparency. We are not sure we will in the future either.

For making items like an inventory, you can try the "match image' feature with a solid background and then use photoshop to remove the background. You can then use icon masking to "AI fill" the center region keeping the background the same. This will effectively allow you to have icons with the same background color which can then be removed later.

One of the users of the asset also recommended RemBG on GitHub by Daniel Gatis. Perhaps that may help you remove the background from multiple sprites.

We hope this information helps you

 

Wow, the last update 3.0 is awesome - thank you very much for it.
You are the best.

 

I can´t use
Unity 2022.3.5
few syntax errors
and no AiFy menu in the "window" options

 

Hello Felixsg,

We will install the editor version 2022.3 and check and get back to you.

Edit: We did not have an issue with using the package inside Unity version 2022.3.6 LTS. Please could you elaborate more on the errors you can see in the console logs?

 

I'm using 2022.3.0 and this is the only console error:
Assets\Aify\Editor\Scripts\AifyEditor.cs(19,13): error CS0234: The type or namespace name 'EditorCoroutines' does not exist in the namespace 'Unity' (are you missing an assembly reference?)

Edit:
Installing the 'Editor Coroutines' package from the 'Unity Registry' in the package manager seems to solve the issue.

 

Hi, Can I use the generated images for commercial games?

 

Hello Starry_seas,

Yes, you can use the generated images for commercial games.

 

Great asset! Help me, please. What prompt should i use to add background? I am try to use car icon with transparent background and want to make art with tat car.

 

Hello BorikN,

Hello NotYou99,

We're glad you liked the asset.

If you want a solid color background (which you can remove in an external software), you can use the "Central Icon Mask" in conjunction with a prompt and an image you want to match for the purpose of creating variants.

Here is an example:

Prompt: tower, iso, game, icon, minimal, vector, simple design

Model: Ai Fy XL

Match Image:



Match Image Strength: 0.012 (<0.1)

Mask: Central Icon Mask



As for transparent backgrounds Ai.Fy, as of now, does not support transparent backgrounds.

The way the latent tensors (the raw data that the AI creates) get turned into images is by scaling the number values in each dimension from 0-255 and assigning them to RGB values.

Adding an alpha channel (transparency) would require adding another dimension to the tensors, which would make all the models that were trained with three color dimensions incompatible. Basically, it could work, but it would require re-training the model from scratch. This would result in a bigger and less accurate model which is not the best trade off. Also we'd require a dataset having transparency to train on. So unfortunately, we do not support transparency. We are not sure we will in the future either.

For making items like an inventory, you can try the "match image' feature with a solid background and then use photoshop to remove the background. You can then use icon masking to "AI fill" the center region keeping the background the same. This will effectively allow you to have icons with the same background color which can then be removed later.

One of the users of the asset also recommended RemBG on GitHub by Daniel Gatis. Perhaps that may help you remove the background from multiple sprites.

Example:

 

Hi, i just bought the Ai.Fy and added it to an empty 2022.3.6f1 Unity project. i made sure that the configuration allows network connection as well as made sure the dependencies are installed: Barracuda.

ive been getting a console.Log for 500 Internal Server Error. im not using the match image feature. all i typed in was 1 word for a prompt to test it out. the documentation doesnt seem to have any information on this. could i get some help?

here's the full log

HTTP/1.1 500 Internal Server Error

UnityEngine.Debug:Log (object)

AiKodex.AifyEditor/<Post>d__89:MoveNext () (at Assets/Aify/Editor/Scripts/AifyEditor.cs:1063)

Unity.EditorCoroutines.Editor.EditorCoroutine/YieldProcessor:MoveNext (System.Collections.IEnumerator) (at ./Library/PackageCache/com.unity.editorcoroutines@1.0.0/Editor/EditorCoroutine.cs:80)

Unity.EditorCoroutines.Editor.EditorCoroutine:ProcessIEnumeratorRecursive (System.Collections.IEnumerator) (at ./Library/PackageCache/com.unity.editorcoroutines@1.0.0/Editor/EditorCoroutine.cs:134)

Unity.EditorCoroutines.Editor.EditorCoroutine:MoveNext () (at ./Library/PackageCache/com.unity.editorcoroutines@1.0.0/Editor/EditorCoroutine.cs:115)

UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

 

Hello,

We made a few chnages to the backend.

Please update the asset to the latest version 1.3 to avail the newest features in terms of generation and security.

The following Ai.Fy - Text to Image versions will no longer be supported:
v1.0, v1.1, v1.2, v2.0, v2.1, v2.1.1

Please let us know if updating the asset fixed your issue.

 

Hello,

Great work!

There is the way to generate a character with frontal, retro, left, right views? I'm struggling to make many trials but I obtain only the front view.

Thanks in advance!

 

We're glad that you liked the asset.

As for the views, you can try and engineer your prompt in a manner in which the model creates different views.
For a left view, you can input something on the lines of "left profile", or "profile view", or "sideways" and flip invert the image for the right view. For the retro view (assuming retro means you want the rear side of the character, for example in a pixel art style game), you can type "Back view", "Rear view".
You can input the front view of the image of your choice in the Match Image section and adjust the Match image strength till you get your desired output. You can also use the central mask if you are working on a single entity roughly in the center of the image.

 

Thanks for reply, but I tried all that but the result is still the same: only front view.

 

in my package manager it says 3.0 Current. i clicked re-download and imported it but there was no change. is there anything else i should check?

wait. suddenly its working now. never mind! time to play with it. thank you

 

okay so for like 50 minutes ive been getting some miserable results. is there a trick to this? i even tried copying the exact prompts from your video + settings and even the seed ->

and had vastly different results.