[RELEASED] Dialogue System Addon for OpenAI (ChatGPT), ElevenLabs, and Overtone

The Dialogue System Addon for OpenAI is available on the Asset Store!

Requires the Dialogue System for Unity (2.2.38+).

Get It On The Store

>> Manual | Video Tutorials
>> Runtime Conversation Demo

Supports:

• GPT-4 Turbo, GPT-3.5 Turbo, and other language models.
• DALL-E for images
• Whisper for speech recognition
• ElevenLabs and Overtone for speech synthesis.
• Dialogue Smith for advanced branching dialogue.

Features:

• Generate conversations using your characters and topics - including at runtime
• Generate NPC barks
• Revise text -- spruce up your dialogue, fix spelling & grammar, add variations
• Translate dialogue and quests to other languages
• Freeform chat with GPT-4/GPT3_5_Turbo for a ChatGPT-like experience
• Generate actor portrait images using DALL-E
• Use Whisper for speech-to-text recognition
• Generate voiceover audio using ElevenLabs or Overtone speech synthesis
• Generate advanced branching dialogue using Dialogue Smith
• Runtime use is optional and includes these modes:
  • Freeform text or speech responses to the ChatGPT-powered NPC
  • Generate entire conversation at runtime and play it like a regular conversation
  • Generate NPC barks
  • Play choose-your-own-adventures with DALL-E generated illustrations
  • The modes can use Whisper for speech recognition and ElevenLabs for voice actor generation
• No scripting required - create NPCs that you can chat with, no programming required
• Includes complete, documented source code

Use OpenAI’s generative AI to write narrative content faster, get suggestions when you’re stuck, fix spelling and grammar, and translate your text to other languages. You can also use the addon for freeform chat similar to ChatGPT -- ask for quest ideas, get help writing code, and more. The addon is fully integrated into the Dialogue System for Unity's Dialogue Editor for a friction-free experience that's easy to use for beginners yet powerful enough for expert content creators. It uses your own OpenAI API key (pricing) to create and update dialogue content at edit time.

Supports optional runtime use to generate dialogue on the fly, in response to the user's freeform text input and/or game state. However, you can use it solely in the Unity editor at design time if you prefer so you won't accrue any OpenAI API costs during play.



Note: This addon uses the OpenAI API, so you must be able to access api.openai.com from your geographic region.

See also: Dialogue System for Unity forum thread

Get the Addon on the Asset Store

 

Dialogue System Addon 1.0.4 for OpenAI Released

The Dialogue System Addon for OpenAI has moved to this thread to help separate discussions between the core Dialogue System for Unity asset and this addon. Previous release notes are in the original thread and the asset's _RELEASE_NOTES.txt file.


Notable New Features:

• This is the first asset to support OpenAI's new GPT-3.5 Turbo 16K model.
• This is also currently the only asset to support the Dialogue Smith API for branching dialogue generation.
• Also supports ElevenLabs, Whisper, and DALL-E.

Release Notes:

• Changed: Requires Dialogue System 2.2.38+.
• Added: GPT-3.5 Turbo 16K support.
• Added: Whisper (OpenAI speech to text transcription) support.
• Added: Dialogue Smith integration.
• Added: Runtime Choose-Your-Own-Adventure mode, with optional image and voice narration generation.
• Improved: Added RuntimeAIConversation.freeformTextInputConversationEnded C# event.
• Improved: Added microphone speech input option to Runtime Demo scene.

 

There's a one-time upgrade step for version 1.0.4:

Please delete SavWav.cs -- or the entire Plugins/Pixel Crushers/Dialogue System Addon for OpenAI folder if you prefer -- and then reimport the addon. This will put SavWav in its new location, since it's now used by the runtime Whisper API.

 

Dialogue System Addon for OpenAI (+ ElevenLabs & Dialogue Smith) in AI > AI-ML Integration Category

You can now find the Addon for OpenAI in the Asset Store's new AI > AI-ML Integration category:



The addon integrates with:

• OpenAI GPT-4, 3.5-Turbo, and other LLMs for text generation integrated well with the Dialogue System
• DALL-E for image generation
• Whisper for speech recognition
• ElevenLabs for great voice acting generation, and
• Dialogue Smith for advanced branching dialogue generation.

All with complete, documented source code. (Although no scripting is actually required to generate content at design time or set up optional runtime dialogue and choose-your-own-adventure scenarios.)

 

A Dialogue System user recently asked in email what advantages the Addon for OpenAI provides over other solutions. Here are some of the points we discussed in case they answer questions for devs interested in using the Addon:

• In runtime-generated dialogue, you can include Dialogue System variable values in prompts. This allows NPCs to change their knowledge over time -- for example, as the world changes and the player's complete actions. Using [lua(code)] tags, you can also include any extra info that you can return in a custom C# method, too.
  
  
• You can also generate pre-made dialogue at design time using OpenAI or Dialogue Smith (a more advanced API for branching dialogue). And/or you can hand-write dialogue, too.
  
  
• You can use all Dialogue System features such as the dialogue UI system to present dialogue to the player.
  
  
• You can use all Dialogue System integrations such as SALSA LipSync Suite to perform lipsync and eye/head movement.
  
  
• It works with the new Input System package. (Some other solutions don't work with the new Input System package yet.) Personally, I prefer using Rewired or the built-in input manager to the Input System, but other devs like the option to use the Input System.

 

Dialogue System Addon 1.0.5 for OpenAI Released

Version 1.0.5 is now available on the Asset Store!

Release Notes:

• Context now supports Actor Knowledge & Goals fields in addition to Description.
• Can now incorporate Location Descriptions in context.
• RuntimeAIConversation.StartFreeformTextInputConversation() now sends OnConversationStart message.
• DialogueSmith uses new UnityWebRequest.PostWwwForm() in Unity 2022+.

There's a new tutorial video that demonstrates how to set the Dialogue System Addon for OpenAI with RPG Builder and SALSA LipSync Suite to create dynamic NPCs that you can chat with about anything:

 

I'll be running a virtual booth this Saturday at Jason Weimann's GameDevGuild AI conference. If you have questions about the Addon for OpenAI or want to chat about chatty AI NPCs, feel free to stop by!

 

Dialogue System Addon for Open AI - Version 1.0.6 Released

Version 1.0.6 is now available on the Asset Store! This version introduces a handy AI Assistant window that functions like ChatGPT:



Release Notes:

• Added ChatGPT-like AI Assistant Window.
• Added backward compatibility with Unity 2020.
• Fixed issue in case where Dialogue Smith returns no text for a dialogue entry.

 

Dialogue System Addon for Open AI 50% Off

The Dialogue System Addon for OpenAI is 50% off in the Asset Store's Fresh Assets Sale.

And it's still the only generative AI addon that:

• Works with OpenAI to generate conversations, barks, and localization translations
• Supports the Dialogue Smith API for advanced branching conversation generation
• Works with ElevenLabs to create AI voice acting for your dialogue
• Work at design time -- or runtime for dynamic conversations using freeform text input or microphone input, text to speech, and image generation
• And integrates directly with the Dialogue System for Unity

 

New Video Tutorial: Dialogue Smith Integration

The Video Tutorial Playlist has a new video on Dialogue Smith:

The Dialogue System Addon for OpenAI is 50% off in the Fresh Assets Sale.

It's the only asset that has Dialogue Smith integration to quickly generate dialogue text for branching conversations.

 

Demo Scene: Using Love/Hate Gameplay State for Runtime Conversation Prompts

Someone recently asked how to incorporate runtime information from assets such as Love/Hate into prompts for runtime conversations. The Dialogue System Extras page has an example scene that uses the Dialogue System, Addon for OpenAI, and Love/Hate to do exactly this (direct download). When you talk to the Princess NPC, the prompt incorporates her currently affinity to the player (i.e., like or dislike). Instructions are included in the package.

 

Dialogue System Addon for OpenAI - 50% Off for Limited Time

The Asset Store's Fresh Asset Sale ends soon. If you've had your eye on the Dialogue System Addon for OpenAI, make sure to get it for 50% off before the sale ends.

 

Version 1.0.8 Released

Version 1.0.8 is now available on the Asset Store!

The highlights of this update are GPT-4 Turbo and ElevenLabs v2 models.


Release Notes:

• Added: Support for GPT-4 Turbo (model currently in preview but fast, very powerful [128k tokens!], and cheap)
• Added: Support for ElevenLabs v2 models.
• Improved: Added top_p, frequency_penalty, and presence_penalty to editor windows, runtime settings, and API.
• Improved: Generate Voice section can now add sequencer command, remembers last folder audio files were saved into.
• Improved: Select Voice section now has button to generate all voice lines for an actor.

 

Dialogue System Addon for OpenAI in Black Friday Sale

The Dialogue System Addon for OpenAI (ChatGPT, and ElevenLabs for voice acting, and DALL-E for images, and Dialogue Smith for branching dialogue), is 50% off in the Black Friday Sale!

This addon requires the Dialogue System for Unity, which is also 50% off!

 

Dialogue System Addon for OpenAI - Version 1.0.8.1 Released

Version 1.0.8.1 is now available on the Asset Store. It contains a small fix to include custom voices in the ElevenLabs voice selection dropdown.


Reminder - The Addon for OpenAI and its dependency (the Dialogue System for Unity) are both 50% off in the Black Friday Sale.

 

Dialogue System Addon for OpenAI - Version 1.0.9 Released w/Overtone Integration

Version 1.0.9 of the Dialogue System Addon for OpenAI is now live on the Asset Store with new integration for Overtone offline speech synthesis, in addition to the existing integration with ElevenLabs' stellar speech synthesis API.

Release Notes

Version 1.0.9:

• Improved: Runtime dialogue now gets field info from Lua so you can change it at runtime.
• Overtone: Added integration.

The Dialogue System and the Addon for OpenAI are both 50% off in the New Year Sale! Quest Machine and Love/Hate are 50% off, too.

 

Dialogue System Addon for OpenAI (& ElevenLabs, Overtone, Dialogue Smith, etc.) 50% Off

All Pixel Crushers assets, including the Dialogue System Addon for OpenAI, are 50% off in the Unity Awards Asset Sale!

 

SALE ENDS IN 24 HOURS

50% off all Pixel Crushers assets -- including the Dialogue System for Unity and the Addon for OpenAI -- ends in just 24 hours.

Make sure to pick up the entire narrative tools suite -- Dialogue System, Addon for OpenAI, Quest Machine, and Love/Hate -- or just the tools you need, while they're on sale.

 

Dialogue System Addon for OpenAI v1.0.10 Released

Version 1.0.10 is now available on the Asset Store!

Highlights: Overtone integration, Text Table asset translation


Release Notes:

• Added: Text Table asset translation.
• Improved: RuntimeAIConversation.ApproximateTokenCount and ApproachedMaxTokens are now settable.
• Improved: RuntimeAIConversation handles additional text formats for dialogue replies from OpenAI.
• Improved: Runtime dialogue now gets field info from Lua so you can change it at runtime.
• Changed: Translation panels now default to GPT-3.5 Turbo 16K (since Curie is deprecated).
• Fixed: Can now tick Welcome Window's Show On Start checkbox even if no OpenAI key is set.
• Fixed: Dialogue Smith now keeps existing text if present.
• Overtone: Added integration.

 

Dialogue System Addon for OpenAI v1.0.11 Released

Version 1.0.11 is now available on the Asset Store!

Highlights: Support for your own fine-tuned OpenAI models!

Release Notes:

• Added: Support for fine-tuned models.
• Improved: Freeform runtime conversation now ends automatically with goodbye line when max tokens are reached.
• Improved: Now handles more varied dialogue response formats.
• Dialogue Smith: Removed. Dialogue Smith has discontinued service.

 

Flash Deal - Limited Time 70% off

The Dialogue System Addon for OpenAI is 70% off for a very limited time in the Asset Store's Flash Deals sale.

 

If you missed the initial flash deal, you can still get the Dialogue System Addon for OpenAI at a whopping 50% off.

There's a detailed Video Tutorial Series, including -- for those RPG Builder users out there -- a step-by-step setup of NPCs that you can chat with freeform at runtime using your microphone and they'll speak back using voiceover and lipsync animation:

 

Dialogue System Addon for OpenAI (and other generative AI) 50% Off

Also keep an eye on the Flash Deals -- the Dialogue System for Unity will be in a flash deal next week!

 

"Directing" Voice Performances With ElevenLabs

​

A dev recently asked how to "direct" the voice performances generated by the Dialogue System Addon's ElevenLabs integration.

You can do it contextually.

For example, I tested this line in the Dialogue System's Demo:

• Private Hart says "Any luck downloading the launch codes?"
• If I change the line to "Any luck downloading the launch codes???" the voice is more emphatic and excited.
• If I change it to "any... luck, uh, downloading the launch codes...?" the voice is subdued and hesitant.

If you want a pause that doesn't imbue the voice with hesitation or shyness, use a dash (-) instead of ellipsis (...).

Also, the choice of voice actor affects the results. Some actors recorded very monotone performances. Other actors overacted every line so you can get very melodramatic readings.

All CAPS makes them shout, btw.

Reminder - The Dialogue System Addon for OpenAI, ElevenLabs, and Overtone is 50% Off right now in the Spring Sale.
(Requires the Dialogue System for Unity, and is nicely integrated with the Dialogue System.)

 

The Dialogue System Addon for OpenAI, ElevenLabs, Overtone, and Other Generative AI is 50% off in the Spring Sale!

Here's a video that shows how to set up an RPG Builder project from scratch with the Addon for OpenAI to run dynamic conversations at runtime with ElevenLabs-generated voice acting and SALSA lipsync animation:

Get Dialogue System for Unity - Addon for Generative AI - Quest Machine

 

how do you generate audio for other languages other than english with elevenlabs? generate voiceover only shows english language

 

Hi! In the upcoming version 1.0.14 (releasing Monday), the localization field's AI button is now a dropdown that lets you select Translate or Voiceover:



With ElevenLabs, you don't need to specify a language. From their manual:

Supported languages

Our TTS API is multilingual and currently supports the following languages:

Chinese, Korean, Dutch, Turkish, Swedish, Indonesian, Filipino, Japanese, Ukrainian, Greek, Czech, Finnish, Romanian, Russian, Danish, Bulgarian, Malay, Slovak, Croatian, Classic Arabic, Tamil, English, Polish, German, Spanish, French, Italian, Hindi and Portuguese.

To use them, simply provide the input text in the language of your choice.​

 

thank you for your fast response. that update is just what I needed.

 

Version 1.0.14 of the Dialogue System Addon for OpenAI, ElevenLabs, and other generative AI is live on the Asset Store -- now with powerful GPT-4o integration!

Another nice addition: The "AI" button in localized dialogue text fields is now a dropdown so you can choose to translate the text using OpenAI or generate localized voiceover audio.



The Addon is currently 50% off in the Template Toolkit Sale, too.

 

Amazing Update. Just tried it out. You guys should add the ability to select the Elevenlabs model when doing voiceover. Because when I tried using Multilingualv2 in Spanish(my native language) for a girl character, sometimes it gives me a male voice. When I changed the model to Multilingualv1(Elevenlabs website recommends using it for this voice when doing it through the website) it gives a female voice every time.

 

Thanks for the feedback! I'll add that option in the next update.