Search Unity

[RELEASED] Dialogue System Addon for OpenAI (ChatGPT), ElevenLabs, and Overtone

Discussion in 'Assets and Asset Store' started by TonyLi, Jun 15, 2023.

  1. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    The Dialogue System Addon for OpenAI is available on the Asset Store!



    Requires the Dialogue System for Unity (2.2.38+).

    >> Manual | Video Tutorials
    >> Runtime Conversation Demo


    Supports:
    • GPT-4 Turbo, GPT-3.5 Turbo, and other language models.
    • DALL-E for images
    • Whisper for speech recognition
    • ElevenLabs and Overtone for speech synthesis.
    • Dialogue Smith for advanced branching dialogue.

    Features:
    • Generate conversations using your characters and topics - including at runtime
    • Generate NPC barks
    • Revise text -- spruce up your dialogue, fix spelling & grammar, add variations
    • Translate dialogue and quests to other languages
    • Freeform chat with GPT-4/GPT3_5_Turbo for a ChatGPT-like experience
    • Generate actor portrait images using DALL-E
    • Use Whisper for speech-to-text recognition
    • Generate voiceover audio using ElevenLabs or Overtone speech synthesis
    • Generate advanced branching dialogue using Dialogue Smith
    • Runtime use is optional and includes these modes:
      • Freeform text or speech responses to the ChatGPT-powered NPC
      • Generate entire conversation at runtime and play it like a regular conversation
      • Generate NPC barks
      • Play choose-your-own-adventures with DALL-E generated illustrations
      • The modes can use Whisper for speech recognition and ElevenLabs for voice actor generation
    • No scripting required - create NPCs that you can chat with, no programming required
    • Includes complete, documented source code

    Use OpenAI’s generative AI to write narrative content faster, get suggestions when you’re stuck, fix spelling and grammar, and translate your text to other languages. You can also use the addon for freeform chat similar to ChatGPT -- ask for quest ideas, get help writing code, and more. The addon is fully integrated into the Dialogue System for Unity's Dialogue Editor for a friction-free experience that's easy to use for beginners yet powerful enough for expert content creators. It uses your own OpenAI API key (pricing) to create and update dialogue content at edit time.

    Supports optional runtime use to generate dialogue on the fly, in response to the user's freeform text input and/or game state. However, you can use it solely in the Unity editor at design time if you prefer so you won't accrue any OpenAI API costs during play.

    upload_2023-6-14_21-20-46.png

    Note: This addon uses the OpenAI API, so you must be able to access api.openai.com from your geographic region.

    See also: Dialogue System for Unity forum thread
     
    Last edited: Dec 26, 2023
    elmar1028 likes this.
  2. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon 1.0.4 for OpenAI Released

    The Dialogue System Addon for OpenAI has moved to this thread to help separate discussions between the core Dialogue System for Unity asset and this addon. Previous release notes are in the original thread and the asset's _RELEASE_NOTES.txt file.


    Notable New Features:
    • This is the first asset to support OpenAI's new GPT-3.5 Turbo 16K model.
    • This is also currently the only asset to support the Dialogue Smith API for branching dialogue generation.
    • Also supports ElevenLabs, Whisper, and DALL-E.

    Release Notes:
    • Changed: Requires Dialogue System 2.2.38+.
    • Added: GPT-3.5 Turbo 16K support.
    • Added: Whisper (OpenAI speech to text transcription) support.
    • Added: Dialogue Smith integration.
    • Added: Runtime Choose-Your-Own-Adventure mode, with optional image and voice narration generation.
    • Improved: Added RuntimeAIConversation.freeformTextInputConversationEnded C# event.
    • Improved: Added microphone speech input option to Runtime Demo scene.
     
  3. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    There's a one-time upgrade step for version 1.0.4:

    Please delete SavWav.cs -- or the entire Plugins/Pixel Crushers/Dialogue System Addon for OpenAI folder if you prefer -- and then reimport the addon. This will put SavWav in its new location, since it's now used by the runtime Whisper API.
     
  4. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for OpenAI (+ ElevenLabs & Dialogue Smith) in AI > AI-ML Integration Category

    You can now find the Addon for OpenAI in the Asset Store's new AI > AI-ML Integration category:

    upload_2023-6-27_14-46-6.png

    The addon integrates with:
    • OpenAI GPT-4, 3.5-Turbo, and other LLMs for text generation integrated well with the Dialogue System
    • DALL-E for image generation
    • Whisper for speech recognition
    • ElevenLabs for great voice acting generation, and
    • Dialogue Smith for advanced branching dialogue generation.
    All with complete, documented source code. (Although no scripting is actually required to generate content at design time or set up optional runtime dialogue and choose-your-own-adventure scenarios.)
     
  5. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    A Dialogue System user recently asked in email what advantages the Addon for OpenAI provides over other solutions. Here are some of the points we discussed in case they answer questions for devs interested in using the Addon:
    • In runtime-generated dialogue, you can include Dialogue System variable values in prompts. This allows NPCs to change their knowledge over time -- for example, as the world changes and the player's complete actions. Using [lua(code)] tags, you can also include any extra info that you can return in a custom C# method, too.

    • You can also generate pre-made dialogue at design time using OpenAI or Dialogue Smith (a more advanced API for branching dialogue). And/or you can hand-write dialogue, too.

    • You can use all Dialogue System features such as the dialogue UI system to present dialogue to the player.

    • You can use all Dialogue System integrations such as SALSA LipSync Suite to perform lipsync and eye/head movement.

    • It works with the new Input System package. (Some other solutions don't work with the new Input System package yet.) Personally, I prefer using Rewired or the built-in input manager to the Input System, but other devs like the option to use the Input System.
     
  6. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon 1.0.5 for OpenAI Released

    Version 1.0.5 is now available on the Asset Store!

    Release Notes:
    • Context now supports Actor Knowledge & Goals fields in addition to Description.
    • Can now incorporate Location Descriptions in context.
    • RuntimeAIConversation.StartFreeformTextInputConversation() now sends OnConversationStart message.
    • DialogueSmith uses new UnityWebRequest.PostWwwForm() in Unity 2022+.

    There's a new tutorial video that demonstrates how to set the Dialogue System Addon for OpenAI with RPG Builder and SALSA LipSync Suite to create dynamic NPCs that you can chat with about anything:

     
    blueivy likes this.
  7. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    I'll be running a virtual booth this Saturday at Jason Weimann's GameDevGuild AI conference. If you have questions about the Addon for OpenAI or want to chat about chatty AI NPCs, feel free to stop by!
     
    blueivy likes this.
  8. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for Open AI - Version 1.0.6 Released

    Version 1.0.6 is now available on the Asset Store! This version introduces a handy AI Assistant window that functions like ChatGPT:

    upload_2023-9-27_12-25-57.png

    Release Notes:
    • Added ChatGPT-like AI Assistant Window.
    • Added backward compatibility with Unity 2020.
    • Fixed issue in case where Dialogue Smith returns no text for a dialogue entry.
     
    blueivy likes this.
  9. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for Open AI 50% Off

    The Dialogue System Addon for OpenAI is 50% off in the Asset Store's Fresh Assets Sale.

    And it's still the only generative AI addon that:
    • Works with OpenAI to generate conversations, barks, and localization translations
    • Supports the Dialogue Smith API for advanced branching conversation generation
    • Works with ElevenLabs to create AI voice acting for your dialogue
    • Work at design time -- or runtime for dynamic conversations using freeform text input or microphone input, text to speech, and image generation
    • And integrates directly with the Dialogue System for Unity
     
  10. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    New Video Tutorial: Dialogue Smith Integration

    The Video Tutorial Playlist has a new video on Dialogue Smith:



    The Dialogue System Addon for OpenAI is 50% off in the Fresh Assets Sale.

    It's the only asset that has Dialogue Smith integration to quickly generate dialogue text for branching conversations.
     
    blueivy likes this.
  11. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Demo Scene: Using Love/Hate Gameplay State for Runtime Conversation Prompts

    Someone recently asked how to incorporate runtime information from assets such as Love/Hate into prompts for runtime conversations. The Dialogue System Extras page has an example scene that uses the Dialogue System, Addon for OpenAI, and Love/Hate to do exactly this (direct download). When you talk to the Princess NPC, the prompt incorporates her currently affinity to the player (i.e., like or dislike). Instructions are included in the package.
     
  12. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for OpenAI - 50% Off for Limited Time

    The Asset Store's Fresh Asset Sale ends soon. If you've had your eye on the Dialogue System Addon for OpenAI, make sure to get it for 50% off before the sale ends.

     
  13. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Version 1.0.8 Released

    Version 1.0.8 is now available on the Asset Store!

    The highlights of this update are GPT-4 Turbo and ElevenLabs v2 models.


    Release Notes:
    • Added: Support for GPT-4 Turbo (model currently in preview but fast, very powerful [128k tokens!], and cheap)
    • Added: Support for ElevenLabs v2 models.
    • Improved: Added top_p, frequency_penalty, and presence_penalty to editor windows, runtime settings, and API.
    • Improved: Generate Voice section can now add sequencer command, remembers last folder audio files were saved into.
    • Improved: Select Voice section now has button to generate all voice lines for an actor.
     
  14. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
  15. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for OpenAI - Version 1.0.8.1 Released

    Version 1.0.8.1 is now available on the Asset Store. It contains a small fix to include custom voices in the ElevenLabs voice selection dropdown.


    Reminder - The Addon for OpenAI and its dependency (the Dialogue System for Unity) are both 50% off in the Black Friday Sale.
     
  16. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for OpenAI - Version 1.0.9 Released w/Overtone Integration

    Version 1.0.9 of the Dialogue System Addon for OpenAI is now live on the Asset Store with new integration for Overtone offline speech synthesis, in addition to the existing integration with ElevenLabs' stellar speech synthesis API.



    Release Notes

    Version 1.0.9:
    • Improved: Runtime dialogue now gets field info from Lua so you can change it at runtime.
    • Overtone: Added integration.

    The Dialogue System and the Addon for OpenAI are both 50% off in the New Year Sale! Quest Machine and Love/Hate are 50% off, too.

     
    blueivy likes this.
  17. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
  18. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    SALE ENDS IN 24 HOURS

    50% off all Pixel Crushers assets -- including the Dialogue System for Unity and the Addon for OpenAI -- ends in just 24 hours.

    Make sure to pick up the entire narrative tools suite -- Dialogue System, Addon for OpenAI, Quest Machine, and Love/Hate -- or just the tools you need, while they're on sale.
     
  19. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for OpenAI v1.0.10 Released

    Version 1.0.10 is now available on the Asset Store!

    Highlights: Overtone integration, Text Table asset translation


    Release Notes:
    • Added: Text Table asset translation.
    • Improved: RuntimeAIConversation.ApproximateTokenCount and ApproachedMaxTokens are now settable.
    • Improved: RuntimeAIConversation handles additional text formats for dialogue replies from OpenAI.
    • Improved: Runtime dialogue now gets field info from Lua so you can change it at runtime.
    • Changed: Translation panels now default to GPT-3.5 Turbo 16K (since Curie is deprecated).
    • Fixed: Can now tick Welcome Window's Show On Start checkbox even if no OpenAI key is set.
    • Fixed: Dialogue Smith now keeps existing text if present.
    • Overtone: Added integration.
     
  20. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
    Dialogue System Addon for OpenAI v1.0.11 Released

    Version 1.0.11 is now available on the Asset Store!

    Highlights: Support for your own fine-tuned OpenAI models!

    Release Notes:
    • Added: Support for fine-tuned models.
    • Improved: Freeform runtime conversation now ends automatically with goodbye line when max tokens are reached.
    • Improved: Now handles more varied dialogue response formats.
    • Dialogue Smith: Removed. Dialogue Smith has discontinued service.
     
  21. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688
  22. TonyLi

    TonyLi

    Joined:
    Apr 10, 2012
    Posts:
    12,688