RT-Voice - Run-time text-to-speech solution

When using Azure, is Speaker Voice the object to use to define which voice to use?
I'm trying to distill the Azure example into a single female voice for our avatar.
However I'm sure I'm missing something.

This example requests text from ChatGPT, then uses RT-Voice and SALSA to speak/lipsync.
It works with the Speaker.Instance.Speak(textArea.text, Source, Speaker.Instance.VoiceForCulture("en"));
but returns an invalid audio file if I try to define a SpeakerVoice.

Code (CSharp):

1. var completionResponse = await openai.CreateCompletion(new CreateCompletionRequest()
2.             {
3.                 Prompt = Instruction,
4.                 Model = "text-davinci-003",
5.                 MaxTokens = 128
6.             });
7.  
8.             textArea.text = completionResponse.Choices[0].Text;
9.             setNarrationText(textArea.text);
10.             Instruction += $"{completionResponse.Choices[0].Text}\nQ: ";
11.  
12.             Debug.Log("+++ Speaking line for RT Voice");
13.             // Speaker.Instance.Speak(textArea.text, Source, Speaker.Instance.VoiceForCulture("en"));
14.             if (!string.IsNullOrEmpty(uid))
15.                 Speaker.Instance.Silence(uid);
16.  
17.             uid = GUISpeech.isNative ? Speaker.Instance.SpeakNative(textArea.text, SpeakerVoice, GUISpeech.Rate, GUISpeech.Pitch, GUISpeech.Volume) : Speaker.Instance.Speak(textArea.text, Source, SpeakerVoice, true, GUISpeech.Rate, GUISpeech.Pitch, GUISpeech.Volume);

 

Hi

Have you tried using Speaker.Instance.VoiceForName("Libby")?
Please don't define your own voice, use the ones that are already provided.


Cheers
Stefan

 

Ah, thank you. At every turn, you've coded RT-Voice to be *very* easy to implement.

 

Hi, I just started using this plugin, I ran your Android Demo, it works fine, but when I add the plugin into the Unity project, then package the demo scenes, after running on the mobile, I encountered some strange things. The word <speak> will be added before and after each speech. There is no problem on the computer, it only appears on the mobile phone!
I don't know how to solve this problem, waiting for your reply, thank you!

 

Hi

Which version of our plugin are you using?


So long,
Stefan

 

the plugin version is 2023.1.0, Unity version is 2022.2.8

 

In addition, if like this, Speaker.Instance.Speak(Text), the Voice is null, does it mean that the Voice in the device will be automatically matched ?
And the Voice is random, or according to the language of the current device, or the current What about the language type of the input text?
For example, if the current text is Japanese and the device language is German, what language is the translated voice in?

 

Hi

Normally, all TTS-systems from Android since version 7 support SSML, but as it seems, the engine on your device doesn't and <speak> is spoken... You could either make sure that the Google-TTS engine is installed under Android or enable "Auto Clear Tags" under "Advanced Settings" in the RTVoice-object.

To prevent using a random voice, you could add it to the Speak-command, like this:
Speaker.Instance.Speak("text", null, Speaker.Instance.VoiceForName("yourVoiceName"));

I hope this helps you further.


Cheers
Stefan

 

Hello- longtime user

I've recently installed several English voices on my Windows 10 install- I installed the voices and restarted Windows- they all work in Windows etc


Rt Voice does not recognize the newly added voices- this is all RtVoice recognizes


I tried hitting reload of course etc-

I bought a Cereproc voice and installed it to see if that would work- it shows in RtVoice but not in the Windows voices- when I try to test the Cereproc voice in RtVoice it throws an error 400- it starts to play then stops and throws that error-



Any ideas on how to get RtVoice to recognize/use these other voices? Thank you

 

Hello dear longtime user

Since version 8, Windows contains two TTS-systems: one for "classic" (= standalone apps) with SAPI. The other one (your first picture) is for UWP apps.
So, if you want to access them, you have to build an UWP (WSA) app.
Alternatively, you could add classic SAPI-voices for standalone. Please see chapter 6 of the documentation:
https://www.crosstales.com/media/data/assets/rtvoice/RTVoice-doc.pdf

Unfortunately, we don't own Cereproc voices, therefore I can't say what's going on :-(
However, you could try eSpeak or a supported online solution like MaryTTS, AWS Polly, Azure or Google.

I hope this helps you further.


Cheers
Stefan

 

Thank you for your help- I'll give the other voices a try.

 

Hello, could someone explain how to embed MaryTTS as library into unity game to work offline? Is there some demo? I saw a note in the documentation "Standalone: MaryTTS could be bundled with the app installer and run locally", but I cannot find more details to it.

Thank you!

 

Hi

You could download MaryTTS to your PC (e.g. the 5.2 Runtime-version):
https://marytts.github.io/download/index.html

Then extract it and run "marytts-server" under "bin" to start the server (Java has to be installed on your machine).
After that, configure MaryTTS in Unity and press "Play":


For an actual build, you have to bundle MaryTTS with the desired voices, we would suggest creating a setup for your app or extract and run it from your app itself. There are many options, it's up to you

I hope this helped you further.


Cheers
Stefan

 

Hi Stefan,
Thanks for the great asset.
A simple question:
Are visemes of your API (on Windows platform) exactly the same 22 ones as in this Microsoft doc:
https://learn.microsoft.com/en-us/a...ots=programming-language-csharp&tabs=visemeid
I am asking since I want to convert viseme-related features from an existing Windows app to Unity app, initially only on Windows platform.
Thank you!

 

Is it a suitable application to make the mouth play realistically while speaking the api data we will give using text to speech?

 

I am unable to save audio clip the android persistent data path. Its working fine in the unity editor. But not in the android build. Can you please help with this. Is it possible in this package

 

I'm try to get an animal crossing style effect to my text voices. But for that I need to be able to crank up the pitch to a high value to get a squeaky chirpy voice effect. I bought the plugin for trying this purpose, but from my tests I cannot increase the pitch beyond 2. I tried modifying the code to remove this clamp, but I couldn't get the high pitch effect in the end. Can you point me to how I can remove the clamp value of the pitch so I can tinker with higher values ?

Thanks!

 

Hi,


I love your product RY-Voice PRO, it’s a wonderful.

I’ve been satisfied your nice work for TTS on Android except one problem.

When I try to connect my app to TV via cable, wireless or set-top box itself, the sound from your asset doesn’t work.

Very curios situation. Definitely the asset works well on the app in mobile. But mirroring or cabling to any TV makes problem about no sound.

All the other application looks fine, sound with mirroring, for instance YouTube or any other app.

Probably I missed some parameter setting for sound of TV connection

Please guide me for the sound on TV. With your asset.

 

Dear all,

I have followed documentation to use Azure API. But at first attempt I get this:

The generated audio file is invalid: Wrapper {Uid='158fac69-3929-4162-8fa0-a4437275449e', Text='Hello', Source='True', Voice='Catherine (en-AU, FEMALE)', SpeakImmediately='True', Rate='1', Pitch='1', Volume='1', OutputFile='', ForceSSML='True', Created='11/06/2023 21:03:13'}
UnityEngine.Debug:LogError (object)
Crosstales.RTVoice.Provider.BaseCustomVoiceProvider/<playAudioFile>d__83:MoveNext () (at Assets/Plugins/crosstales/RTVoice/Scripts/Provider/BaseCustomVoiceProvider.cs:329)

I cannot find any info online on what this could mean, could you please advise where to start? Thanks!

 

Hi Stefan, do you plan to add support to Elevenlabs text to speech, probably the best and newest tts ? Thanks
Massimo

 

We've just released version 2023.2.0 of RT-Voice.
Main changes:

• macOS/iOS: genders for new voices added
• AWS Polly: 28 new voices added and option for additional voices
• Azure: support for speaking styles/roles and option for additional voices
• Google Cloud: option for additional voices
• Voice-class has a new variable called "isNeural"
• Threading improved
• Enums improved
• Updated to Common 2023.2.2

 

We heard a lot of it, maybe we will add it someday, but no promises ;-)

 

I'm new to RT-Voice, but loving it so far! I've run into one strange new problem that I may need a little help with. Every rare once in a while my game will crash. The log and my code are below. This crash was not happening until I added the RT-Voice package, so I suspect it's from this update.

Has anyone else had this issue?


Processing speech: so chat, how was your day?
receiving comment from: ShortRider message: someone is stuck above blue pipe
receiving comment from: Federal Pursuit Vehicle message: Nya uwu nuzzles you
receiving comment from: Gourav Kumar message: 3
receiving comment from: Hecadodron message: 1
receiving comment from: James Copley - Resoldier message: 1
C:\build\output\unity\unity\Modules\Audio\Public\sound\SoundChannel.cpp(420) : Error executing m_FMODChannel->setFrequency(m_Pitch * m_DopplerPitch * m_BaseFrequency) (The specified channel has been reused to play another sound. )
Crash!!!
SymInit: Symbol-SearchPath: '.;E:\UnityBuilds\ChaosLeagueLiveYT_Local;E:\UnityBuilds\ChaosLeagueLiveYT_Local;C:\Windows;C:\Windows\system32;', symOptions: 534, UserName: 'merob'
OS-Version: 10.0.0
E:\UnityBuilds\ChaosLeagueLiveYT_Local\ChaosLeagueLiveYT.exe:ChaosLeagueLiveYT.exe (00007FF78DCB0000), size: 675840 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 2021.3.17.33041
C:\Windows\SYSTEM32\ntdll.dll:ntdll.dll (00007FFDF0580000), size: 2134016 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.22000.778
C:\Windows\System32\KERNEL32.DLL:KERNEL32.DLL (00007FFDEEB80000), size: 774144 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.22000.708
C:\Windows\System32\KERNELBASE.dll:KERNELBASE.dll (00007FFDEDD60000), size: 3641344 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.22000.795

-----------------------------------------------------------------------------------
my speech code:

Code (CSharp):

1. public class MyTextToSpeech : MonoBehaviour
2. {
3.     public AudioSource speechAudioSource;
4.  
5.     public static MyTextToSpeech inst;
6.     [SerializeField] private string textToSpeak;
7.     [SerializeField] private bool speakButton;
8.  
9.     [SerializeField] public bool speechEnabled;
10.  
11.     public void Start()
12.     {
13.         inst = this;
14.     }
15.  
16.     private void OnValidate()
17.     {
18.         if (speakButton)
19.         {
20.             speakButton = false;
21.             Speech(textToSpeak);
22.         }
23.     }
24.  
25.     public void SetSpeechEnabled(bool _enabled)
26.     {
27.         speechEnabled = _enabled;
28.     }
29.  
30.     public void Speech(string textToSpeak)
31.     {
32.         textToSpeak = textToSpeak.Trim();
33.  
34.         if (!speechEnabled || textToSpeak.Length <= 0)
35.             return;
36.  
37.         string pattern = "<.*?>"; // Matches everything between '<' and '>'
38.         textToSpeak = Regex.Replace(textToSpeak, pattern, string.Empty);
39.  
40.  
41.         pattern = @"[^\w\s.,?!]";
42.  
43.         // Remove non-letter, non-number, and non-punctuation characters using regular expression
44.         textToSpeak = Regex.Replace(textToSpeak, pattern, "");
45.  
46.  
47.         Debug.Log("Processing speech: " + textToSpeak);
48.         Speaker.Instance.Speak(textToSpeak);
49.  
50.     }
51.  
52. }

 

Hi

Unfortunately, I've never seen this error - what's your Unity version?
However, you could try using Speaker.Instance.SpeakNative instead of "Speaker.Instance.Speak" to test if it helps.


Cheers
Stefan

 

I'm in Unity 2021.3.17f1.

Thank you for the response! I'm in 2021.3.17f1, but my apologies, I think I misattributed the cause of the sound error.
After doing some more digging, I believe this silly mistake I made (unrelated to TTS) was the cause of the crash. RT-Voice must be running smoothly!

 

Question about 3rd party providers, are they included in https://assetstore.unity.com/packages/tools/audio/rt-voice-pro-41068 package, or needed to be purchased separately?

I couldn't figure out that from documentation, and from "package content" section it doesn't look like Azure provider is included in the package.

 

Hi

Thank you for your interest in our asset!
Azure is included in our RT-Voice, but obviousely without subscription
The only 3rd party providers that have to be bought seperately are the native WebGL-support and Klattersynth.

I hope this helps you further.


Cheers
Stefan

 

Thank you, purchasing RT-Voice now

 

when speaking Chinese, it is fine in windows platform, yet not work in android.
when speaking English-Chinese mixed sentences, it is fine in windows, yet only English part is spoke out in android.
when speaking English, both fine.
The scripts as following:
Speaker.Instance.Speak("sentence for speaking out", audiosource, Speaker.Instance.VoiceForCulture("zh"))

 

Hi

RT-Voice uses the installed TTS on your device, which also depends on the manufacturer. E.g. Samsung and many "cheap" brands use their own engine or something like Pico etc. Those engines may not provide sufficient support for Chinese.
However, if you install the Google default TTS engine, you will get a variety of great Chinese voices.
The other option is using one of the supported online providers.


Cheers
Stefan

 

Thank Stefan.

 

I installed Google TTS engine v1.7 on my android device, and have chosen the Google engine in the settings of TTS output. However, it still donot speak chinese. Is there other settings on my android device, or on the unity editing, such as the "Android Settings Engine" in the RTVoice object.

 

You can get all installed TTS engines under Android with "Speaker.Instance.Engines" and set it with "Speaker.Instance.AndroidEngine".
If that doesn't help, do you see any entries in Logcat?