RT-Voice - Run-time text-to-speech solution

When using Azure, is Speaker Voice the object to use to define which voice to use?
I'm trying to distill the Azure example into a single female voice for our avatar.
However I'm sure I'm missing something.

This example requests text from ChatGPT, then uses RT-Voice and SALSA to speak/lipsync.
It works with the Speaker.Instance.Speak(textArea.text, Source, Speaker.Instance.VoiceForCulture("en"));
but returns an invalid audio file if I try to define a SpeakerVoice.

Code (CSharp):

1. var completionResponse = await openai.CreateCompletion(new CreateCompletionRequest()
2.             {
3.                 Prompt = Instruction,
4.                 Model = "text-davinci-003",
5.                 MaxTokens = 128
6.             });
7.  
8.             textArea.text = completionResponse.Choices[0].Text;
9.             setNarrationText(textArea.text);
10.             Instruction += $"{completionResponse.Choices[0].Text}\nQ: ";
11.  
12.             Debug.Log("+++ Speaking line for RT Voice");
13.             // Speaker.Instance.Speak(textArea.text, Source, Speaker.Instance.VoiceForCulture("en"));
14.             if (!string.IsNullOrEmpty(uid))
15.                 Speaker.Instance.Silence(uid);
16.  
17.             uid = GUISpeech.isNative ? Speaker.Instance.SpeakNative(textArea.text, SpeakerVoice, GUISpeech.Rate, GUISpeech.Pitch, GUISpeech.Volume) : Speaker.Instance.Speak(textArea.text, Source, SpeakerVoice, true, GUISpeech.Rate, GUISpeech.Pitch, GUISpeech.Volume);

 

Hi

Have you tried using Speaker.Instance.VoiceForName("Libby")?
Please don't define your own voice, use the ones that are already provided.


Cheers
Stefan

 

Ah, thank you. At every turn, you've coded RT-Voice to be *very* easy to implement.

 

Hi, I just started using this plugin, I ran your Android Demo, it works fine, but when I add the plugin into the Unity project, then package the demo scenes, after running on the mobile, I encountered some strange things. The word <speak> will be added before and after each speech. There is no problem on the computer, it only appears on the mobile phone!
I don't know how to solve this problem, waiting for your reply, thank you!

 

Hi

Which version of our plugin are you using?


So long,
Stefan

 

the plugin version is 2023.1.0, Unity version is 2022.2.8

 

In addition, if like this, Speaker.Instance.Speak(Text), the Voice is null, does it mean that the Voice in the device will be automatically matched ?
And the Voice is random, or according to the language of the current device, or the current What about the language type of the input text?
For example, if the current text is Japanese and the device language is German, what language is the translated voice in?

 

Hi

Normally, all TTS-systems from Android since version 7 support SSML, but as it seems, the engine on your device doesn't and <speak> is spoken... You could either make sure that the Google-TTS engine is installed under Android or enable "Auto Clear Tags" under "Advanced Settings" in the RTVoice-object.

To prevent using a random voice, you could add it to the Speak-command, like this:
Speaker.Instance.Speak("text", null, Speaker.Instance.VoiceForName("yourVoiceName"));

I hope this helps you further.


Cheers
Stefan

 

Hello- longtime user

I've recently installed several English voices on my Windows 10 install- I installed the voices and restarted Windows- they all work in Windows etc


Rt Voice does not recognize the newly added voices- this is all RtVoice recognizes


I tried hitting reload of course etc-

I bought a Cereproc voice and installed it to see if that would work- it shows in RtVoice but not in the Windows voices- when I try to test the Cereproc voice in RtVoice it throws an error 400- it starts to play then stops and throws that error-



Any ideas on how to get RtVoice to recognize/use these other voices? Thank you

 

Hello dear longtime user

Since version 8, Windows contains two TTS-systems: one for "classic" (= standalone apps) with SAPI. The other one (your first picture) is for UWP apps.
So, if you want to access them, you have to build an UWP (WSA) app.
Alternatively, you could add classic SAPI-voices for standalone. Please see chapter 6 of the documentation:
https://www.crosstales.com/media/data/assets/rtvoice/RTVoice-doc.pdf

Unfortunately, we don't own Cereproc voices, therefore I can't say what's going on :-(
However, you could try eSpeak or a supported online solution like MaryTTS, AWS Polly, Azure or Google.

I hope this helps you further.


Cheers
Stefan

 

Thank you for your help- I'll give the other voices a try.

 

Hello, could someone explain how to embed MaryTTS as library into unity game to work offline? Is there some demo? I saw a note in the documentation "Standalone: MaryTTS could be bundled with the app installer and run locally", but I cannot find more details to it.

Thank you!

 

Hi

You could download MaryTTS to your PC (e.g. the 5.2 Runtime-version):
https://marytts.github.io/download/index.html

Then extract it and run "marytts-server" under "bin" to start the server (Java has to be installed on your machine).
After that, configure MaryTTS in Unity and press "Play":


For an actual build, you have to bundle MaryTTS with the desired voices, we would suggest creating a setup for your app or extract and run it from your app itself. There are many options, it's up to you

I hope this helped you further.


Cheers
Stefan

 

Hi Stefan,
Thanks for the great asset.
A simple question:
Are visemes of your API (on Windows platform) exactly the same 22 ones as in this Microsoft doc:
https://learn.microsoft.com/en-us/a...ots=programming-language-csharp&tabs=visemeid
I am asking since I want to convert viseme-related features from an existing Windows app to Unity app, initially only on Windows platform.
Thank you!

 

Is it a suitable application to make the mouth play realistically while speaking the api data we will give using text to speech?

 

I am unable to save audio clip the android persistent data path. Its working fine in the unity editor. But not in the android build. Can you please help with this. Is it possible in this package

 

I'm try to get an animal crossing style effect to my text voices. But for that I need to be able to crank up the pitch to a high value to get a squeaky chirpy voice effect. I bought the plugin for trying this purpose, but from my tests I cannot increase the pitch beyond 2. I tried modifying the code to remove this clamp, but I couldn't get the high pitch effect in the end. Can you point me to how I can remove the clamp value of the pitch so I can tinker with higher values ?

Thanks!

 

Hi,


I love your product RY-Voice PRO, it’s a wonderful.

I’ve been satisfied your nice work for TTS on Android except one problem.

When I try to connect my app to TV via cable, wireless or set-top box itself, the sound from your asset doesn’t work.

Very curios situation. Definitely the asset works well on the app in mobile. But mirroring or cabling to any TV makes problem about no sound.

All the other application looks fine, sound with mirroring, for instance YouTube or any other app.

Probably I missed some parameter setting for sound of TV connection

Please guide me for the sound on TV. With your asset.

 

Dear all,

I have followed documentation to use Azure API. But at first attempt I get this:

The generated audio file is invalid: Wrapper {Uid='158fac69-3929-4162-8fa0-a4437275449e', Text='Hello', Source='True', Voice='Catherine (en-AU, FEMALE)', SpeakImmediately='True', Rate='1', Pitch='1', Volume='1', OutputFile='', ForceSSML='True', Created='11/06/2023 21:03:13'}
UnityEngine.Debug:LogError (object)
Crosstales.RTVoice.Provider.BaseCustomVoiceProvider/<playAudioFile>d__83:MoveNext () (at Assets/Plugins/crosstales/RTVoice/Scripts/Provider/BaseCustomVoiceProvider.cs:329)

I cannot find any info online on what this could mean, could you please advise where to start? Thanks!

 

Hi Stefan, do you plan to add support to Elevenlabs text to speech, probably the best and newest tts ? Thanks
Massimo

 

We heard a lot of it, maybe we will add it someday, but no promises ;-)

 

I'm new to RT-Voice, but loving it so far! I've run into one strange new problem that I may need a little help with. Every rare once in a while my game will crash. The log and my code are below. This crash was not happening until I added the RT-Voice package, so I suspect it's from this update.

Has anyone else had this issue?


Processing speech: so chat, how was your day?
receiving comment from: ShortRider message: someone is stuck above blue pipe
receiving comment from: Federal Pursuit Vehicle message: Nya uwu nuzzles you
receiving comment from: Gourav Kumar message: 3
receiving comment from: Hecadodron message: 1
receiving comment from: James Copley - Resoldier message: 1
C:\build\output\unity\unity\Modules\Audio\Public\sound\SoundChannel.cpp(420) : Error executing m_FMODChannel->setFrequency(m_Pitch * m_DopplerPitch * m_BaseFrequency) (The specified channel has been reused to play another sound. )
Crash!!!
SymInit: Symbol-SearchPath: '.;E:\UnityBuilds\ChaosLeagueLiveYT_Local;E:\UnityBuilds\ChaosLeagueLiveYT_Local;C:\Windows;C:\Windows\system32;', symOptions: 534, UserName: 'merob'
OS-Version: 10.0.0
E:\UnityBuilds\ChaosLeagueLiveYT_Local\ChaosLeagueLiveYT.exe:ChaosLeagueLiveYT.exe (00007FF78DCB0000), size: 675840 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 2021.3.17.33041
C:\Windows\SYSTEM32\ntdll.dll:ntdll.dll (00007FFDF0580000), size: 2134016 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.22000.778
C:\Windows\System32\KERNEL32.DLL:KERNEL32.DLL (00007FFDEEB80000), size: 774144 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.22000.708
C:\Windows\System32\KERNELBASE.dll:KERNELBASE.dll (00007FFDEDD60000), size: 3641344 (result: 0), SymType: '-deferred-', PDB: '', fileVersion: 10.0.22000.795

-----------------------------------------------------------------------------------
my speech code:

Code (CSharp):

1. public class MyTextToSpeech : MonoBehaviour
2. {
3.     public AudioSource speechAudioSource;
4.  
5.     public static MyTextToSpeech inst;
6.     [SerializeField] private string textToSpeak;
7.     [SerializeField] private bool speakButton;
8.  
9.     [SerializeField] public bool speechEnabled;
10.  
11.     public void Start()
12.     {
13.         inst = this;
14.     }
15.  
16.     private void OnValidate()
17.     {
18.         if (speakButton)
19.         {
20.             speakButton = false;
21.             Speech(textToSpeak);
22.         }
23.     }
24.  
25.     public void SetSpeechEnabled(bool _enabled)
26.     {
27.         speechEnabled = _enabled;
28.     }
29.  
30.     public void Speech(string textToSpeak)
31.     {
32.         textToSpeak = textToSpeak.Trim();
33.  
34.         if (!speechEnabled || textToSpeak.Length <= 0)
35.             return;
36.  
37.         string pattern = "<.*?>"; // Matches everything between '<' and '>'
38.         textToSpeak = Regex.Replace(textToSpeak, pattern, string.Empty);
39.  
40.  
41.         pattern = @"[^\w\s.,?!]";
42.  
43.         // Remove non-letter, non-number, and non-punctuation characters using regular expression
44.         textToSpeak = Regex.Replace(textToSpeak, pattern, "");
45.  
46.  
47.         Debug.Log("Processing speech: " + textToSpeak);
48.         Speaker.Instance.Speak(textToSpeak);
49.  
50.     }
51.  
52. }

 

Hi

Unfortunately, I've never seen this error - what's your Unity version?
However, you could try using Speaker.Instance.SpeakNative instead of "Speaker.Instance.Speak" to test if it helps.


Cheers
Stefan

 

I'm in Unity 2021.3.17f1.

Thank you for the response! I'm in 2021.3.17f1, but my apologies, I think I misattributed the cause of the sound error.
After doing some more digging, I believe this silly mistake I made (unrelated to TTS) was the cause of the crash. RT-Voice must be running smoothly!

 

Question about 3rd party providers, are they included in https://assetstore.unity.com/packages/tools/audio/rt-voice-pro-41068 package, or needed to be purchased separately?

I couldn't figure out that from documentation, and from "package content" section it doesn't look like Azure provider is included in the package.

 

Hi

Thank you for your interest in our asset!
Azure is included in our RT-Voice, but obviousely without subscription
The only 3rd party providers that have to be bought seperately are the native WebGL-support and Klattersynth.

I hope this helps you further.


Cheers
Stefan

 

Thank you, purchasing RT-Voice now

 

when speaking Chinese, it is fine in windows platform, yet not work in android.
when speaking English-Chinese mixed sentences, it is fine in windows, yet only English part is spoke out in android.
when speaking English, both fine.
The scripts as following:
Speaker.Instance.Speak("sentence for speaking out", audiosource, Speaker.Instance.VoiceForCulture("zh"))

 

Hi

RT-Voice uses the installed TTS on your device, which also depends on the manufacturer. E.g. Samsung and many "cheap" brands use their own engine or something like Pico etc. Those engines may not provide sufficient support for Chinese.
However, if you install the Google default TTS engine, you will get a variety of great Chinese voices.
The other option is using one of the supported online providers.


Cheers
Stefan

 

Thank Stefan.

 

I installed Google TTS engine v1.7 on my android device, and have chosen the Google engine in the settings of TTS output. However, it still donot speak chinese. Is there other settings on my android device, or on the unity editing, such as the "Android Settings Engine" in the RTVoice object.

 

You can get all installed TTS engines under Android with "Speaker.Instance.Engines" and set it with "Speaker.Instance.AndroidEngine".
If that doesn't help, do you see any entries in Logcat?

 

Thank Stefan. It works. However, on android, there is a 10 or more secondes delay. Each time when speaking new text, the time delay occured.

I tried another TTS engine, XunFei (chinese tts), which works in other app. It does not work in mine. The script snippet is as follows:

Code (CSharp):

1. Speaker.Instance.AndroidEngine = "com.iflytek.speechcloud";
2. Speaker.Instance.Speak(TXT.text.Trim(), audioplayer, Speaker.Instance.VoiceForCulture("zh"));

 

Hello again

The delay can occur due two problems:

1. It's a network voice (=remote): not much we can do, it can take time, especially on bad connections...
2. Some voices have to be "loaded" before they are ready. You could mitigate this issue by using the "VoiceInitializer" in your first scene (e.g. splash).

You may have to generate the speeches in advance (by setting "speakImmediately" to false).

About the Chinese TTS - unfortunately, I don't know it, but does it show up in the "Engines"-property?


Cheers
Stefan

 

Hi!
Amazing asset!!
I bought it to provide accesibility to blind users, I managed to get it working in the Editor with the windows voices (spanish and english), but I have some issues and I cannot find anything related in the docs.

First: Sometimes, the RTVoice prefab shows that there is no voices, if I click reload and restart the scene it works, it only happens a few times.

Second, and most important: I created a build with my scene, the voice is not working and I can see this in the player logs:

Could not write the TTS-wrapper to the destination: System.IO.DirectoryNotFoundException: Could not find a part of the path "C:\Users\NIGHTC~1\AppData\Local\Temp\NightCouncil\AtlasNegro_Infernum\RTVoiceTTSWrapper.exe".

  at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x00000] in <00000000000000000000000000000000>:0 

  at System.IO.File.InternalWriteAllBytes (System.String path, System.Byte[] bytes) [0x00000] in <00000000000000000000000000000000>:0 

  at Crosstales.RTVoice.Provider.VoiceProviderWindows.get_applicationName () [0x00000] in <00000000000000000000000000000000>:0 

  at Crosstales.RTVoice.Provider.VoiceProviderWindows+<getVoices>d__45.MoveNext () [0x00000] in <00000000000000000000000000000000>:0 

  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00000] in <00000000000000000000000000000000>:0 

  at Crosstales.RTVoice.Provider.VoiceProviderWindows.Load (System.Boolean forceReload) [0x00000] in <00000000000000000000000000000000>:0 

  at Crosstales.RTVoice.Speaker.initProvider () [0x00000] in <00000000000000000000000000000000>:0 

  at Crosstales.RTVoice.Speaker.Awake () [0x00000] in <00000000000000000000000000000000>:0 

UnityEngine.Logger:Log(LogType, Object)

UnityEngine.Debug:LogError(Object)

Crosstales.RTVoice.Provider.VoiceProviderWindows:get_applicationName()

Crosstales.RTVoice.Provider.<getVoices>d__45:MoveNext()

UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

Crosstales.RTVoice.Provider.VoiceProviderWindows:Load(Boolean)

Crosstales.RTVoice.Speaker:initProvider()

Crosstales.RTVoice.Speaker:Awake()

If I go to C:\Users\NIGHTC~1\AppData\Local\Temp\NightCouncil I can see a folder called Atlas Negro_ Infernum created, but not AtlasNegro_Infernum, is the plugin having issues with spaces in the name of the project?

Do I need to do something to get it working in builds?
Thanks!!

 

Hi!

Thank you for using our asset!

The first problem can occur only in some Editor versions and won't impact the builds. Normally hitting "Play" will solve the issue.
About your second point: unfortunately, I can't confirm this issue with spaces. I think it could be Unity related...
What are your versions of Unity and RT-Voice?


Cheers
Stefan

 

Thanks for the support!!

My Unity version is 2021.3.30f1 and my Package Manager says that the RT-Voice PRO version is 2023.2.3

 

Just encountered the same issue as the above user, and figured I'd comment on it as well.

I have multiple projects with the following versions of Unity / RT-Voice Pro that work fine.
Unity Version: 2022.3.10f1
RT-Voice Pro Version: 2022.2.2

I recently upgrade some projects, only changing the RT-Voice Pro Version, and they have the same error.
Unity Version: 2022.3.10f1
RT-Voice Pro Version: 2023.2.3

In this situation, RT-Voice Pro works fine in the editor but does not work in the builds.

The error logs call-out "Could not write the TTS-wrapper to the destination: System.IO.DirectoryNotFoundException".

The full report is the same as the above user. It does seem like the issue is related to spaces in the 'Company Name'. I can post full logs if needed. Reverting to an earlier version on RT-Voice Pro resolves the error messages.

 

In my case I have the issue with the project name "Dark Atlas: Infernum", I had to remove the spaces and the ":" to create a correct folder in the build.

How can I move my RT-Voice Pro to 2022? (or 2021) from the package manager? I only can see the last version

 

I don't believe there's an easy way to revert to an older package. I got lucky that I had some other projects with previous packages I could make use of.

Best solution for the time being is to just remove the spaces / special characters. (Ex: Dark_Atlas_Infernum)

 

We have all versions of our assets - just request a specific version and send us the invoice.
However, we recommend the approach from @nlp_eschipellite - the ":" is the problem (not the spaces).

 

Hi, I'm trying to integrate webgl speech synthesis plugin but I can't make it work, it always say No OS voices found - TTS not possible! The WebGL speech synthesis plugin object is in my Hierarchy already, Am I missing some configuration? btw I'm using the Amplitude demo scene, all versions are latest (RT-Voice, salsa, amplitude) and I'm using unity 2022.3.20f1

 

Hi

For Amplitude in WebGL, you have to use one of the supported online providers like MaryTTS (free), Mimic or Google Cloud.
The WebGL-plugin you're using is generating the audio outside of Unity, so Amplitude won't work.


Regards,
Stefan

 

We've just released version 2024.1.1 of RT-Voice.
Main changes:

• Support for Unity Cloud Build improved
• Updated to Common 2024.1.1

 

Hi

Thank you for your interest in RT-Voice!
Unfortunately, our asset does the opposite, it creates speech from text.


Regards,
Stefan

 

Hi I purchased your asset but am struggling to add a rate and pitch slider to the Simple_Example. Basically I am having text come in from ChatGpt and I need to speak it live and slow it down and pitch it. I tried adapting the other examples but it wont let me change the text on runtime and keeps repeating the original text. I just need something simple with the rate and pitch. Please help! Thank you!

 

Hi

Do you call RT-Voice via script? If so, you can add the rate and pitch to the "Speak"-call. For more, please see the API:
https://www.crosstales.com/media/da...ass_crosstales_1_1_r_t_voice_1_1_speaker.html

The other possibility is to use the component "SpeechText":


I hope this helps you further.


Cheers
Stefan

 

Hi, I am doing it through code as I mentioned I was starting with the Simple_Example. The problem is the rate does not affect text that is being added at runtime.
I call: public void SpeakerA()
{
//Don't speak the text immediately
uidSpeakerA = Speaker.Instance.Speak(textA, SourceA, Speaker.Instance.VoiceForGender(Crosstales.RTVoice.Model.Enum.Gender.MALE, "en"), false, RateSpeakerA);
}

And the rate works for the first text at startup but not when I change the text and call the SpeakerA() method. It seems like a glitch? How do I get the rate to work on text dynamically added at runtime?

 

Hi,
We are trying to Integrate RTVoice Pro + WebGL Synthesis + Salsa lipsync just as shown in the Web Demo app.
I tried making a webGL build with the Amplitude_Demo scene but its not working as expected. Some times the audio itself is not playing and some times the audio comes but no lip sync.