RT-Voice - Run-time text-to-speech solution

Hi Tony & Stefan,

I have imported the revised integration, but the RT Voice Actor script still returns to the default male voice when using "name" as preferences. I have also tried using windows' Paulina, with the same return to default male voice result.

The "gender" choice does work.

Outside of the Dialogue System, note that I am using voice recognition to obtain responses from my AI chatbot via RT Voice. This setup works perfectly with RT Voice named voices. I am using the following line of code, with the AiName as a public string variable (i.e., Caitlin).

Speaker.Instance.Speak(speaktext, audioSource, Speaker.Instance.VoiceForName(AiName));

Perhaps we are using a different process?

Thanks!
Pascal

 

@TonyLi Are you waiting for the "OnVoicesReady"-callback before assigning any voices?

 

Hi Stefan - Since RTVoiceActor isn't necessarily active when Speaker is initializing, the latest version of the integration waits for areVoicesReady instead of waiting for OnVoicesReady.

@StudioIncorrect - Please make sure you're using the file DS_RTVoiceSupport_2022-01-31a.unitypackage

 

Hi Tony,

I used the DS_RTVoiceSupport_2022-01-31 file.

There does not appear to be a DS_RTVoiceSupport_2022-01-31a file at https://www.pixelcrushers.com/dialogue-system/dialogue-system-extras/

 

Try refreshing the page. Maybe your browser has cached an old copy.

If that doesn't fix the issue, here's some insight into how RTVoiceActor chooses a voice:

• First it looks for a matching voice in the culture that matches the Dialogue System's current language setting. For example, if you've set the Dialogue Manager's Language to 'es', it will look check Speaker.Instance.VoicesForCulture("es").
• Failing that, or if no language is set, it will look for a matching voice in all available voices (Speaker.Instance.Voices).
• And, failing that, it will use the first voice in the list of available voices.

To match a voice:

• If the RTVoiceActor's gender is set to Male or Female, it matches gender; otherwise it ignores gender.
• The voice's age must be within the RTVoiceActor's age range.
• If the RTVoiceActor specifies a name, the name must be contained with the voice's name.

 

Hi Tony & Stefan,

Thank you for the refresh hint, I imported the new version and the voices (both window's and Cerevoice's) work perfectly when using "name" as preferences.

Thanks again to the both of you, it is much appreciated!
Pascal

 

Hi,
Please help in this issue coming up while building unity exported ios app using xcode. The filename is RTVoiceIOSBridge.mm. Attaching an image for ref.

Thanks

 

Hi

Please update to the latest version from the store.


Cheers
Stefan

 

Done and solved.
Thanks

 

Hey so I feel like I'm almost there but I'm getting a lang error code when I try to pass my Speak Function.

My Speak Call as follows:

Code (CSharp):

1.     public void Speak(string _result) {
2.         if (!string.IsNullOrEmpty(uid)) {
3.             Speaker.Instance.Silence(uid);
4.         }
5.         uid = useNative ? Speaker.Instance.SpeakNative(_result, SpeakerVoice, speechRate, speechPitch, speechVolume) : Speaker.Instance.Speak(_result, speechAudioSource, SpeakerVoice, true, speechRate, speechPitch, speechVolume);
6.     }

Error Code as follows:

Code (CSharp):

1. Could not speak the text: Wrapper {Uid='fae78efa-8de3-4aee-94f3-f36c62d860c4', Text='Selected: Microsoft Zira Desktop', Source='True', Voice='Microsoft Zira Desktop (, FEMALE)', SpeakImmediately='True', Rate='1', Pitch='1', Volume='1', OutputFile='', ForceSSML='True', isPartial='False', Created='2/8/2022 3:31:04 PM'}
2. Exit code: 400
3. Could not speak: System.FormatException: 'speak' requires attribute 'lang'.
4.    at System.Speech.Internal.Synthesis.VoiceSynthesis.Speak(Prompt prompt)
5.    at Crosstales.RTVoice.TTSWrapper.speak(String speechText, SpeechSynthesizer speechSynthesizer)
6.  
7. UnityEngine.Debug:LogError (object)
8. Crosstales.RTVoice.Provider.VoiceProviderWindows/<Speak>d__39:MoveNext () (at Assets/Plugins/crosstales/RTVoice/Scripts/Provider/VoiceProviderWindows.cs:336)
9. UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)
10.  

I try to get all my voices beforehand and I am trying to select a voice automatically when a Culture/Gender Selection results in a list of Voices with Count == 1

Code (CSharp):

1.     void Start() {
2.         language = Application.systemLanguage;
3.  
4.         //UI Start
5.         genders.Clear();
6.         genders.Add(Crosstales.RTVoice.Model.Enum.Gender.MALE.ToString());
7.         genders.Add(Crosstales.RTVoice.Model.Enum.Gender.FEMALE.ToString());
8.         genders.Add(Crosstales.RTVoice.Model.Enum.Gender.UNKNOWN.ToString());
9.         genderSelectDropdown.options.Clear();
10.         genderSelectDropdown.AddOptions(genders);
11.         if (gender == Crosstales.RTVoice.Model.Enum.Gender.MALE) {
12.             genderSelectDropdown.value = 0;
13.         } else if (gender == Crosstales.RTVoice.Model.Enum.Gender.FEMALE) {
14.             genderSelectDropdown.value = 1;
15.         } else if (gender == Crosstales.RTVoice.Model.Enum.Gender.UNKNOWN) {
16.             genderSelectDropdown.value = 2;
17.         }
18.         //Speech Interface Initialization
19.         if (speechRate <= 0f) { speechRate = 1.0f; }
20.         if (speechPitch <= 0f) { speechPitch = 1.0f; }
21.         if (speechVolume < 0f) { speechVolume = 1.0f; }
22.  
23.         Speaker.Instance.OnProviderChange += onProviderChange;
24.         Speaker.Instance.OnVoicesReady += onVoicesReady;
25.         Speaker.Instance.OnSpeakStart += onSpeakStart;
26.         Speaker.Instance.OnSpeakComplete += onSpeakComplete;
27.  
28.         cultures = Speaker.Instance.Cultures;
29.         supportsSSML = Speaker.Instance.isSSMLSupported;
30.  
31.  
32.         //Speech Wrapper Start
33.         speechAudioSource = GetComponent<AudioSource>();
34.     }
35.  
36.     private void Update() {
37.  
38.         if (Speaker.Instance.areVoicesReady) {
39.  
40.             if (culture != null && !lastCulture.Equals(culture)) {
41.                 BuildVoicesList();
42.  
43.                 lastCulture = culture;
44.  
45.                 cultures = Speaker.Instance.Cultures;
46.             }
47.         }
48.     }
49.  
50.     private void onVoicesReady() {
51.         //Debug.Log("+++ Voices ready... +++");
52.         lastCulture = "ForceUpdate"; //force update      
53.  
54.         cultureSelectDropdown.options.Clear(); //Pull in a fresh list.
55.         cultures = Speaker.Instance.Cultures;
56.         cultureSelectDropdown.AddOptions(cultures); //Populate dropdown.
57.         Debug.Log(cultures.Count + " Cultures Available");
58.  
59.         Speaker.Instance.VoiceForCulture("en", 0);
60.     }
61.  
62.     public void BuildVoicesList() {
63.         if (!Speaker.Instance.areVoicesReady) {
64.             Debug.Log("Speaker.cs - Voices not ready");
65.             return;
66.         } else {
67.             //Inspector-Side List pulled from Speaker.cs centrally
68.             //voices.Clear(); //Clear our local list
69.             voices = Speaker.Instance.VoicesForGender(gender, culture); //Check the master list with new Gender/Culture
70.  
71.             //Handle UI Dropdown Refresh
72.             voiceSelectDropdown.options.Clear();
73.             List<string> availableVoices = new List<string>();
74.             foreach (Crosstales.RTVoice.Model.Voice voice in voices) {
75.                 availableVoices.Add(voice.Name);
76.             }
77.             voiceSelectDropdown.AddOptions(availableVoices);
78.  
79.             Debug.Log("Voices refreshed, " + voices.Count + " Voices found for " + gender + ", " + culture);
80.  
81.             //If there is only one per the selected parameters, go ahead and use it.
82.             if (voices.Count == 1) {
83.                 Speaker.Instance.VoiceForName(voices[0].Name);
84.                 SpeakerVoice.Name = voices[0].Name;
85.                 SpeakerVoice.Description = voices[0].Description;
86.                 SpeakerVoice.Gender = voices[0].Gender;
87.                 SpeakerVoice.Age = voices[0].Age;
88.                 SpeakerVoice.Identifier = voices[0].Identifier;
89.                 SpeakerVoice.Vendor = voices[0].Vendor;
90.                 SpeakerVoice.Version = voices[0].Version;
91.                 SpeakerVoice.SampleRate = voices[0].SampleRate;
92.                 textToSpeak = "Selected: " + voices[0].Name;
93.                 Speak(textToSpeak);
94.             }
95.         }
96.     }
97.  
98.  

Not sure I understand, feel like I'm only a parameter away from getting it to speak. I had to change up the wrapper style because this speech is for the output of an AI and so it needs a direct string pass to a speak method instead of looking at an input or text box.

Not sure how to force a language setting for the Speaker to pick up?

 

Hi!

I think you missed the culture for the voice (as the error requires attribute 'lang' indicates).
You should use the constructor from Voice and add all necessary parameters:

I also think line 83 in your code is not needed.

I hope this helps you further, please let me know.


So long,
Stefan

 

Perfect. Much appreciated. Super happy with this tool's universality overall. Thanks for being quick to reply.

Best,
Jason

 

What should I do about crackles/stutters in the audio? Basically never happens on first clip, but eventually all the audio stutters a bit after enough speech calls. I can see it always disposes the wrapper by uid at the end when it calls nonnative speech, I would prefer nonnative for many reasons so

Should I be passing a sample rate in to the voice constructor? My devices vary. I'm on a headset and I'm running at 48000 rather than 44100 generally, have a bunch of audio equipment I run also, but I need to prepare for 44100 or whatever the system sample rate is as well.

Thanks,
Jason

 

Hi Jason

Unfortunately, I can't exactly follow your issue with "crackles/stutters in the audio". We never experienced it by ourself and it wasn't reported by anyone else. Therefore, can you please give me more details and answer my questions?

1. Version of Unity?
2. Version of RT-Voice?
3. Editor OS?
4. Current build platform?
5. Does it happen in the Editor or in builds?
6. What's the used voice?
7. After how many Speak-calls starts the issues?

Furthermore, all voices I know are capped at 24000Hz, most are 22050Hz or even lower and all are mono.
It's better to not enter any values for all optional parameters of the Voice constructor.


Cheers
Stefan

 

2020.3.26f1, Windows 10 Pro, RT Voice 2021.3.0, Build Platform Windows, Inside Editor, Microsoft Zira Desktop, about 3 calls, inconsistent though.

Probably just my system.

I am not setting any sample rate on voices or doing any effects. Will report back when I do builds and let you know if still happens or not.

Thanks~

 

Hey Stefan, thanks so much for your help.

One more thing, I am trying to subscribe to the various delegate callbacks for word/phoneme/viseme as follows:

Code (CSharp):

1.        
2.         Speaker.Instance.OnSpeakCurrentWord += onSpeakCurrentWord;
3.         Speaker.Instance.OnSpeakCurrentWordString += onSpeakCurrentWordString;
4.         Speaker.Instance.OnSpeakCurrentPhoneme += onSpeakCurrentPhoneme;
5.         Speaker.Instance.OnSpeakCurrentViseme += onSpeakCurrentViseme;
6.  

I have them all set to take the standard method inputs on subscribe:

Code (CSharp):

1.     //For catching/handling active speech events.
2.     private void onSpeakCurrentWord(Wrapper wrapper, string[] speechTextArray, int wordIndex) {
3.         if (debugWord) { Debug.Log(string.Concat("onSpeakCurrentWord: " + speechTextArray + ", ") + " || Word Index: " + wordIndex + "|| SpeechTime: " + wrapper.SpeechTime); }
4.     }
5.     private void onSpeakCurrentWordString(Wrapper wrapper, string wordString) {
6.         if (debugWordString) { Debug.Log("onSpeakCurrentWordString: " + wordString + "|| SpeechTime: " + wrapper.SpeechTime); }
7.     }
8.     private void onSpeakCurrentPhoneme(Wrapper wrapper, string phoneme) {
9.         if (debugPhoneme) {
10.             Debug.Log("onSpeakCurrentPhoneme: " + phoneme + "|| SpeechTime: " + wrapper.SpeechTime);
11.         }
12.     }
13.     private void onSpeakCurrentViseme(Wrapper wrapper, string viseme) {
14.         if (debugViseme) {
15.             Debug.Log("onSpeakCurrentViseme: " + viseme + "|| SpeechTime: " + wrapper.SpeechTime);            
16.         }
17.     }

But, I'm not getting any Debug logs when I check my debugs true. I *do* get the other callbacks for setting up, starting and completing speech though:

Code (CSharp):

1.         Speaker.Instance.OnProviderChange += onProviderChange;
2.         Speaker.Instance.OnVoicesReady += onVoicesReady;
3.         Speaker.Instance.OnSpeakStart += onSpeakStart;
4.         Speaker.Instance.OnSpeakComplete += onSpeakComplete;
5.  

These all work.

I am trying to get to phonemes and visemes so I can assess their content and timing to do lip sync, I know support varies on platforms so I'm trying to look at every method available but those four delegates don't print, yet.

Again thank you Sir.

 

Hi

I would recommend that you're using "Speaker.Instance.SpeakNative" if you like to receive those callbacks.
There is a workaround for the "Speak"-call as well, but it's more complicated. If you like to try it, please take a look at our demo scene 04-Simple.

I hope this helps you further.


Cheers
Stefan

 

Hello,

I've been making use of RT-Voice, and it's been a great means of adding text-to-speech and accessibility options to my programs.

I do just have a quick question, just to make sure I am not overlooking a feature.

I am looking to dynamically change the volume and rate of the text-to-speech based on user input. As far as I can tell, these variables can only be set when the text is initially voiced.

I've established a solution that effectively just 'restarts' the voice to the most recent word or phrase whenever the volume or playback values are adjusted.

But, I just wanted to quickly check if I was missing any more accessible means of accomplishing this while any given voice is playing a block of text. Not a huge issue if not possible, as I've already established a manageable alternative.

Thanks.

 

Hi

If you are using the "Speak"-call, the result is an AudioClip for an AudioSource, which has pitch and volume and can therefore be adjusted at "speak time"
Apart from that, there are no other features that would allow it, since basically RT-Voice prepares the text and sends it to the TTS which generates the complete audio - so no changes without restarting the whole speak.

I hope this helps you further.


Cheers
Stefan

 

Thanks,

That should allow me to adjust the volume accordingly, although the playback probably isn't as simple. Regardless, restarting from the current word seems to be more than enough to allow for meaningful adjustments to the playback / volume while running.

Thanks a lot, that answers my question.

 

Hey Stefan,

I was looking into smoothing out my means of adjusting the volume and playback rate, and I believe having access to the audio clip result would definitely be beneficial in accomplishing this as you suggested.

However, I can't seem to find where I might retrieve the AudioClip result.

Speaker.Instance.Speak() / Speaker.Instance.SpeakNative both return the speakerUIDs. There is a parameter in the wrapper for an AudioSource, but it doesn't seem to return or set an AudioClip for it. Are these the functions you were referring to?

 

Hello

The second parameter of the "Speak"-call is an AudioSource. If you provide your own AudioSource, the clip will be part of it:
https://www.crosstales.com/media/da...ass_crosstales_1_1_r_t_voice_1_1_speaker.html

I hope this helps you further.


Cheers
Stefan

 

Thanks, that did help.

Grabbing the AudioSource allows for dynamically changing the volume, pitch, and rate with Audio Mixers.

 

Hi Stefan. I have been trying many ways to get samples from the RT Voice AudioSource component on the prefab. I can get no values at all. I started checking by trying to turn all AudioSources off and on and adjusting their volumes. Apparently if I disable the RT Voice AudioSource component the TTS still plays which leads me to believe I am not sampling the audio correctly from the proper source. I am using this to produce faked visemes from some lip sync algorithms by averaging samples in blocks of 512 to create volume curves that sustain or rise or fall quickly. I then pick a viseme shape based on those slopes. Just can't figure out where to get those samples from your system.

 

Hi

You could provide your own AudioSource to the "Speak"-call and then either listen to the "OnAudioFilterRead"-callback or get the data from "AudioSource.GetSpectrumData" / "AudioSource.GetSamples".

I hope this helps you further.


Cheers
Stefan

 

Finally figured out to set useNative to false. Bang zoom..the samples appeared after a few hours of WTF.

EDIT: Actually no. I was elated to see the samples appear when the voice first spoke but as soon as we used the RT Voice to speak the samples from the audioSource were all zero again. I see you have a AudioMixer in the mix so to speak and is on the output of the RTVoice AudioSource..but that does not show any activity during speaking either. So you seem to have some kind of voodoo going on here. I can disable every audioSource in the scene and RTVoice still speaks. So the audio is being generated and piped somewhere but not an audioSource.

 

I apologize for the hassle! I hope it works now for you.

 

Glad you are around. I am perplexed about being able to briefly glimpse the samples like I have when using my own lip sync component prior. ..but as soon as text is sent to RTVoice and generated there are no samples from any audioSource and they can all be disabled and still RTVoice is audible. I want one thing from RTVoice beside the TTS and that is to pipe the audio to an avatars audioSource directly..no mixers..no OS audio pipeline..pure Unity AudioSource.

 

If you don't pass an AudioSource to the "Speak"-method (2nd parameter), RT-Voice will generate one on the fly - this is why you still hear the speech.
So please provide your own source and you will be able to get the samples.

 

Got it..thx.

 

Hi Stefan..and anybody having a clue here.. If you have a standalone, non-internet connected application and want to run MaryTTS from a command line..IOW start it up as a server but is part of the local app install and access that from Unity RT-Voice..how could I do that or what can I look at or read to figure this out? Seems doable as the MaryTTS docs show starting it on PC and OS X desktops with one line in the terminal. Then I would guess I can talk to it via ??? WebRequest? some kind of Interop process?

 

Hello again

If you install and run MaryTTS locally, you could simply use our MaryTTS-provider and add as URL "localhost" and it should work. No need to implement your own provider.

I hope this helps you further.
Have a nice weekend!


Cheers
Stefan

 

Hello RT-Voice staff!

I am wondering if you can help me with your product that I purchased through the asset store. I am developing a timing application and use your module to speak at certain intervals of time. I primarily develop using unity on MacOS, and want to make builds for Windows machines.

When i make a windows build, and run it on even a high speed windows gaming machine, there is a delay in the start of speech, and a delay on triggering the onSpeakCompleted call when I send a Speaker command like this :

Code (CSharp):

1. Speaker.Instance.Speak(speakText, audioSource, voice, true, speakSpeed);
2.  

Where speakText is the text string to speak, audioSource is a proper audioSource type, voice is a Voice object and speakSpeed is the float speak speed.

The voice happens properly, but there seems to be almost a second of delay before it starts speaking. My application is speaking a countdown timer that must match the countdown clock that is shown, which is why this is a problem. It does not have that problem in the runtime of the Mac build.

Here is my info :

Using Unity 2020.3.29f1
Using RT-Voice 2021.3.7

I have looked for any bottlenecks that could be causing delays, but cannot seem to get the voice to start immediately, so I appreciate any help you could lend to determine what is happening.

Thanks,

Tim

 

Hi Tim

I wrote you an email.
This is the answer for other customers:

You could try it with the "SpeakNative"-method or use the "SAPI Unity"-provider (from the "Extras"-folder) in combination with the "PlatformProvider":



Cheers
Stefan

 

Hi Stefan,

Recently we have purchased the RT-Voice asset. And we are building a WebGL application where we are using both Dialogue system and RT-Voice. As you have mentioned in one of the reply that for webgl platform we need to use MaryTTS or WebGL speech synthesis.

So my query is, 1) If we want to use MaryTTS, how to create our own server for runtime only. Instead of using yours test server which runs locally.
2) And if we change the name in RT-Voice Actor for different voice other than default, while running the application we are able to hear only the default male voice. So how to set different voices for multiple NPC's right now testing for single npc.

The document doesn't provide how to create maryTTs server of our own with credentials. Please suggest us the process of achieving it.
And as you mention in one of your reply that request for a development account, does that development account server is free of cost or do we neeed to pay for the marytts server. Can we use the server for production purpose ?


Thank you,
Lakshman

 

RT-Voice Actor is a Dialogue System integration component, so it's supported by Pixel Crushers. Are you using the Dialogue System's latest RT-Voice Support package?

 

Hi

There is a manual called "MaryTTS with HTTPS.pdf" under Documentation in the package which shows you all necessary steps to setup your own server.

The account on our test-server is free (just send us your desired username and invoice), but it's not allowed to use it in production! The traffic from a successful app would be too much for our little server ;-)

I hope this helps you further.


Cheers
Stefan

 

Thank you Tony. It is working fine.

 

Thank you Stefan, we tested our app using MaryTTS and decided to go with WebGL Speech synthesis. As you have mentioned for using WebGL speech synthesis we don't need any server.

 

Hi Stefan,

If I enable the custom provider I'm getting caution msg saying that "Custom provider does not support current platform".
I'm using Unity 2021.1.25f and in build settings I have switched the platform to WebGL. I have attached the screenshot of both inspector and console window.
Do I need to change any settings, and after taking WebGL build custom provider will be disabled. Will it affect the build or internally after the build will it communicate with the custom provider that is WebGL Speech synthesis?



And how to change the default voice as per our requirement, I have downloaded the voices which has been mentioned in the document, but when I mention the name of the desired voice in RT Voice actor component, the default voice is played not the mentioned voice.
Please help me out to fix these issues.

Thank you,
Lakshman

 

The issue has been fixed, instead of WebGL speech synthesis asset we are using Google cloud Text to speech asset. Because GC has wide range of voices to access.
Hence closing this thread.

Thank you,
Lakshman

 

Is it just me, or are prefabs in the package scenes broken for everyone?
Tried importing in new projects on 2019, 2021, and 2022 versions of Unity. Package version 2022.1.0.

Edit: Nevermind. Found a "demos" package that had the components. Not terribly obvious.

 

Hi

I apologize for the hassle it has caused - this issue is fixed in the upcoming release 2022.1.1.


All the best,
Stefan

 

Is there a SSML code for setting voice that works with Azure? I cannot seem to get "<voice name=" working for that provider (the SSML examples tab suggests that it should work).

 

Hi

Please send us an email with your invoice and we will give you access to a fix.


Cheers
Stefan

 

Hi! I´m new to RT-Voice.
Windows 10, Unity 2020. Just opened one of the demo scenes (speech).

My windows system has several voices. Even though I have several spanish ones, I cannot access whole of them
For example, Spanish (Mexican) voices are Sabina and Raúl. But I can only access Sabina.
Same happens with others. I can access some from the demo, and not others.

Any hints?
Thanks!

 

Hi

Please see this post:
https://forum.unity.com/threads/rt-...o-speech-solution.340046/page-22#post-5301570


Cheers
Stefan

 

Thanks! I had missed that post.

Eduardo

 

It is typed when I will speak?, similar to Google Assistant

 

Hi

Unfortunately, I'm not sure what you exactly mean. It is possible to show the current spoken word, but only under Windows and iOS.

I hope this is the answer you were looking for, otherwise please send us more details.


Cheers
Stefan