RT-Voice - Run-time text-to-speech solution

Hi Massimo

The pitch is unified in RTV for all TTS systems. This means, the range will always be from 0 to 2 and 1 defining the default.
However, the distribution was wrong - this is now fixed.
The original range in GC is from -20 to +20 and we mapped it like this:

1. RTV-Pitch of 1 = 0 in GC
2. RTV-Pitch of 0 = -20 in GC
3. RTV-Pitch of 2 = +20 in GC

The values between are calculated based on the pitch value.

I hope this is what you need.
If you want to have access to the latest beta, please send me your invoice via email.


Cheers
Stefan

 

Perfect! I'll wait for the next release

 

Ok, but be aware: due holidays of the UAS-team, the update won't be available in the store before mid January 2021.

 

We've just released version 2021.1.0 of RT-Voice.
Main changes:

• Native provider for XBox!
• AWS Polly: volume added
• Google Cloud: pitch is now correctly distributed
• Editor integration improved
• Demo scenes improved
• URLs updated
• Updated to Common 2021.1.0

 

Hello Stefan,

I bought a standalone SAPI Cereproc voice (not their cloud service), installed it successfully on my windows 10, but I have problems using it with RTVoice pro.
I want to use it for realtime speaking in a Unity game (using Unity 2019.4.8f1 and Adventure creator)

First, a general question: How do I choose people to speak in demo scenes like 02-Dialog (which doesn't have buttons or "Voices dropdown" in the inspector)

About the Cereproc voice, the main 2 issues (tested on demo scene 01-Speech) are:
1. Not in native mod: It takes about 10 sec until Cereproc voice start the speech, while Microsoft voices (David, Hazel, Zira) start immediately.
2. In native mod: it doesn't speak and throw an error:
Could not speak the text: Wrapper ...
Exit code: 400
Could not speak: System.NullReferenceException: Object reference not set to an instance of an object.
at System.Speech.Internal.Synthesis.VoiceSynthesis.Speak(Prompt prompt)
at Crosstales.RTVoice.TTSWrapper.speak(String speechText, SpeechSynthesizer speechSynthesizer)

Other demo scenes have same results, demo scene 04-Simple: it speaks but throw the same error.

How should I use Cereproc voice with RTVoice pro?
Can I use CereVoice XML for speech tuning and emotional synthesis?

greetings,
Shayk

 

Hi Shayk

Thank you for investing into RT-Voice!

You can take inspiration from the "Dialog"-class. The main trick is to listen to the callbacks "OnSpeakStart" and "OnSpeakComplete". To differentiate the Speak-calls, use the returned UID. You could use the "VoiceAlias"-class to setup voices for all actors. That's basically all there is to know

About Cereproc: unfortunately, we don't own any voice from them, so we can't test it on our side.
However, some thoughts about your questions:

1. Is this happening only for the first time or every time you call "Speak"? If it's only the first time, you could use the "VoiceInitializer" to get it ready at startup.
2. Can you please enable "Debug" in the configuration of RT-Voice? Afterwards, please post the complete log of the SpeakNative-call.

So long,
Stefan

 

Hi Stefan,
Thanks for your kind reply.

Cereproc voice tested in demo scene 01-Speech:
1. Non native mode: The 10 sec delay before start talking happens every time I change the text it should speak.
Adding Voice Initializer throws same error as described earlier.
2. Native mode: doesn't work, attached is the editor log.

Really hope you can solve this
Greetings,
Shayk

 

Can you please try it with the "SAPI Unity"-provider from the 3rd party-folder?
Does that work?

You can delete the Editor.log from your last post.

 

I didn't find "SAPI Voice" in the 3rd party folder.
I found "SAPI Unity", I extracted it and ran the demo scene that was in it, it display red error on screen "No OS voices found - TTS not possible!" and in the console:
"SAPI Unity is not supported under the current platform!
UnityEngine.Debug:LogError(Object, Object)
Crosstales.RTVoice.SAPI.VoiceProviderSAPI:Load(Boolean) (at Assets/Plugins/crosstales/RTVoice/3rd party/SAPI Unity/Scripts/VoiceProviderSAPI.cs:189)..."
BTW, if I run this demo scene without debug option it crashes unity when stop playing the scene.

I also tried to add the prefab "SAPI Unity" to demo scene 01, it didn't effect the issues.

 

Sorry, my fault - it's SAPI Unity.
However, is your active build platform "Windows"?

 

Yes, It's Windows.

 

That's really strange. This is my Unity 2018.4:


Everything works - can you verify the build settings?
What's your Unity-version?

 

Build Settings - anything else to verify beside the platform=Windows?
My Unity version is 2019.4.8f1 (yours is 2018.4)

 

I did a test with 2019.4.12 and I had to switch the scripting backend to 4.x and it worked.

 

I switched to API compatibility 4.x (The Scripting Backend is Mono), It didn't work (still "No OS voices found - TTS not possible!"), BUT it changed the console error to:
Could not get any voices: System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber (System.String str, System.Globalization.NumberStyles options, System.Number+NumberBuffer& number, System.Globalization.NumberFormatInfo info, System.Boolean parseDecimal) [0x00057] in <fb001e01371b4adca20013e0ac763896>:0
at System.Number.ParseInt32 (System.String s, System.Globalization.NumberStyles style, System.Globalization.NumberFormatInfo info) [0x00013] in <fb001e01371b4adca20013e0ac763896>:0
at System.Int32.Parse (System.String s, System.Globalization.NumberStyles style) [0x0000d] in <fb001e01371b4adca20013e0ac763896>:0
at Crosstales.RTVoice.SAPI.VoiceProviderSAPI.Load (System.Boolean forceReload) [0x0012e] in D:\Unity_2019 projects\Tests assets\RTVoice_2020\Assets\Plugins\crosstales\RTVoice\3rd party\SAPI Unity\Scripts\VoiceProviderSAPI.cs:158
UnityEngine.Debug:LogError(Object, Object)
Crosstales.RTVoice.SAPI.VoiceProviderSAPI:Load(Boolean) (at Assets/Plugins/crosstales/RTVoice/3rd party/SAPI Unity/Scripts/VoiceProviderSAPI.cs:184) ......

 

What's your version of our asset - 2021.10? If not, please update it.

Furthermore, please add this before line 158: "Debug.Log(lang);"
What's the result?

Btw, it's maybe a better idea to continue our discussion via email since it's a very specific problem.

 

Greetings! I'd like to try RT-Voice with Game Creator. Has anyone tried this integration?

 

Hi

I haven't tried but it should work without any problems.
Have you tried it?
For send messages, please use the "LiveSpeaker" as receiver instead of the Speaker.


Cheers
Stefan

 

Hi
I can use salsa with Rtx-Voice for the character move the mouth, but he does not
pronounce none sound?

thanks

 

Can RT-Voice PRO use Google WaveNet voices?

 

What do you mean with "pronounce none sound"?

 

Yes, Google Wavenet-voices are working:



But you will also need Google Cloud Text To Speech.


Cheers
Stefan

 

Hi, i want use Dialogue System,RT Voice and Salsa, i want to get a Lip Sync with the text, but i don't want the character speak (the voice in mute) is possible that?

sorry for my bad english

 

SALSA works based on the audio that's playing through its audio source. What about using SALSA's TextSync instead, which performs lipsync approximation based on text?

 

i didn't know this function. i think i've been away a long time
thanks

 

Has anything else changed in the latest version I've missed in the documentation? Having issues now that not experienced before the update. Effecting iOS and Android, but also getting odd error in player runtime in editor. For example exception that the audio source has been destroyed line 418 of BaseHelper.cs?

Struggling to troubleshoot so just after a pointer that may help. Issues are apparent in part of my project I haven't touched in months. Many Thanks.

 

The issue in BaseHelper has been fixed, thank you for pointing it out!

I'm not sure what you exactly mean by issues with Android and iOS - can you please provide more information, like logs etc.?

 

[QUOTE="I'm not sure what you exactly mean by issues with Android and iOS - can you please provide more information, like logs etc.?[/QUOTE]

Hello Stefan. Thanks for coming back so quickly. Is there a fix I can try.

My issue is happening on screens with a carousel that act as help information for the user. The user can swipe between panels in a carousel and I use RT Voice Pro to give a quick voice over for the panel. I'm finding my first speaker.speak which is called very shortly after the scene loads is throwing an exception on Android as below.

NullReferenceException: Object reference not set to an instance of an object.
Crosstales.RTVoice.Speaker.SpeakWithUID (Crosstales.RTVoice.Model.Wrapper wrapper) (at <00000000000000000000000000000000>:0)
Crosstales.RTVoice.Speaker.Speak (System.String text, UnityEngine.AudioSource source, Crosstales.RTVoice.Model.Voice voice, System.Boolean speakImmediately, System.Single rate, System.Single pitch, System.Single volume, System.String outputFile, System.Boolean forceSSML) (at <00000000000000000000000000000000>:0)
SomeshireWelcomeManager.SpeakPanel (UnityEngine.GameObject gameObject) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.UnityAction`1[T0].Invoke (T0 arg0) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.UnityEvent`1[T0].Invoke (T0 arg0) (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.ResetScroll () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.SetupStartingSettings () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.Awake () (at <00000000000000000000000000000000>:0)


If I prevent the first panel from firing a speaker.speak, but then run through the carousel quickly, I get the following error as below. (I use speaker.instance.silence to end the current speech before starting the next).

'Source' is null: Wrapper {Uid='40878ee1-e199-485c-9a35-d9e312582838', Text='Big Town is a small and friendly international aerodrome with a long runway zero five and too tree. Big Town is fully A T C controlled and has a class D control zone. Entry to the zone is via one of the V R pees. They operate a Tower and Radar frequency, providing a lars servicefrom Big Town Radar. No request for engine start is required here. Neither is a radio check. You are advised to listen to the Ay-tiss but this isn't enforced.', Source='False', Voice='en-gb-x-fis-network (en-gb, UNKNOWN)', SpeakImmediately='True', Rate='1', Pitch='1', Volume='1', OutputFile='', ForceSSML='True', isPartial='False', Created='02/08/2021 15:46:16'}
Crosstales.RTVoice.Provider.<playAudioFile>d__72:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)


I'll try iOS next and see if it causes the same issue. I don't get this in the editor player.

Many Thanks,

Bob.

 

Hello Stefan. Thanks for coming back so quickly. Is there a fix I can try.

My issue is happening on screens with a carousel that act as help information for the user. The user can swipe between panels in a carousel and I use RT Voice Pro to give a quick voice over for the panel. I'm finding my first speaker.speak which is called very shortly after the scene loads is throwing an exception on Android as below.

NullReferenceException: Object reference not set to an instance of an object.
Crosstales.RTVoice.Speaker.SpeakWithUID (Crosstales.RTVoice.Model.Wrapper wrapper) (at <00000000000000000000000000000000>:0)
Crosstales.RTVoice.Speaker.Speak (System.String text, UnityEngine.AudioSource source, Crosstales.RTVoice.Model.Voice voice, System.Boolean speakImmediately, System.Single rate, System.Single pitch, System.Single volume, System.String outputFile, System.Boolean forceSSML) (at <00000000000000000000000000000000>:0)
SomeshireWelcomeManager.SpeakPanel (UnityEngine.GameObject gameObject) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.UnityAction`1[T0].Invoke (T0 arg0) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.UnityEvent`1[T0].Invoke (T0 arg0) (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.ResetScroll () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.SetupStartingSettings () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.Awake () (at <00000000000000000000000000000000>:0)


If I prevent the first panel from firing a speaker.speak, but then run through the carousel quickly, I get the following error as below. (I use speaker.instance.silence to end the current speech before starting the next).

'Source' is null: Wrapper {Uid='40878ee1-e199-485c-9a35-d9e312582838', Text='Big Town is a small and friendly international aerodrome with a long runway zero five and too tree. Big Town is fully A T C controlled and has a class D control zone. Entry to the zone is via one of the V R pees. They operate a Tower and Radar frequency, providing a lars servicefrom Big Town Radar. No request for engine start is required here. Neither is a radio check. You are advised to listen to the Ay-tiss but this isn't enforced.', Source='False', Voice='en-gb-x-fis-network (en-gb, UNKNOWN)', SpeakImmediately='True', Rate='1', Pitch='1', Volume='1', OutputFile='', ForceSSML='True', isPartial='False', Created='02/08/2021 15:46:16'}
Crosstales.RTVoice.Provider.<playAudioFile>d__72:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)


I'll try iOS next and see if it causes the same issue. I don't get this in the editor player.

Many Thanks,

Bob.[/QUOTE]
Can you please show me the code from the SpeakPanel, where you call the "Speak"-method?

 

Hello. Just an update. The same exception is thrown on iOS.

NullReferenceException: Object reference not set to an instance of an object.
Crosstales.RTVoice.Speaker.SpeakWithUID (Crosstales.RTVoice.Model.Wrapper wrapper) (at <00000000000000000000000000000000>:0)
Crosstales.RTVoice.Speaker.Speak (System.String text, UnityEngine.AudioSource source, Crosstales.RTVoice.Model.Voice voice, System.Boolean speakImmediately, System.Single rate, System.Single pitch, System.Single volume, System.String outputFile, System.Boolean forceSSML) (at <00000000000000000000000000000000>:0)
SomeshireWelcomeManager.SpeakPanel (UnityEngine.GameObject gameObject) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.UnityAction`1[T0].Invoke (T0 arg0) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.InvokableCall`1[T1].Invoke (T1 args0) (at <00000000000000000000000000000000>:0)
UnityEngine.Events.UnityEvent`1[T0].Invoke (T0 arg0) (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.InvokeOnSelectionChangeEvent () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.ResetScroll () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.AssignElements () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.SetupStartingSettings () (at <00000000000000000000000000000000>:0)
MagneticScrollView.MagneticScrollRect.Awake () (at <00000000000000000000000000000000>:0)

 

Sorry. Missed you comment about the seeing the line of code.

Speaker.Instance.Speak(textToSpeak, null, Speaker.Instance.VoiceForName(AppManager.Instance.selectedVoiceForATC));

I'm just building a test project to try the same without anything else confusing things.

Bob.

 

Hello Stefan. I have recreated the issue in a simple test project. I basically rebuilt what the real project does with the carousel, firing a method when a the user swipes from one panel to the next. This has always worked until I updated RT Pro.

As only the very first speaker.speak caused the exception, I removed it and found subsequent speaks would work without the exception. Thinking it may be a timing issue, I moved the speaker.speak into a coroutine with a 2 second real-time wait. This worked, so I reduced the wait from 2 to 1 second and it still worked. I tried again with a zero second wait and that still works.

So effectively (as part of case statement)....

case "Panel 1":
textToSpeak = "Some shire is the fictional county in which good ar tee flights take place. " +
"Swipe left and right to learn more about our aerodromes. Use the Menu option to return " +
"to the module list when you're ready. ";
Speaker.Instance.Speak(textToSpeak, null, Speaker.Instance.VoiceForName(AppManager.Instance.selectedVoiceForATC));
break;

became....

case "Panel 1":
textToSpeak = "Some shire is the fictional county in which good ar tee flights take place. " +
"Swipe left and right to learn more about our aerodromes. Use the Menu option to return " +
"to the module list when you're ready. ";
StartCoroutine(SpeakViaCoRoutine(textToSpeak));
break;

with the coroutine...

IEnumerator SpeakViaCoRoutine(string CrTextToSpeak)
{
yield return new WaitForSecondsRealtime(0f);
Speaker.Instance.Speak(CrTextToSpeak, null, Speaker.Instance.VoiceForName(AppManager.Instance.selectedVoiceForATC));
}


I have no idea whilst this now works when the same speak command outside of the coroutine does not. I mention it in case it may throw some light onto the issue for you. I'm worried I've papered over a crack rather than solved the problem.

Is there a fix available for the BaseHelper issue.

Many Thanks,

Bob.

 

This is really strange and we have to investigate it.
Just three additional questions:

1. What's your Unity version?
2. What's the version of the asset?
   
3. Does the error also happen in the Editor?

Maybe it's best to continue our discussion via email.

 

Thanks Stefan. Sorry for the stream of posts.

Unity version is, 2019.14f1 LTS.
Version of the asset is, "# RT-Voice PRO 2021.1.2"
The exception doesn't happen in the editor, just iOS and Android. The other error (BaseHelper) does happen in the editor.

I upgraded to 2021.1.2 a few days ago, which is when I noticed this issue. Noting else has changed on the effected scenes that I am aware.

Appreciate your help.

Bob.

 

No problem! Please contact us via email and we can fix it together

 

I am using the December version of RT-Voice with Unity 2019.4 and windows 7 to create a windows stand-alone version. I have successfully implemented the callbacks to trigger when the speaking starts and ends, but I am unable to get anything to callback from OnSpeakCurrentPhoneme or OnSpeakCurrentViseme. I have included RTVoice.model and .util. Any thoughts on what else I can do that might help? Thank you.

 

Hi

Do you see them working with our demo-scene "03-Simple_Native"?


Cheers
Stefan

 

I unpacked the demo scenes and replaced the default scene with the 03-Simple_Native scene, but it says I am missing a prefab under the UI tab in the hierarchy view. It also shows a number of missing prefabs in the UI->Canvas section. The debug console only shows [06:13:08] and the app says 'All compiler errors must be fixed before you can enter playmode'

 

Please install the "UI"-package from "Assets\Plugins\crosstales\Common".
That should solve the problem.

 

Thank you for your assistance Stefan.

After installing the UI package, I am still seeing some errors of the form:
1.) You are trying to replace or create a Prefab from the instance 'WindowAbout' that contains the script 'UIDrag', which does not derive from MonoBehaviour. This is not allowed.
Please change the script to derive from MonoBehaviour or remove it from the GameObject.

Similar messages with excess removed:
2.) ...from the instance 'WindowAbout' that contains the script 'UIFocus'...
3.) ...from the instance 'WindowAbout' that contains the script 'WindowManager'...
4.) ...from the instance 'Social' that contains the script 'Social'...

I couldn't find any reference to WindowAbout in the code.

 

This is really strange, all those scripts are inheriting from MonoBehaviour.
Maybe restart Unity?

If you still experience problems, please try it in an empty project.

 

I was able to get the 03-SimpleNative example working after a shutdown and reboot of Unity. I will exlore getting it to work in my app. Thank you.

 

Lets say my Windows 10 game uses several Windows voices- does the user of the game need to install the voices on their PC or are they embedded into the application? Thanks.

 

That depends on your effort You can theoretically create a setup program that installs your desired voices - or you let the user do the work.

 

Thank you for the quick reply. So it just an established thing with that you cannot embed them in a game. Ok, got it. Thank you.

 

No, you could create an installer that also installs the desired voices. But this is not part of our asset, so you have to do that by yourself.

 

Are there examples of this that you could point me too?

In your opinion what would the simplest way of adding lets say 8-10 voices to a standalone windows game? I noticed for English Win 10 only has a few available.

Would that be the MaryTTS or something else?

I don't care about the quality of the voices, I just need variation with human sounding voices and low friction for the end user.

Thank you.

 

You could use eSpeak or MaryTTS if you want to have free TTS.
Otherwise, the supported cloud services from AWS Polly, Azure or Google are very good.

 

Hello Stefan.

I have been using RT-Voice Pro for several years with different Windows voices (en-US David, es-ES Helena, en-US Zira, pt-BR Maria...).

Usually when I download a new voice through Windows 10 UI, RT-Voice recognizes it, but I am having problems with pt-PT (Helia) voice. It does not show in the list of available voices in RTVoice Prefab and I can not access it in the Speaker's voices.

I have seen in the RT-Voice Pro documentation that "Microsoft Server Speech Text to Speech Voice (pt-PT, Helia)" is one of the available voices but as I have explained it seems I can not make it work.

Do you have any advice? Thanks in advance

 

Hi Aaron

Windows has two separate TTS-systems, one for "normal" standalone apps (SAPI5) and one for UWP.
I assume you installed the voice for UWP so it won't show up in the Editor and non-UWP-builds.

This is the panel for UWP:


and this is the "old" SAPI5-panel (via "Control Panel"):


If the voice is not here, you can't use it in non-UWP apps and vice versa.

I hope this clarifies the situation.


Cheers
Stefan