RT-Voice - Run-time text-to-speech solution

Hi Jean

SSML works only on Windows standalone or together with MaryTTS.


Cheers
Stefan

 

Thanks for the answer ,
Is it any plan to add this functionnality later (or Emotion ml)?

On iphone i m able to download more voice package (in accessibility menu) i succeed to use the downloaded voice but not the siri voice, is it possible to use siri voice ?

 

Unfortunately, iOS doesn't support SSML, so we can't do much against it.
But MaryTTS should be a solution - the only downside is imho the desired Internet connection.

Regarding "Siri" - this is also not possible since Apple hasn't integrated it into their speech synthesizer

 

I m trying to read list of sentence , i try to use event for wait end of sentence . it s working well on my laptop (macbook) but when i try on iphone there is no waiting time.

i try to use same concept than in your dialog scene (who work great on my phone) but it s not working here my code :

Code (CSharp):

1. using System.Collections;
2. using System;
3. using System.Collections.Generic;
4. using UnityEngine;
5. using Crosstales.RTVoice;
6. using Crosstales.RTVoice.Model;
7. using UnityEngine.UI;
8.  
9. public class TalkOnClick : MonoBehaviour
10. {
11.  
12.     public string text;
13.    
14.  
15.     private Book theBook;
16.     private Dictionary<string,Crosstales.RTVoice.Model.Voice> currentVoices = new Dictionary<string, Crosstales.RTVoice.Model.Voice>();
17.     public bool IsSpeaking=false;
18.     private string uidSpeaker;
19.  
20.     private TranslationManager translationManager = new TranslationManager();
21.  
22.     private void Start()
23.     {
24.         // Subscribe event listeners
25.         Speaker.OnSpeakStart += speakStartMethod;
26.         Speaker.OnSpeakComplete += speakCompleteMethod;
27.  
28.         try
29.         {
30.             Debug.Log("start");
31.             TextAsset textFile = Resources.Load<TextAsset>("books");
32.             theBook = JsonUtility.FromJson<Book>(textFile.text);
33.         }
34.         catch (System.Exception ex)
35.         {
36.             Debug.Log(ex.Message);
37.         }
38.    
39.     }
40.  
41.     void OnDestroy()
42.     {
43.         // Unsubscribe event listeners
44.         Speaker.OnSpeakStart -= speakStartMethod;
45.         Speaker.OnSpeakComplete -= speakCompleteMethod;
46.     }
47.  
48.     private void speakStartMethod(Wrapper wrapper)
49.     {
50.         if (wrapper.Uid.Equals(uidSpeaker))
51.         {
52.             Debug.Log("speakStartMethod - Speaker : " + wrapper);
53.             IsSpeaking = true;
54.         }
55.      
56.     }
57.  
58.     private void speakCompleteMethod(Wrapper wrapper)
59.     {
60.         if (wrapper.Uid.Equals(uidSpeaker))
61.         {
62.             Debug.Log("speakCompleteMethod - Speaker : " + wrapper);
63.             IsSpeaking = false;
64.         }
65.     }
66.  
67.     private bool startReading = false;
68.  
69.     void Update()
70.     {
71.         if (startReading)
72.             return;
73.      
74.         if (Input.GetMouseButtonDown(0))
75.         {
76.             startReading = true;
77.             StartCoroutine(ReadBook());
78.         }
79.     }
80.  
81.    
82.  
83.     IEnumerator ReadBook()
84. {
85.         yield return SayAndWait(theBook.name,theBook.culture,0.5f);
86.  
87.         foreach (var sentence in theBook.sentences)
88.         {
89.               yield return SayAndWait(sentence, theBook.culture,0.5f);
90.         }
91.  
92.         startReading = false;
93.        
94. }
95.  
96.  
97.     private IEnumerator SayAndWait(string sentence,string culture="EN", float waitSeconds=0)
98.     {
99.         Say(sentence,culture);
100.  
101.         //wait until ready
102.         do
103.         {
104.             yield return null;
105.         } while (!IsSpeaking && startReading);
106.  
107.         //wait until played
108.         do
109.         {
110.             yield return null;
111.         } while (IsSpeaking && startReading);
112.  
113.         yield return new WaitForSeconds(waitSeconds);
114.  
115.     }
116.  
117.     private void Say(string sentence,string culture="EN") {
118.  
119.         if (string.IsNullOrEmpty(sentence))
120.             return;
121.      
122.         if (string.IsNullOrEmpty(culture))
123.             culture = "EN";
124.        
125.             currentVoices.Add(culture, Speaker.VoiceForCulture(culture,index));
126.         }
127.      
128.         Debug.Log("[" + culture + "]" + "Say :" +sentence );
129.         uidSpeaker =      Speaker.Speak(sentence, null, currentVoices[culture]);
130.     }
131. }
132.  
133.  

did i missed something ?

thanks for your support !



EDIT : i just noticed that i have this error :
[TTS] _BeginSpeaking: couldn't begin playback

 

Please make sure you wait for the event "OnVoicesReady" before starting the first speech.
That should solve the problem.

 

hi , i create on new standalone version of my script who include the OnVoicesReady event but i still have the same problem.

Can you try it ?
- start new project
- switch to iphone platform
- import RT Voice
- add RT Voice prefab
- add this script to RT Voice instance
- start and touch the screen for launch the reading

here the script

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Crosstales.RTVoice;
5. using Crosstales.RTVoice.Model;
6.  
7. public class TalkOnClick : MonoBehaviour
8. {
9.     private Dictionary<string,Crosstales.RTVoice.Model.Voice> currentVoices = new Dictionary<string, Crosstales.RTVoice.Model.Voice>();
10.     public bool IsSpeaking = false;
11.     public bool voiceReady = false;
12.  
13.     private string uidSpeaker;
14.     private void Start()
15.     {
16.         Speaker.OnVoicesReady += onVoicesReady;
17.         // Subscribe event listeners
18.         Speaker.OnSpeakStart += speakStartMethod;
19.         Speaker.OnSpeakComplete += speakCompleteMethod;
20.     }
21.  
22.     private void onVoicesReady()
23.     {
24.         voiceReady = true;
25.  
26.         Debug.Log("voiceReady");
27.     }
28.  
29.     void OnDestroy()
30.     {
31.         // Unsubscribe event listeners
32.         Speaker.OnSpeakStart -= speakStartMethod;
33.         Speaker.OnSpeakComplete -= speakCompleteMethod;
34.         Speaker.OnVoicesReady -= onVoicesReady;
35.  
36.     }
37.     private void speakStartMethod(Wrapper wrapper)
38.     {
39.         if (wrapper.Uid.Equals(uidSpeaker))
40.         {
41.             Debug.Log("speakStartMethod - Speaker : " + wrapper);
42.             IsSpeaking = true;
43.         }
44.      
45.     }
46.  
47.     private void speakCompleteMethod(Wrapper wrapper)
48.     {
49.         if (wrapper.Uid.Equals(uidSpeaker))
50.         {
51.             Debug.Log("speakCompleteMethod - Speaker : " + wrapper);
52.             IsSpeaking = false;
53.         }
54.     }
55.  
56.     private bool startReading = false;
57.  
58.     void Update()
59.     {
60.         if (!voiceReady || startReading)
61.             return;
62.      
63.         if (Input.GetMouseButtonDown(0))
64.         {
65.             startReading = true;
66.  
67.             StartCoroutine(ReadBook());
68.         }
69.  
70.  
71.         for (int i = 0; i < Input.touchCount; ++i)
72.         {
73.             startReading = true;
74.             if (Input.GetTouch(i).phase == TouchPhase.Began)
75.                 StartCoroutine(ReadBook());
76.            
77.         }
78.     }
79.  
80.    
81.    
82.     IEnumerator ReadBook()
83.     {
84.         Debug.Log("talk !!");
85.    
86.             for (int i = 0; i < 10; i++)
87.             {
88.                
89.             yield return SayAndWait(string.Format("this is the sentence {0}",i+1));
90.             }
91.         startReading = false;
92.     }
93.  
94.  
95.     private IEnumerator SayAndWait(string sentence,string culture="EN", float waitSeconds=0)
96.     {
97.         Say(sentence,culture);
98.  
99.         //wait until ready
100.         do
101.         {
102.             yield return null;
103.         } while (!IsSpeaking && startReading);
104.  
105.         //wait until played
106.         do
107.         {
108.             yield return null;
109.         } while (IsSpeaking && startReading);
110.         yield return new WaitForSeconds(waitSeconds);
111.  
112.     }
113.  
114.     private void Say(string sentence,string culture="EN") {
115.         if (string.IsNullOrEmpty(sentence))
116.             return;
117.        
118.         if (string.IsNullOrEmpty(culture))
119.             culture = "EN";
120.        
121.         if (!currentVoices.ContainsKey(culture))
122.             currentVoices.Add(culture, Speaker.VoiceForCulture(culture));
123.        
124.         Debug.Log("[" + culture + "]" + "Say :" +sentence );
125.         uidSpeaker =      Speaker.Speak(sentence, null, currentVoices[culture]);
126.     }
127. }

Thanks

 

Hi

I'm currently on a business trip and don't have an iOS-device available.
However, I found some potential problems and fixed them in your script (see #CT).
Can you please try it again and tell me if it helps?

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Crosstales.RTVoice;
5. using Crosstales.RTVoice.Model;
6.  
7. public class TalkOnClick : MonoBehaviour
8. {
9.     private Dictionary<string, Voice> currentVoices = new Dictionary<string, Voice>();
10.     private bool IsSpeaking = false;
11.     private bool voiceReady = false;
12.  
13.     private string uidSpeaker;
14.     private bool startReading = false;
15.  
16.     public void Start()
17.     {
18.         Speaker.OnVoicesReady += onVoicesReady;
19.         // Subscribe event listeners
20.         Speaker.OnSpeakStart += speakStartMethod;
21.         Speaker.OnSpeakComplete += speakCompleteMethod;
22.     }
23.  
24.     public void OnDestroy()
25.     {
26.         // Unsubscribe event listeners
27.         Speaker.OnSpeakStart -= speakStartMethod;
28.         Speaker.OnSpeakComplete -= speakCompleteMethod;
29.         Speaker.OnVoicesReady -= onVoicesReady;
30.  
31.     }
32.  
33.     public void Update()
34.     {
35.         if (!voiceReady || startReading)
36.             return;
37.  
38. #if UNITY_STANDALONE || UNITY_EDITOR //#CT: added to prevent double speak
39.         if (Input.GetMouseButtonDown(0))
40.         {
41.             startReading = true;
42.  
43.             StartCoroutine(ReadBook());
44.         }
45. #else
46.  
47.         for (int i = 0; i < Input.touchCount; ++i)
48.         {
49.             if (Input.GetTouch(i).phase == TouchPhase.Began)
50.             {
51.                 StartCoroutine(ReadBook());
52.                 startReading = true;
53.                 break; //#CT: multiple calls prevented
54.             }
55.         }
56. #endif
57.     }
58.  
59.     private void onVoicesReady()
60.     {
61.         voiceReady = true;
62.  
63.         Debug.Log("voiceReady");
64.     }
65.  
66.     private void speakStartMethod(Wrapper wrapper)
67.     {
68.         if (wrapper.Uid.Equals(uidSpeaker))
69.         {
70.             Debug.Log("speakStartMethod - Speaker : " + wrapper);
71.             IsSpeaking = true;
72.         }
73.  
74.     }
75.  
76.     private void speakCompleteMethod(Wrapper wrapper)
77.     {
78.         if (wrapper.Uid.Equals(uidSpeaker))
79.         {
80.             Debug.Log("speakCompleteMethod - Speaker : " + wrapper);
81.             IsSpeaking = false;
82.         }
83.     }
84.  
85.     private IEnumerator ReadBook()
86.     {
87.         Debug.Log("talk !!");
88.  
89.         for (int i = 0; i < 10; i++)
90.         {
91.  
92.             yield return SayAndWait(string.Format("this is the sentence {0}", i + 1));
93.         }
94.         startReading = false;
95.     }
96.  
97.  
98.     private IEnumerator SayAndWait(string sentence, string culture = "EN", float waitSeconds = 0.1f) //#CT: small delay added
99.     {
100.         Say(sentence, culture);
101.  
102.         //wait until ready
103.         do
104.         {
105.             yield return null;
106.         } while (!IsSpeaking && startReading);
107.  
108.         //wait until played
109.         do
110.         {
111.             yield return null;
112.         } while (IsSpeaking && startReading);
113.  
114.         yield return new WaitForSeconds(waitSeconds);
115.     }
116.  
117.     private void Say(string sentence, string culture = "EN")
118.     {
119.         if (string.IsNullOrEmpty(sentence))
120.             return;
121.  
122.         if (string.IsNullOrEmpty(culture))
123.             culture = "EN";
124.  
125.         if (!currentVoices.ContainsKey(culture))
126.             currentVoices.Add(culture, Speaker.VoiceForCulture(culture));
127.  
128.         Debug.Log("[" + culture + "]" + "Say :" + sentence);
129.         //uidSpeaker = Speaker.Speak(sentence, null, currentVoices[culture]);
130.         uidSpeaker = Speaker.SpeakNative(sentence, currentVoices[culture]); //#CT: iOS only supports native
131.     }
132. }
133.  

Thank you!


So long,
Stefan

 

Yes it s working .

Thanks a lot for your support !!!

Another question , I plan to buy lipsync asset (like salsa or lipsync pro), which one is easier to integrate and have best support with RT Voice ?

 

You're welcome!

I suggest SALSA - it offers a great integration and support.


So long,
Stefan

 

Thanks for the advice , i buy it .

 

Is it possible to change language using SSML in combination with RT-Voice. If so, which tags should be used?

 

You have to change the voice. Here is an example:

For more, please see the SSML.txt under "Documentation".

 

Hello there,

I've been using RTVoice to generate audio files within the editor but with the different voices between Windows and mac I was looking into using MaryTTS for a more unified solution. However I noticed that the audio generation is not supported for editor mode and that you had commented all the code used by SpeakNativeInEditor. Is there any way of doing it? Will adapting the generation code for editor work?

 

On Unity Cloud Build I get this error, ( I'm just getting familiar with cloud build)

1552: [Unity] Player export failed. Reason: IOException: Failed to Copy File / Directory from '/BUILD_PATH/my_projectname-windows-desktop-64-bit/Assets/crosstales/RTVoice/Wrapper/Windows/RTVoiceTTSWrapper.exe' to '/BUILD_PATH/my_projectname-windows-desktop-64-bit/temp.XXXXXX20180223-4447-je2s6b/Default Windows desktop 64-bit_Data/RTVoiceTTSWrapper.exe'.

Strangely it builds fine locally , it would be easier to figure out if both failed I think.

Thanks, we like this plugin!

 

Hi

It's not that easy - it won't work in the Editor without a lot of modifications...

However, you could define all your speeches and add them to the "AudioFileGenerator". Then hit play and your done. So it works, but not directly in the Editor. The generated files are afterwards ready to use ;-)

I hope this gets you further, please let me know.


Cheers
Stefan

 

Looks really good for an AI project, goodjob

 

I wrote you an email.

Cheers
Stefan

 

Great news!

We successfully integrated AWS Polly into RTV!

RTV will now be able to work with ANY TTS system via new providers.
This means, we also add support for Klattersynth TTS

 

Hi ,
I am getting this error whenever i try to run the function , Speaker.Speak (SpeakText, null ,Speaker.VoiceForName(VoiceName));
i have read the whole blog didn't get much help except of installing MicrosoftSpeechPlatform SDK . Even after that same error pops up.
Kindly guide me how to fix this i am near to deadline and this package can save my day

TypeLoadException: Could not load type 'UnityEngine.WWWAudioExtensions' from assembly 'RTVoice'.
UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)
UnityEngine.MonoBehaviour:StartCoroutine(IEnumerator)
Crosstales.RTVoice.Speaker:SpeakWithUID(Wrapper)
Crosstales.RTVoice.Speaker:Speak(String, AudioSource, Voice, Boolean, Single, Single, String, Single)
GameManager:Speak() (at Assets/Scripts/GameManager.cs:412)
GameManagerrag(String) (at Assets/Scripts/GameManager.cs:67)
UnityEngine.EventSystems.EventSystem:Update()

 

Hi

I suspect you updated from a previous version of Unity to 2017.3, right?
However, please re-download RTV with your current Unity version (of the project) and import it again.

That should solve the problem, please let me know.


So long,
Stefan

 

Hi !
I am in the process of upgrading from RTVoice PRO 2.7.2 to 2.9.5 with Unity 2017.3.0f3 but I am getting a compilation error that didn't show up in 2.7.2 :

It looks like the common folder didn't exist in the 2.7.2
Do you know where the problem could come from ?

Thanks !

 

Hi

Do you have two references of NAudio.dll in your project?
Please delete one of them and try again.

 

Hi,

Actually i am using 2.7.2 version and i have re imported the package again as you said but the error is still same and secondly in demo scene following error is showing at the top of canvas " No OS voices found - TTS not possible " . Currently i am working for Android Support and later on IOS too in Unity 2017.3 .
Kindly Guide me on this.

Thanks.

 

You have to download the asset again from the store. The actual version is 2.9.5, so please make sure you follow the upgrade instructions in chapter 9:
https://www.crosstales.com/media/data/assets/rtvoice/RTVoice-doc.pdf

 

I managed to silence the error by deleting the NAudio.dll that was in the Common/NAudio/stub folder (the smaller one in term of size), which I think was in conflict with the NAudio.dll in the Common/NAudio folder because they share the same name.
Do I really need both of these dlls to make RT Voice work properly or is there a workaround to keep them both without error ?

Thanks !

 

It should not happen at all! The two DLLs have different import settings...
I think the problem is Unity 2017.3.0 - can you please upgrade to 2017.3.1 and import RTV again?

 

I upgraded to Unity 2017.3.1, reimported RT-Voice and it worked fine, thank you !

 

Hi,
I do have a question about Android TTS.
I am using RT Voice with MakeHuman and Salsa . I am writing an android app that will display a Male or Female 3D avatar, depending on user preference. My plan is to have the Male avatar speaking with a Male voice and the Female avatar with a Female voice.

I am using the following command present in Salsa_RTVoices.cs to speak :

Speaker.Speak(speakText, audioSrc, Speaker.VoicesForCulture("en")[genderindex]);

with genderindex selecting which voice to pick from.

However, I am now discovering the complex world of TTS and that the user Android system may not have more than TTS voice installed and that the voices are not in a specific universal order. So here are my series of questions :

1) Can I figure out which voices are installed in my user Android system ?
2) Can I figure out which voices are male or female ?
3) Can I include, at least 2 specific TTS voices (one male, one female) WITHIN my app so that it can be used by my application ? if yes...how ?

4) I noticed the Mary TTS option but I am not sure if it will give me what I want, and I am not sure how to use it the previous line I wrote.
Also, I would like not to have too many download required by my application, since the intended users target may not be familiar with technology.
Thanks

 

Hi

I try to answer your questions:

1. Yes, Speaker.Voices contains all available voices. BUT: Android has no flag for the gender, so you can't decide it from that list...
2. No, Android is very weak on the TTS side. On every device, the voices are different.
3. You could buy voices from Cereproc etc. but I think you aren't allowed to redistribute them.
4. MaryTTS has a gender flag for every voice, so it would work. You have to use your own server, a manual is included under "Documentation". For testing purposes, you could send me the invoice and I will provide you a test account for our server.

I would choose MaryTTS to accomplish your task. The only alternative I can think off for native TTS on Android is to maintain a list of names for the female and male voices and use it to decide on every device...

I hope this helps you further!


Cheers
Stefan

 

Thank you for those information. I will see what I can do with MaryTTS.

 

I've just completed the provider for eSpeak - this works native under Linux, Windows and macOS!
This means, RTV has now another multi-platform alternative to MaryTTS

 

Hi,
I am currently having a hard time setting up a server to host MaryTTS. I am thinking about using a remote server service like Amazon Serices or Azure. I was wondering if you have any recommendation, experiences or advices when trying to set up MaryTTS on one of those services and then use it on an Android App.

Thanks

 

Hello again

I would use an instance from DigitalOcean with Ubuntu and then follow our manual.

However, for testing purposes, I can give you an account on our MaryTTS server. Just send me the invoice and desired username.

We have added support for AWS Polly and will release it with the upcoming v3 of RTV within the next two to three months.

 

Hi ,

wanted to know what will be given in the parameter in the below function and why is it for and how to call in the parameter.
Speak(string text, AudioSource source = null, Model.Voice voice = null, bool speakImmediately = true, float rate = 1.5f, float pitch = 1f, float volume = 1f, string outputFile = "")

i am trying to give parameters in function call but unable to understand Model.Voice voice = null , how to call in this parameter. i am trying to call like this.
Speaker.Speak (SpeakText, SpeakAudio ,Call of Model.voice ?,true ,0.8f,1f ,1f ,Speaker.VoiceForName(VoiceName));

Kindly let me know on this. i have been searching documents and tutorials didn't found anything on this.

Thanks.

 

Hi

The third parameter of the speak call is the voice you would like to use.
Every platform has its own voices, like "Microsoft David Desktop".
RTV reads all those voices and provides them as list via "Speaker.Voices".

You could either do someting like:
Speak(string text, AudioSource source = null, Speaker.Voices[0], bool speakImmediately = true, float rate = 1.5f, float pitch = 1f, float volume = 1f, string outputFile = "")

or use one of the various "voices" methods in the Speaker:
Speak(string text, AudioSource source = null, Speaker.VoiceForName("Microsoft David Desktop"), bool speakImmediately = true, float rate = 1.5f, float pitch = 1f, float volume = 1f, string outputFile = "")

Here are the other methods:

• bool isVoiceForCultureAvailable (string culture)
• bool isVoiceForNameAvailable (string name)
• System.Collections.Generic.List< Model.Voice > VoicesForCulture (string culture)
• Model.Voice VoiceForCulture (string culture, int index=0, string fallbackCulture="")

If you leave it "null" it uses the default voice of the platform.

For more please see the API:
https://www.crosstales.com/media/da...ass_crosstales_1_1_r_t_voice_1_1_speaker.html

I hope this helps, please let me know.


Cheers
Stefan


Edit:
The new version (currently in review) will simplify the voices with the new class "VoiceAlias":
https://www.crosstales.com/media/da..._1_1_r_t_voice_1_1_model_1_1_voice_alias.html

 

Hi,

Thanks alot for clearing my confussion i was mixing it with the output string the last parameter. Can you tell me if i give the last parameter a string of a file name . will it generate and store the Audio which i am playing on each text ?

Thanks

 

You're welcome!

Yes, the last parameter is the destination of the generated files (if you wanna store them for future usage). If you intend to generate a lot of texts, consider using the "AudioFileGenerator" - it has some neat features

 

I just purchased RT-Voice to use with Pixel Crushers Dialogue System and I have an error with the Third Party Support - RTVoiceActor.cs - voice.Gender string.Equals( invalid. I downloaded from the Asset store directly into my 2017.3 project. Have you seen this before?

 

Hi

I think the problem is that I changed gender from a string to an enumeration.

@TonyLi can you please take a look at it? Is it possible to use the "Gender"-enum?

 

I should have mentioned that the RT-Voice package works fine in a clean project, this is a conflict created in the Pixel Crushers Third Party Support scripts ... I was just hoping that you may have heard this before.

 

I never heard of that before. But it's an easy fix and I think Tony will reply soon.
Meanwhile, you could send me the conflicting ("RTVoiceActor.cs") script via email and I can fix it it...

 

Thanks Stefan, I just sent that to you.
Btw, awesome support! Wow.

 

I've sent you the fixed version

 

Hi @Jim-Kiggens - Please download the updated RT-Voice Support package from the Dialogue System Extras page. RT-Voice's gender definition changed from string to enum since the last release of the Dialogue System. This updated package will also be included in the next release of the Dialogue System.

 

Hi
I imported rt-voice but i have many warnings

 

Hi

Please close and restart Unity.
After that, the warnings should be gone.

Please let me know if it helped.


Cheers
Stefan

 

thanks works, but now when add the script speaker and i clicked in speak does not worked (there isn't sound)

 

Hmm, have you installed eSpeak on your machine? If not, please disable "eSpeak Mode".

 

Hello
i disabled, but this is showed
thanks

 

Please answer the following questions:

1. Which is your Unity version?
2. OS?