Android Speech/Voice Recognition in unity [SpeakNow]

Hey, vishnugude

When the next version with beeps eliminated is planned for release?

 

Can i compare the correct spelling with this plugin? So for example someone is trying to say "Boy" in English, and the plugin will check the voice and say if it's correct or wrong ? I'm doing a language learning app and need this functionality.

Also, what about iOS support?? I need both

 

I will try to include your idea... Contact me in Skype...We can work on it.

Skype Id : vishnugude

 

Hello friend,

Sorry about the delay...I was in vacation....Will let you know in skype...May be this week.

 

vishnugude, thanks for the replay.

Meanwhile I've bought your plugin and it works OK, but I've a weird issue: every time I've got the results for the phrase I've said before.

I mean first time I say "Hello" - it displays me nothing. Second time I say "Bye" - it displays me "Hello" and so on.

That's my code - I doing it Whatsapp-like way, by pressing and holding the button

Code (CSharp):

1.  
2.   public void recordingButton_ClickDown(iGUIButton caller)
3.   {
4.          SpeechToText.StartSpeech();
5.   }
6.  
7.   public void recordingButton_ClickUp(iGUIButton caller)
8.   {
9.         try
10.        {
11.             testResult = SpeechToText.SpeechResult();
12.             confidenceScore = SpeechToText.getConfidenceScore();
13.        }
14.        catch (System.Exception ex)
15.        {
16.              //Handle exception
17.        }
18.   }

Why do you think that may happen?

Generally, I thought it will be API which looks like Android SpeechRecognizer class, e.g you call SpeechToText.StartSpeech(); then SpeechToText.StopSpeech(); and then there is a callback which is called when the result is ready.

 

ping me in skype ... I will help you

Skype id : vishnugude

 

hi
I can speak in German or any other language? or only English

THX

 

I will add language support in next release.

 

I'm interested in using your plugin at the same time as Google Cardboard.
There was an earlier post about conflicts due to having multiple Activities.
Was this issue resolved?
Is there a way to resolve it?
I'm quite new to Unity and haven't had any experience dealing with multiple activities on Android with Unity.

 

Within a week new version will be released with Multi plugin support and multi language support for the speech recognition.
Thank You.

 

hi
nice to hear that you implementing multi language support !!!
work it with android 4.0 ? I think 4.1 for downloading offline Language.!!

 

Yup .... already implemented Multi Language .... Its in testing phase now...Once I get a stable version....I will release it.

I am also adding so many exciting features.... one more week to go for exciting release.

 

can it detect bahasa indonesia? like "ba, pa, nga, nya, and etc.."

 

Yes added Indonesian language . Waiting for unity approval... Just 5 days to go for release.

 

Released New version : 3.0.1
New Features in 3.0.1 :
**** Added Multi Language support. ****
**** 52 languages supported. ****
**** Multi Plugin Support. Now you can add any number of plugins in parallel to speaknow. ****

 

If anyone had any problem pls let me know.

skype id : vishnugude

 

Its released officially now...check it out..

 

its doesn't support ios eh?

 

Right now it is only for Android.

 

awww.. ohh btw thanks for reply.. ..

 

Thanks for including support for multiple plugins.
I'm trying to use your package with Cardboard, and the new Unity UI. In the demo app you had looked for SpeakNow.getConfidenceScore().Length>0 in onGUI(), and that works fine.
With the new UI, I have an onClick() method that gets called when the user clicks an object. I need a way to look for the result every frame after that click event. I tried to look for a result in Update(), but when I do that I get:

Exception: JNI: Init'd AndroidJavaClass with null ptr!
at UnityEngine.AndroidJavaClass..ctor (IntPtr jclass) [0x00016] in C:\buildslave\unity\build\Runtime\Export\AndroidJavaImpl.cs:539

at UnityEngine.AndroidJavaObject.get_JavaLangClass () [0x0000a] in C:\buildslave\unity\build\Runtime\Export\AndroidJavaImpl.cs:517

at UnityEngine.AndroidJavaObject.FindClass (System.String name) [0x00000] in C:\buildslave\unity\build\Runtime\Export\AndroidJavaImpl.cs:508

at UnityEngine.AndroidJavaClass._AndroidJavaClass (System.String className) [0x00011] in C:\buildslave\unity\build\Runtime\Export\AndroidJavaImpl.cs:528

at UnityEngine.AndroidJavaClass..ctor (System.String className) [0x00006] in C:\buildslave\unity\build\artifacts\generated\common\runtime\AndroidJavaBindings.gen.cs:93

at SpeakNow..cctor () [0x00000] in <filename unknown>:0
Rethrow as TypeInitializationException: An exception was thrown by the type initializer for SpeakNow
at test2.Update () [0x00000] in C:\Users\Rick\Documents\Unity\Speech2\Assets\SpeakNow\test2.cs:35

Can I use the plugin with the new UI?

Thanks

 

Wait - I think I figured it out - Update() is getting called before SpeakNow.startSpeech() ever gets called, that's why I'm getting the error....nevermind, but thanks again

 

Yup its not speaknow error....

 

if you have any queries or doubts pls contact me in skype.

skype : vishnugude

 

I just used a flag to let Update know when speech was active, and that works fine.

But I also ran into a different issue - it looks like we need to be sure to import the SpeakNow plugin before other plugins. When I inadvertently imported Cardboard before SpeakNow, the second plugin didn't work (silently - no pun intended, but there were no error messages). The problem appears to be the AndroidManifest.xml's - when Unity combines them, for whatever reason it wants SpeakNow to be AndroidManifest.xml, and Cardboard (and maybe others) to be AndroidManifest 1.xml, etc.

Anyway, not a blocker, working fine now, but thought you'd want to know.

 

Yeah its always confusing about manifest files...But we have to be careful when we use more manifest files...we need to combine them as single manifest..Then all the plugins works correctly.

 

Do you have any guidelines about combining manifests? It can be pretty confusing - I usually end up with "trial and error", which isn't the most efficient approach.

 

Just use other manifests activities and services...paste them in main manifest file...that sit.

 

Dear vishnugude

i already bought the plugin but i am not able to implement Arabic recognition, do you have a tutorial or reference

regards

 

Re: merging manifests:

But I can't just insert another activity that declares itself to be MAIN to the LAUNCHER, so what do I do there. Specifically the Google Cardboard manifest includes

Code (CSharp):

1.         <activity android:name="com.google.unity.GoogleUnityActivity"
2.                   android:label="@string/app_name"
3.                   android:screenOrientation="landscape"
4.                   android:launchMode="singleTask"
5.                   android:configChanges="mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|uiMode|screenSize|smallestScreenSize|fontScale">
6.             <intent-filter>
7.                 <action android:name="android.intent.action.MAIN" />
8.                 <category android:name="android.intent.category.LAUNCHER" />
9.                 <category android:name="com.google.intent.category.CARDBOARD" />
10.             </intent-filter>
11.             <meta-data android:name="unityplayer.UnityActivity" android:value="true" />
12.         </activity>
13.  

Your manifest includes:

Code (CSharp):

1.         <activity
2.             android:name="com.unity3d.player.UnityPlayerNativeActivity"
3.             android:configChanges="mcc|mnc|locale|touchscreen|keyboard|keyboardHidden|navigation|orientation|screenLayout|uiMode|screenSize|smallestScreenSize|fontScale"
4.             android:label="@string/app_name"
5.             android:launchMode="singleTask"
6.             android:screenOrientation="portrait" >
7.             <intent-filter>
8.                 <action android:name="android.intent.action.MAIN" />
9.                 <category android:name="android.intent.category.LAUNCHER" />
10.             </intent-filter>
11.             <meta-data android:name="unityplayer.UnityActivity" android:value="true" />
12.             <meta-data android:name="unityplayer.ForwardNativeEventsToDalvik" android:value="true" />
13.  
14.         </activity>
15.  

Both activities can't be the one that the Android launcher launches...

Thanks

 

use first manifest and add speaknow service to that manifest... For speaknow plugin it doesnt needs to be main launcher...Any activity can be main launcher..Just add service call to your manifest....Thats it.

 

Start speaknow with Languageutil and your language ...

three options available for arabic those are Arabic_Israel, Arabic_Kuwait, Arabic_UAE

if you still have queries on how to launch...pls ping me in skype.. I will help you to get arabic speaknow launched...I also recommend reading documentation if you have a moment.

skype ID : vishnugude

 

Released new version in which you can do this now....cheers for the idea...If you have any more ideas i can add to the plugin...Thank you.

 

hello

i am not able to find the Languageutil class

please help me

regards

 

Add me in skype...I can help you.

Id : vishnugude

 

Demo Please!

 

add me in skype and indicate which language you want to test.

ID : vishnugude

 

If anyone didnt get LanguageUtil... Upgrade your unity and download asset.. that should get you latetst version of speaknow.

 

Hi, I just bought the plugin - I was wondering if there was anyway to surpress the native Android "ping" sound when resetting the detection / when it detects the speech?

We're using it for something quite atmospheric and the sound it makes kinda "detracts from the experience" somewhat?

 

Add me in skype i will help u...

Username : vishnugude

 

@vishnugude Same problem... any solution?

 

Working on that.. Expect supress audio feature pretty soon..

 

We are launching a new product on the assets store that does pronunciation assessment, voice recognition, pitch contour and waveform. This SDK works locally on osx, ios and android.

To get a preview of the project checkout our github page
https://github.com/ispikit/ispikit-unity

Or

Email me at sydney@ispikit.com

 

You can give some approximate date?

 

Dont worry about the android ping. We have a voice solution that solves the problem.

ispikit.com

 

Dont worry about surpress audio...I will include that in new version and lot of excieting features will be coming guys... Keep checking the thread... Expect new version within 3 weeks.

 

Dont worry about surpress audio...I will include that in new version and lot of excieting features will be coming guys... Keep checking the thread... Expect new version within 3 weeks.

 

<?xmlversion="1.0"encoding="utf-8"?>
<manifest
xmlns:android="http://schemas.android.com/apk/res/android"
package="com.unity3d.player"
android:installLocation="preferExternal"
android:theme="@android:style/Theme.NoTitleBar"
android:versionCode="1"
android:versionName="1.0">
<uses-featureandroid:name="android.hardware.camera"/>
<supports-screens
android:smallScreens="true"
android:normalScreens="true"
android:largeScreens="true"
android:xlargeScreens="true"
android:anyDensity="true"/>
<uses-permissionandroid:name="android.permission.INTERNET"/>
<uses-permissionandroid:name="android.permission.CAMERA"/>
<uses-permissionandroid:name="android.permission.ACCESS_NETWORK_STATE"/>
<uses-permissionandroid:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

<application
android:icon="@drawable/app_icon"
android:label="@String/app_name"
android:theme="@android:style/Theme.NoTitleBar.Fullscreen"
android:debuggable="false">
<activityandroid:name="com.unity3d.player.UnityPlayerNativeActivity"
android:label="@String/app_name">
<intent-filter>
<actionandroid:name="android.intent.action.MAIN"/>
<categoryandroid:name="android.intent.category.LAUNCHER"/>
</intent-filter>
<meta-dataandroid:name="unityplayer.UnityActivity"android:value="true"/>
<meta-dataandroid:name="unityplayer.ForwardNativeEventsToDalvik"android:value="false"/>
</activity>
<activityandroid:name="com.unity3d.player.VideoPlayer"
android:label="@String/app_name"
android:screenOrientation="portrait"
android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen">
</activity>
<!--
TosupportdevicesusingtheTIS3Dlibraryforstereomodewemust
addthefollowinglibrary.
Devicesthatrequirethisare: ODGX6
-->
<uses-libraryandroid:name="com.ti.s3d"android:required="false"/>
<!--
TosupporttheODGR7instereomodewemustaddthefollowinglibrary.
-->
<uses-libraryandroid:name="com.osterhoutgroup.api.ext"android:required="false"/>
</application>
</manifest>














Sorry for asking.. how should i merge this vuforia manifest with SpeakNow manifest

 

Hello,

Just add the SpeakNow service line to vuforia manifest file...Thats it you can njoy two plugins at a time...

ping me in skype if you still face any problems.

ID : vishnugude

 

News?