[Announce] Google Cloud Streaming Speech Recognition [VR\AR\Desktop]

NON STREAMING SPEECH RECOGNITION | WEBSITE

Google Cloud Streaming Speech Recognition a true tool for Unity which provides functionality for:
• The recording of voice and the recognition of it
• Setup of Speech Context
• Support of 88+ languages
• Fast Speech Recognition
• Streaming Speech Recognition

Based on Google Cloud Speech Recognition

Warning:
• Be sure to read the terms of service of Google Cloud Speech Recognition
• The plugin does not cover the cost of Google Cloud Service

Most necessary for communications in VR\AR games and applications!

Current Progress:
• Available first demo version
• Percent completness 60%


In this thread we can discuss all features that you want to see in the asset.

 

Hi! I'm a bit confused... does the asset currently support streaming speech recognition? It says "Google Cloud Streaming Speech Recognition" on the asset store but further down it says that streaming speech recognition isn't included. Which is true?

 

Yes!
We are waiting for the streaming speech recognition feature...
any ETA ?

 

Would like to use streaming speech recognition in a project for Oculus Go. Will it support android? Thanks!

 

Hello, you're about this asset? https://assetstore.unity.com/packag...peech-recognition-vr-ar-desktop-desktop-72625

Actually I dont know where you found that is our asset has streaming speech recognition support.(just links to this thread I see)

True in that we announced the new asset that support streaming speech recognition..

Thanks and Best Regards

 

For now it supporting only Desktop.

I guess in the future we will make native libs for the android\ios

Best

 

Still no
Have troubles with unity engine..

 

Hello guys!

Good news! We will release the new asset that named as 'Google Cloud Machine Learning Kit' that includes all of our asset that connected to google service such as:

• Speech Recognition API
• Vision API
• Natural Language API
• Translation API

~ All in one and optimized to use services together
~ Support of both Unity Network Methods: UnityWebRequest and WWW class.
~ Optimized network options.
~ Updated assets with fixes and improvements.

Thanks for your support!

Best Regards,
your Frostweep Games team

 

Excellent news!

Is speech streaming supported?

P.S. the first link http://u3d.as/14pC gave me a 404 on the asset store page.

 

in this asset speech recognition doenst implemented.

yes. for now this page doesnt works.. because the asset still waiting for review

Best

 

Hello guys!

We are happy to say to you that our new asset published on the Asset Store!

Check it out right now on Google Cloud Machine Learning Kit page.

Thanks for your support and we are very happy that you use our assets in your Unity projects!

Best Regards,

your Frostweep Games team.

 

Hello guys!

We're happy to say that our new asset was published on the Asset Store!

Check it out right now on Google Cloud Text To Speech page.

Thanks for your support!

Best Regards,

your Frostweep Games team.

 

SO, Please Clarify DOES IT NOW SUPPORT GOOGLE TRUE STREAMING SPEECH RECOGNITION?? (Not where you record the audio then submit for recognition)

 

Hello,

The STREAMING SPEECH RECOGNITION via gRPC now in alpha state and we are working on it.
we are trying to optimize it in the Unity but still have some troubles with dll's that doesnt works in Unity Editor..

Best

 

We are waiting for the streaming speech recognition. Especially for the Android

 

Hello,

our plan to implement the workable version of the gRPC with Protobuf inside Unity without dll errors and that it will work correct in the editor and in the builds.. for now we are focused on PC version...

yes, we know that we a small team and not have enough hands to implement this plugins inside Unity super fast but we trying to do our best to make it possible to use to whole Unity users who wants this.

After the PC demo we will continue working on Mobile version.. yes, we have some code of the streaming api on android, but for now we are prepairing to test it in the new Unity that the java classes will be directly inside Unity.. also problem with multidex and similar.

Thanks for interesting and we will keep in tocuh to you about news.

Thanks,
Best

 

Hi Frostweep, My team have brought your Google Cloud Speech Recognition plugin for unity.
I am facing some issue regarding it. The google speech recognition does not support ma custom names so I have gone through its documentation and noticed that we can add hints to it.
Here is the link to my point. Your plugin does not have function to add hints. Kindly please resolve this issue.

 

Hello,

answered on your email.

Just use Speech Context, it mean the 'phrase hints'.

Best

 

Hi!

I have bought this weekend the Cloud Speech Recognizer and the Text-to-Speech from the asset store. Now I have found that there is a "super plugin" that includes all of them plus vision that I was thinking of it.

Can I have a discount if I upgrade to the Google Machine Learning Kit plugin?
Thanks in advance
Robert

 

Does this Speech Recognition works on Samsung Gear VR?

 

Hello,

Actually it will work,

but I guess you need to make plugin for request microphone permission and I guess solve problem with this requesting in general.
looks like VR platform blocking requesting permissions. so you need to find solution how to proceed.
it not depends from our pugin, just a troubleshooting faq.

Best

 

Hello, actually we have upgrade plans from every asset to the machine learning kit. but Unity havent tool to make discount if you bought few of them..

Best

 

@FrostweepGames What's the latest on the Streaming Speech Recognition asset? Your initial post has it at 60% complete, how far are you from release do you think?

 

Hello,

it works, but not insude Unity Editor.

We are trying to solve this issue on our side instead of waiting for fixes from Unity Engine side.

Best

 

@FrostweepGames Would you consider releasing a beta version? Even with that limitation I would definitely be keen to buy the asset. It's a strategy quite a few assets seem to be taking. That way you also have an active user base for testing.

 

How ridiculous I have to ask the same question again,

SO, Please Clarify DOES IT NOW SUPPORT GOOGLE TRUE STREAMING SPEECH RECOGNITION?? (Not where you record the audio then submit for recognition)??????

I cannot believe there is no robust speech reco solution for Unity..

 

Hi.

it can be.

for now we are working on Video Intelligence asset.
after that we will continue working on Streaming Recognition in Unity.
I guess it will be in march.
regarding beta version - currently we are working on ystem on our web site to make get it before official release on Unity Asset Store.

Best

 

Hello,

Our new plugin support it but with some limitations as I described above in this thread.

We will post info here when we will have stable release beta or production

Best

 

Hello,

I need this plugin for detect voice in android with vuforia sdk.
Does this Speech Recognition works on all smartphone android?

Thanks

 

Is there a demo available we can try?

 

hello, I just want to ask question where or possible we could select a model? eg phone_call,video, or default?

 

Hi its a streaming speech recognition that only works on Desktop for now. check non streaming speech recognition
Best

 

hello,
not yet for public

 

Hello,

not sure that its possible for now. you can use speech context I guess.

Best

 

Hello,

everything works fine in the Unity Editor, but if i try to test it on my Android Device, it doesnt recognize my voice at all. I dont even get asked if I should allow my microphone in the app (like how it is in your demo in app store). Your demo works fine with android. Did i forgot to change something?

Thanks

 

Hi,

did you inserted plugin that asking microphone permission?
or you can use new Unity API https://docs.unity3d.com/Manual/android-RequestingPermissions.html

just request microphone permission and it will work well.

Best

 

Hello,

which Plugin do you mean exactly? I will try the Unity API, thank you very much. But I dont see in which class you are handling the permissions.

 

hi,

actually you can call API I sent above at the start of app inside your code. or you can insert it before calling start recording.

Best

 

Hello,

thank you for your replies. It works now! But I still found a problem: If I allow the microphone permission for the first time, than i need to restart the app in order to record my voice, otherwise it wont record it. After I start the app again and again, it works great. Only the first time were i allow the permissions it wont work. The same problem also appears in your Demo. Is there any solutions for it?

Thank you

 

Reposted from the less active thread related to this asset:

I'm looking for a relatively simple voice recognition solution.
For Mac and Win desktop standalone.

1. Menu : User can speak 1 word, or just a few words, and it opens a specific scene.

2. Single word or short words trigger an action/response within a scene.

Can this frostsweep specific voice recognition asset do this right out of the box?

If not, is it relatively straightforward to use, so i can implement the above?

To start, I'd prefer to buy just this asset, and not the Google Cloud Machine Learning Kit.

Does this asset have all the same voice recognition functionality/demos/uses as the more expensive kit?
Is there an upgrade path to the Cloud Machine kit, after i buy the voice recognition asset?

thanks!

 

Hello,

you can use it like you want.

in result you will get recognized text. so you just need to compare it with your list of commands and then change scene or whatever you want^^.
yes. you could by only speech recognition asset not whole kit.
and yeah, you will have a discount provided by asset store to buy whole kit.


Best regards

 

Hello, we areworking on solution. will be fixed in new vesion of an asset soon.

Best

 

Hi. Is the streaming speech recognition and synthesis working on Android yet? I'd really like to use this for an Oculus Quest project. Thank you.

 

Hi,

Speech recognition is a powerful tool for certain kinds of unity projects.
I bought this asset, and am trying to figure out how to do a simple Menu system.

Player says the word Home, and it triggers leaving the current scene and going to the home scene.
Or
Player says the word Water, and it triggers leaving the current scene and going to the water scene.

I'm not really a coder, more a big picture person, designer/ artist.

Trying to get the code below to work, but running into some syntax errors.
There may very well be other things missing/or not done properly, since my coding skills are not strong.
I tried using the Visual Editor Debugger, but I couldn't sort things out.

Thanks in advance, if you can help.
/ shwa

Code (CSharp):

1. using UnityEngine;
2.  
3. public class VScene8 : MonoBehaviour{}
4.  
5. public class Command
6. {
7. public string commandText;
8. public System.Action commandAction;
9. }
10.  
11. List<Command> commands = new List<Command>();
12.  
13. void Start()
14. {
15. command.Add(new Command()
16. {
17. commandText= "hello world",
18. commandAction = () => {  Application.LoadLevel("VoiceResult1") }
19. });
20. }
21.  
22. void voiceRecognizedResponse(ResponseType response)
23. {
24.     foreach(var result in response.results)
25. {
26. foreach(var command in command)
27. {
28.    if(result.alternative == command.commandText)
29. {
30. command.commandAction();
31. }
32. }
33. }
34. }
35.  

 

Microphone devices not found! in first install, but after restarting the phone it was working..any fix for this? im using unity 2018.3.8f1 in windows 10..building for andorid, but the permission show up during start.. then accepts.. but when starts record cant get a callback or error

EDIT : Found solution.. make dummy since to load prefabs during first start of the app to accept the permission

 

is this plugin possble to build in webgl?

 

I know you've heard this a lot already but my deadline is coming up. Any news on the streaming functionality?

 

If you have your own server, I was able to get Google Speech recognition working live inside Unity using a 3rd party server and could post the general idea. Basically Unity processes the audio and sends it to your server, your server then sends that to Google, and the results are passed live the other way. I wrote the server in Java, but you could write the server in any language that can interface with Google's API.

 

Hey, just to clarify, was this done with streaming recognition? Or were you passing audio files/data between the servers? I am looking to implement something like this idea for streaming speech so any further info would really help me out! Thanks!

 

@itra
Streaming recognition, here's a short video clip of a test app I wrote last year using the system. It detected the spoken language and provided a live transcription of spoken words using Google's streaming speech recognition.

That project is not on my machine right now and lives on the cloud, but if it would help I can pull it down and share some scripts with you. What it is is a Unity client connecting to a Java server over a TCP socket connection, the Unity client processes and streams the live audio to the Java server, which in turn uses Google's streaming speech recognition to get the transcript and information, which is streamed live back to the client. If I remember right, the biggest issues I had were with processing and transferring the audio bytes properly between the client and server and maintaining proper endian-ness.