Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[RELEASED] Google Cloud Streaming Speech Recognition [VR\AR\Mobile\Desktop]

Discussion in 'Assets and Asset Store' started by FrostweepGames, Jun 3, 2018.

  1. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264

    NON STREAMING SPEECH RECOGNITION | WEBSITE | BUY NOW

    Google Cloud Streaming Speech Recognition a true tool for Unity which provides functionality for:
    • The recording of voice and the recognition of it
    • Setup of Speech Context
    • Support of 120 languages and variants
    • Fast Speech Recognition
    • Offers selection of pre-built models, tailored for your use case
    • Automatically transcribes proper nouns and context-specific formatting
    • Full included Google Cloud Speech REST API*
    • Runtime Voice Detection feature

    Based on Google Cloud Speech Recognition

    Warning:
    • Be sure to read the terms of service of Google Cloud Speech Recognition
    • The plugin does not cover the cost of Google Cloud Service

    Most necessary for communications in VR\AR games and applications!
     
    Last edited: Jul 19, 2021
  2. solarisn

    solarisn

    Joined:
    Apr 6, 2016
    Posts:
    13
    Hi! I'm a bit confused... does the asset currently support streaming speech recognition? It says "Google Cloud Streaming Speech Recognition" on the asset store but further down it says that streaming speech recognition isn't included. Which is true?
     
    rglevy likes this.
  3. akil-active-ai

    akil-active-ai

    Joined:
    Apr 11, 2018
    Posts:
    3
    Yes!
    We are waiting for the streaming speech recognition feature...
    any ETA ?
     
  4. raeldor

    raeldor

    Joined:
    Jul 8, 2013
    Posts:
    55
    Would like to use streaming speech recognition in a project for Oculus Go. Will it support android? Thanks!
     
    taguados likes this.
  5. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello, you're about this asset? https://assetstore.unity.com/packag...peech-recognition-vr-ar-desktop-desktop-72625

    Actually I dont know where you found that is our asset has streaming speech recognition support.(just links to this thread I see)

    True in that we announced the new asset that support streaming speech recognition..

    Thanks and Best Regards
     
  6. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    For now it supporting only Desktop.

    I guess in the future we will make native libs for the android\ios

    Best
     
  7. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Still no :(
    Have troubles with unity engine..
     
  8. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello guys!

    Good news! We will release the new asset that named as 'Google Cloud Machine Learning Kit' that includes all of our asset that connected to google service such as:

    Speech Recognition API
    Vision API
    Natural Language API
    Translation API

    ~ All in one and optimized to use services together
    ~ Support of both Unity Network Methods: UnityWebRequest and WWW class.
    ~ Optimized network options.
    ~ Updated assets with fixes and improvements.

    Thanks for your support!

    Best Regards,
    your Frostweep Games team
     
  9. dillon_yeti

    dillon_yeti

    Joined:
    May 11, 2015
    Posts:
    9
    Excellent news!

    Is speech streaming supported?

    P.S. the first link http://u3d.as/14pC gave me a 404 on the asset store page.
     
  10. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    in this asset speech recognition doenst implemented.

    yes. for now this page doesnt works.. because the asset still waiting for review

    Best
     
  11. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello guys!

    We are happy to say to you that our new asset published on the Asset Store!

    Check it out right now on Google Cloud Machine Learning Kit page.

    Thanks for your support and we are very happy that you use our assets in your Unity projects!

    Best Regards,

    your Frostweep Games team.
     
  12. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello guys!

    We're happy to say that our new asset was published on the Asset Store!

    Check it out right now on Google Cloud Text To Speech page.

    Thanks for your support!

    Best Regards,

    your Frostweep Games team.
     
  13. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    SO, Please Clarify DOES IT NOW SUPPORT GOOGLE TRUE STREAMING SPEECH RECOGNITION?? (Not where you record the audio then submit for recognition)
     
  14. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    The STREAMING SPEECH RECOGNITION via gRPC now in alpha state and we are working on it.
    we are trying to optimize it in the Unity but still have some troubles with dll's that doesnt works in Unity Editor..

    Best
     
    SachinGanesh likes this.
  15. taneryigit_75

    taneryigit_75

    Joined:
    Jul 25, 2018
    Posts:
    1
    We are waiting for the streaming speech recognition. Especially for the Android :)
     
  16. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    our plan to implement the workable version of the gRPC with Protobuf inside Unity without dll errors and that it will work correct in the editor and in the builds.. for now we are focused on PC version...

    yes, we know that we a small team and not have enough hands to implement this plugins inside Unity super fast but we trying to do our best to make it possible to use to whole Unity users who wants this.

    After the PC demo we will continue working on Mobile version.. yes, we have some code of the streaming api on android, but for now we are prepairing to test it in the new Unity that the java classes will be directly inside Unity.. also problem with multidex and similar.

    Thanks for interesting and we will keep in tocuh to you about news.

    Thanks,
    Best
     
  17. amirkazim

    amirkazim

    Joined:
    May 13, 2016
    Posts:
    2
    Hi Frostweep, My team have brought your Google Cloud Speech Recognition plugin for unity.
    I am facing some issue regarding it. The google speech recognition does not support ma custom names so I have gone through its documentation and noticed that we can add hints to it.
    Here is the link to my point. Your plugin does not have function to add hints. Kindly please resolve this issue.
     
  18. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    answered on your email.

    Just use Speech Context, it mean the 'phrase hints'.

    Best
     
  19. IRobb

    IRobb

    Joined:
    Apr 29, 2014
    Posts:
    16
    Hi!

    I have bought this weekend the Cloud Speech Recognizer and the Text-to-Speech from the asset store. Now I have found that there is a "super plugin" that includes all of them plus vision that I was thinking of it.

    Can I have a discount if I upgrade to the Google Machine Learning Kit plugin?
    Thanks in advance
    Robert
     
  20. nishant1994

    nishant1994

    Joined:
    May 25, 2017
    Posts:
    1
    Does this Speech Recognition works on Samsung Gear VR?
     
  21. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    Actually it will work,

    but I guess you need to make plugin for request microphone permission and I guess solve problem with this requesting in general.
    looks like VR platform blocking requesting permissions. so you need to find solution how to proceed.
    it not depends from our pugin, just a troubleshooting faq.

    Best
     
  22. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264

    Hello, actually we have upgrade plans from every asset to the machine learning kit. but Unity havent tool to make discount if you bought few of them..

    Best
     
  23. jeromeWork

    jeromeWork

    Joined:
    Sep 1, 2015
    Posts:
    427
    @FrostweepGames What's the latest on the Streaming Speech Recognition asset? Your initial post has it at 60% complete, how far are you from release do you think?
     
  24. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    it works, but not insude Unity Editor.

    We are trying to solve this issue on our side instead of waiting for fixes from Unity Engine side.

    Best
     
    founders_unity and jeromeWork like this.
  25. jeromeWork

    jeromeWork

    Joined:
    Sep 1, 2015
    Posts:
    427
    @FrostweepGames Would you consider releasing a beta version? Even with that limitation I would definitely be keen to buy the asset. It's a strategy quite a few assets seem to be taking. That way you also have an active user base for testing.
     
  26. superjayman

    superjayman

    Joined:
    May 31, 2013
    Posts:
    185
    How ridiculous I have to ask the same question again,

    SO, Please Clarify DOES IT NOW SUPPORT GOOGLE TRUE STREAMING SPEECH RECOGNITION?? (Not where you record the audio then submit for recognition)??????

    I cannot believe there is no robust speech reco solution for Unity..
     
  27. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hi.

    it can be.

    for now we are working on Video Intelligence asset.
    after that we will continue working on Streaming Recognition in Unity.
    I guess it will be in march.
    regarding beta version - currently we are working on ystem on our web site to make get it before official release on Unity Asset Store.

    Best
     
  28. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    Our new plugin support it but with some limitations as I described above in this thread.

    We will post info here when we will have stable release beta or production

    Best
     
  29. anwarsafei

    anwarsafei

    Joined:
    Dec 8, 2017
    Posts:
    1
    Hello,

    I need this plugin for detect voice in android with vuforia sdk.
    Does this Speech Recognition works on all smartphone android?

    Thanks
     
  30. Straafe

    Straafe

    Joined:
    Oct 15, 2012
    Posts:
    73
    Is there a demo available we can try?
     
  31. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    hello, I just want to ask question where or possible we could select a model? eg phone_call,video, or default?
     
  32. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hi its a streaming speech recognition that only works on Desktop for now. check non streaming speech recognition
    Best
     
  33. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    hello,
    not yet for public
     
  34. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    not sure that its possible for now. you can use speech context I guess.

    Best
     
  35. Conspyra

    Conspyra

    Joined:
    May 15, 2019
    Posts:
    3
    Hello,

    everything works fine in the Unity Editor, but if i try to test it on my Android Device, it doesnt recognize my voice at all. I dont even get asked if I should allow my microphone in the app (like how it is in your demo in app store). Your demo works fine with android. Did i forgot to change something?

    Thanks
     
  36. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hi,

    did you inserted plugin that asking microphone permission?
    or you can use new Unity API https://docs.unity3d.com/Manual/android-RequestingPermissions.html

    just request microphone permission and it will work well.

    Best
     
  37. Conspyra

    Conspyra

    Joined:
    May 15, 2019
    Posts:
    3
    Hello,

    which Plugin do you mean exactly? I will try the Unity API, thank you very much. But I dont see in which class you are handling the permissions.
     
  38. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    hi,

    actually you can call API I sent above at the start of app inside your code. or you can insert it before calling start recording.

    Best
     
  39. Conspyra

    Conspyra

    Joined:
    May 15, 2019
    Posts:
    3
    Hello,

    thank you for your replies. It works now! But I still found a problem: If I allow the microphone permission for the first time, than i need to restart the app in order to record my voice, otherwise it wont record it. After I start the app again and again, it works great. Only the first time were i allow the permissions it wont work. The same problem also appears in your Demo. Is there any solutions for it?

    Thank you
     
  40. shwa

    shwa

    Joined:
    Apr 9, 2012
    Posts:
    461
    Reposted from the less active thread related to this asset:

    I'm looking for a relatively simple voice recognition solution.
    For Mac and Win desktop standalone.

    1. Menu : User can speak 1 word, or just a few words, and it opens a specific scene.

    2. Single word or short words trigger an action/response within a scene.

    Can this frostsweep specific voice recognition asset do this right out of the box?

    If not, is it relatively straightforward to use, so i can implement the above?

    To start, I'd prefer to buy just this asset, and not the Google Cloud Machine Learning Kit.

    Does this asset have all the same voice recognition functionality/demos/uses as the more expensive kit?
    Is there an upgrade path to the Cloud Machine kit, after i buy the voice recognition asset?

    thanks!
     
  41. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello,

    you can use it like you want.

    in result you will get recognized text. so you just need to compare it with your list of commands and then change scene or whatever you want^^.
    yes. you could by only speech recognition asset not whole kit.
    and yeah, you will have a discount provided by asset store to buy whole kit.


    Best regards
     
  42. FrostweepGames

    FrostweepGames

    Joined:
    Jan 2, 2015
    Posts:
    264
    Hello, we areworking on solution. will be fixed in new vesion of an asset soon.

    Best
     
  43. raeldor

    raeldor

    Joined:
    Jul 8, 2013
    Posts:
    55
    Hi. Is the streaming speech recognition and synthesis working on Android yet? I'd really like to use this for an Oculus Quest project. Thank you.
     
  44. shwa

    shwa

    Joined:
    Apr 9, 2012
    Posts:
    461
    Hi,

    Speech recognition is a powerful tool for certain kinds of unity projects.
    I bought this asset, and am trying to figure out how to do a simple Menu system.

    Player says the word Home, and it triggers leaving the current scene and going to the home scene.
    Or
    Player says the word Water, and it triggers leaving the current scene and going to the water scene.

    I'm not really a coder, more a big picture person, designer/ artist.

    Trying to get the code below to work, but running into some syntax errors.
    There may very well be other things missing/or not done properly, since my coding skills are not strong.
    I tried using the Visual Editor Debugger, but I couldn't sort things out.

    Thanks in advance, if you can help.
    / shwa


    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. public class VScene8 : MonoBehaviour{}
    4.  
    5. public class Command
    6. {
    7. public string commandText;
    8. public System.Action commandAction;
    9. }
    10.  
    11. List<Command> commands = new List<Command>();
    12.  
    13. void Start()
    14. {
    15. command.Add(new Command()
    16. {
    17. commandText= "hello world",
    18. commandAction = () => {  Application.LoadLevel("VoiceResult1") }
    19. });
    20. }
    21.  
    22. void voiceRecognizedResponse(ResponseType response)
    23. {
    24.     foreach(var result in response.results)
    25. {
    26. foreach(var command in command)
    27. {
    28.    if(result.alternative == command.commandText)
    29. {
    30. command.commandAction();
    31. }
    32. }
    33. }
    34. }
    35.  
     
  45. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    Microphone devices not found! in first install, but after restarting the phone it was working..any fix for this? im using unity 2018.3.8f1 in windows 10..building for andorid, but the permission show up during start.. then accepts.. but when starts record cant get a callback or error

    EDIT : Found solution.. make dummy since to load prefabs during first start of the app to accept the permission
     
    Last edited: Jul 8, 2019
  46. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    375
    is this plugin possble to build in webgl?
     
  47. Privateer

    Privateer

    Joined:
    Jun 5, 2016
    Posts:
    23
    I know you've heard this a lot already but my deadline is coming up. Any news on the streaming functionality?
     
    FaberVi likes this.
  48. Straafe

    Straafe

    Joined:
    Oct 15, 2012
    Posts:
    73
    If you have your own server, I was able to get Google Speech recognition working live inside Unity using a 3rd party server and could post the general idea. Basically Unity processes the audio and sends it to your server, your server then sends that to Google, and the results are passed live the other way. I wrote the server in Java, but you could write the server in any language that can interface with Google's API.
     
  49. itra

    itra

    Joined:
    May 21, 2019
    Posts:
    4
    Hey, just to clarify, was this done with streaming recognition? Or were you passing audio files/data between the servers? I am looking to implement something like this idea for streaming speech so any further info would really help me out! Thanks!
     
  50. Straafe

    Straafe

    Joined:
    Oct 15, 2012
    Posts:
    73
    @itra
    Streaming recognition, here's a short video clip of a test app I wrote last year using the system. It detected the spoken language and provided a live transcription of spoken words using Google's streaming speech recognition.



    That project is not on my machine right now and lives on the cloud, but if it would help I can pull it down and share some scripts with you. What it is is a Unity client connecting to a Java server over a TCP socket connection, the Unity client processes and streams the live audio to the Java server, which in turn uses Google's streaming speech recognition to get the transcript and information, which is streamed live back to the client. If I remember right, the biggest issues I had were with processing and transferring the audio bytes properly between the client and server and maintaining proper endian-ness.
     
    Last edited: Sep 22, 2020