Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Automatically capturing user speech in Meta Quest 3

Discussion in 'VR' started by Tyke18, Dec 7, 2023.

  1. Tyke18

    Tyke18

    Joined:
    Oct 6, 2023
    Posts:
    11
    Hi :)

    I'm creating an app whereby a user chats to an NPC, the NPC is powered by Open AI. I want my app to automatically detect when the user speaks, and do stuff with the microphone input (send the audio to openai for speech to text transcription etc) and detect when the speech has stopped.

    Meta's Wit AI can capture mic audio and transcribe it but it offers no automatic voice detection feature, you have to press a key/button first to let it know you're speaking, I don't want that. Can anyone point me in the direction of what I want i.e. an existing software solution etc?
     
  2. DevDunk

    DevDunk

    Joined:
    Feb 13, 2020
    Posts:
    4,847
    Maybe read out the decibels of the mic?
    Start when it goes over a threshold, then stop when it's under a threshold for a few seconds.

    This is why most smart assistants use a phrase like hey google to start recording
     
    FarmerInATechStack likes this.
  3. Tyke18

    Tyke18

    Joined:
    Oct 6, 2023
    Posts:
    11
    thx for the suggestion, I tried that, it worked sometimes but not reliably. I found a python library that does pretty good voice detection, i ran it's code in a websocket server connected to my unity app. bit messy but it works.
     
    DevDunk likes this.