Search Unity

Feature Request Tracking Developer Interest in Access to Audio Buffers for the Vivox Unity SDK

Discussion in 'Vivox (Voice & Text Chat)' started by emilyryan, Feb 2, 2022.

Thread Status:
Not open for further replies.
  1. emilyryan

    emilyryan

    Unity Technologies

    Joined:
    Nov 22, 2019
    Posts:
    129
    UPDATE: This feature is now available as part of the new v16 Unity Vivox SDK.

    You can now attach the Vivox audio tap components to game objects and gain access to:
    • Mixed audio of all remote participants in a channel.
    • Audio coming from a single remote participant.
    • Audio coming from the local user's microphone, as captured by Vivox.
    Read more in the developer guide:
    https://docs.unity.com/ugs/en-us/ma...veloper-guide/audio-taps/audio-tap-components



    ---- Original Message Below ----

    Hello Everyone!

    If you are a Unity Developer using Vivox and would like to see access to the Audio Buffers in the Vivox SDK, please like or comment on this article to help us track interest in this feature.

    Access to audio buffers would allow you to do the following to your game/application:
    • Add an audio effect that modifies the voice (for example, a robot voice, radio distortion, or static)
    • Use a third-party audio engine (such as Wwise) for voice render instead of an automatic render provided by Vivox
    • Record audio
    • Perform phoneme analysis

    If you would like to learn more about Audio Buffers, feel free to review this article: https://support.unity.com/hc/en-us/articles/4418149054356
     
    Last edited by a moderator: Nov 6, 2023
  2. lastmilers

    lastmilers

    Joined:
    Dec 13, 2018
    Posts:
    1
    I want to implement AEC in Vivox and I need access to the audio buffer in the Vivox SDK.
     
    lildub likes this.
  3. tophoftheworld

    tophoftheworld

    Joined:
    Dec 23, 2016
    Posts:
    1
    I need to record individual player audio streams. This would be very helpful
     
    lildub likes this.
  4. lildub

    lildub

    Joined:
    May 6, 2015
    Posts:
    14
  5. r06m00r3

    r06m00r3

    Joined:
    Feb 11, 2022
    Posts:
    1
    I registered to like this post. (Would love to add distortion/radio filters to vivox).
     
  6. NuclearCookieTF

    NuclearCookieTF

    Joined:
    Mar 29, 2021
    Posts:
    17
    We need this to use a third party audio engine to render voices. It would allow us to only have 1 system handling positional sounds and additionally would open the door for voice occlusion and spatialization.
     
  7. TFPolygon

    TFPolygon

    Joined:
    Mar 4, 2015
    Posts:
    5
    This is a very important feature for us, definitely working with mobile VR. We don't want 2 audio engines running at the same time. Please implement this! :)
     
  8. Clrj14

    Clrj14

    Joined:
    Nov 25, 2017
    Posts:
    3
    This would be great to have to integrate with Wwise.
     
  9. rick_unity789

    rick_unity789

    Joined:
    Jul 16, 2021
    Posts:
    1
    This is a very important feature for our company as well, as far as I know there is no way for us to stream from an android device and capture voice audio without this.
     
  10. BHostyle

    BHostyle

    Joined:
    Jul 25, 2018
    Posts:
    4
    Definitely need this.
     
  11. SpikeDevelops

    SpikeDevelops

    Joined:
    Mar 12, 2018
    Posts:
    8
  12. dadamsj1

    dadamsj1

    Joined:
    Jul 25, 2017
    Posts:
    42
    It would be nice to have access to the audio buffer. At the very least, the audio should be able to routed through Unity's AudioMixer.
     
  13. virgilcwyile

    virgilcwyile

    Joined:
    Jan 31, 2016
    Posts:
    73
    Yes, Need it so I can do Speech to Text. Right now the API has wrong Method names like SpeechDetected. Where its actually detecting any sound. Using the Audio buffer we can differ between speech and sound. Cant do lip sync until we know exactly if it was speech or random sound.
     
  14. RamenSea

    RamenSea

    Joined:
    Mar 4, 2019
    Posts:
    2
    This is a requirement for my company to use this service. Any updates?
     
    austinmackrell likes this.
  15. jashan

    jashan

    Joined:
    Mar 9, 2007
    Posts:
    3,307
    We'd need this to be able to route Vivox through FMOD and do proper spatialization.
     
  16. austinmackrell

    austinmackrell

    Joined:
    Jan 21, 2020
    Posts:
    2
    I Need this for audio voice effects. Hopefully it’s coming soon.
     
  17. d41602031

    d41602031

    Joined:
    Jan 5, 2015
    Posts:
    7
    Yes please, I need this for recording.
    I hope it's coming very soon. Thank you!
     
  18. SebastianM

    SebastianM

    Joined:
    Jun 10, 2015
    Posts:
    1
    One of the features of our project is record the players voices.
    I'm going to be very grateful with you all, when release that.
     
  19. Javi-K

    Javi-K

    Joined:
    Mar 9, 2022
    Posts:
    6
    I would be very much interested in this, I would like to use the audio as a source for OVRLipSync
     
    ferbailer and mbalcipt like this.
  20. mbalcipt

    mbalcipt

    Joined:
    Aug 5, 2021
    Posts:
    1
    same
     
    Javi-K likes this.
  21. azukiidx

    azukiidx

    Joined:
    Nov 22, 2017
    Posts:
    9
  22. Sourabh_PTW

    Sourabh_PTW

    Joined:
    Mar 7, 2022
    Posts:
    33
    Can this be help full for Noise Cancellation?
     
  23. gamesplease

    gamesplease

    Joined:
    Apr 26, 2021
    Posts:
    1
    Would like to have it implemented for more accurate 3D spatial audio
     
  24. MaximTriangle

    MaximTriangle

    Joined:
    Oct 25, 2017
    Posts:
    2
    Would love to see this
     
  25. waffle_maker

    waffle_maker

    Joined:
    Feb 1, 2015
    Posts:
    3
    I need this feature for spatial audio, LipSync and AudioMixer! !
     
    Polaaa, gamesplease and mbalcipt like this.
  26. Qwolf

    Qwolf

    Joined:
    Feb 1, 2014
    Posts:
    28
    Would love this feature to do custom spatialization on audio.
    Any word on this coming soon?
     
    gamesplease likes this.
  27. ZKapps

    ZKapps

    Joined:
    Nov 9, 2016
    Posts:
    4
    This is really funny. Any reason not giving it ?
     
  28. jlindskov

    jlindskov

    Joined:
    Feb 12, 2020
    Posts:
    6
    Yes! Would for sure be an welcome feature.
     
  29. niko_unity957

    niko_unity957

    Joined:
    Oct 18, 2022
    Posts:
    1
    We are currently battling Oculus Lipsync X Vivox issues resulting to echo.
    Being able to chain lipsync to Vivox would solve them.
     
  30. Wolvenmoon

    Wolvenmoon

    Joined:
    Sep 15, 2018
    Posts:
    5
    I was prototyping something in Unity before moving it to Vivox Core. Learning that there isn't direct access to the audio buffers just blew me out of the water. I would strongly suggest putting a notification on the Vivox Core documentation that shows direct access to audio buffers that it is exclusively accessible in Core or with modification on the Unreal engine.
     
  31. mitaywalle

    mitaywalle

    Joined:
    Jul 1, 2013
    Posts:
    247
    we are implementing voice chat, and wanna make decorative radio-voice, when player is in police-car
     
    sa-muratbalci likes this.
  32. Mutimir

    Mutimir

    Joined:
    Dec 6, 2018
    Posts:
    36
    This is a big hit for our project. We swapped to vivox as the echo cancelation on mobile was superior then anything else on the market, but we have the need to record the voice chat locally (there is a record on server functionality that doesn't work for us) and to stream the audio through Unity render streaming. Which is both blocked by this issue.
     
    quangthai likes this.
  33. smartandwise

    smartandwise

    Joined:
    Jul 18, 2022
    Posts:
    6
    I need to record voice chat!
     
  34. adrian-taylor09

    adrian-taylor09

    Joined:
    Dec 22, 2016
    Posts:
    63
    I would like access to the audio buffers in order to use lip sync on VR Avatars. Without this feature the service isn’t really useful for us
     
    sa-muratbalci likes this.
  35. joel_unity992

    joel_unity992

    Joined:
    Nov 9, 2022
    Posts:
    1
    Have been wanting this for quite some time. Would be an awesome addition.
     
  36. luis_unity981

    luis_unity981

    Joined:
    Oct 11, 2022
    Posts:
    12
    Definitely need this
     
    gamesplease likes this.
  37. mark_gr

    mark_gr

    Joined:
    Jan 16, 2015
    Posts:
    26
    Yes we need this!
     
    gamesplease and sa-muratbalci like this.
  38. rnjena

    rnjena

    Joined:
    Dec 20, 2015
    Posts:
    1
    Please provide the feature! Definitely, useful feature.
     
    gamesplease likes this.
  39. Ahmed10

    Ahmed10

    Joined:
    May 15, 2015
    Posts:
    2
    We need this feature for VR Avatars LipSync .
     
  40. Sholms

    Sholms

    Joined:
    Nov 15, 2014
    Posts:
    85
    Yes, we need to use Vivox and Fmod . Currently, we are experiencing problems when running the Unity audio engine and Fmod simultaneously.
     
    Last edited: Jun 21, 2023
  41. AleBraFinGroup

    AleBraFinGroup

    Joined:
    May 17, 2023
    Posts:
    9
    Absolutely necessary to perform any lip syncing for VR games especially. Really surprised it isn't available.
     
  42. AleBraFinGroup

    AleBraFinGroup

    Joined:
    May 17, 2023
    Posts:
    9
    Not to mention spatialization, occlusion, reverberation...this is really upsetting. Plz add it.
     
  43. Deleted User

    Deleted User

    Guest

  44. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    Doesn't make sense that it's not supported. So absolutely not the "one stop solution" it claims to be when you can't do basic things with it
     
  45. johnbernhelm_SN

    johnbernhelm_SN

    Joined:
    Jun 3, 2021
    Posts:
    1
  46. quangthai

    quangthai

    Joined:
    Jul 8, 2017
    Posts:
    5
    Desperately need this, how long until I can use this feature? :(:(:(
     
  47. bluepilkintonching

    bluepilkintonching

    Joined:
    Nov 29, 2019
    Posts:
    30
    I need this for a multiplayer co-op game I am developing. Its looking like my team might have to switch to Microsoft Playfab because this isn't implemented. :/
     
    quangthai likes this.
  48. StanK_

    StanK_

    Joined:
    Dec 5, 2017
    Posts:
    13
    This was possible with previous versions and it's easy to make with overriding external functions. But in the latest versions of sdk all external functions are internal and aren't possible to override. Why? It's a high-priority feature for voice analysis and lipsync.
     
  49. Antsa42

    Antsa42

    Joined:
    May 14, 2017
    Posts:
    1
    Definitely looking forward to this.
     
  50. marcrem

    marcrem

    Joined:
    Oct 13, 2016
    Posts:
    340
    It's kind of crazy that Vivox is advertised as the go-to solution. It looks like THE thing to use for your game. Yet, you can't even access the audio feed and add a simple filter effect on it. And then if you want that feature, you need to go like a forum post so they can "see if there is enough interest". That's very nice for production.
     
Thread Status:
Not open for further replies.