Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

uSpeak Voice Chat - work in progress

Discussion in 'Works In Progress' started by PhobicGunner, Jul 25, 2012.

  1. orb

    orb

    Joined:
    Nov 24, 2010
    Posts:
    2,960
    Yeah, haven't found any serious bugs. I must write something more involved with it now :)
     
  2. AFrisby

    AFrisby

    Joined:
    Apr 14, 2010
    Posts:
    212
    Is this fully managed? (e.g. can we run it in the webplayer fine?) and do you have a source licensing option?
     
  3. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    This is for the most part a source distribution (a few DLLs in there), and the whole thing is fully managed. It uses no classes or namespaces which are blacklisted in the webplayer, so it should run just fine.
     
  4. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
  5. dalupro

    dalupro

    Joined:
    Jan 23, 2012
    Posts:
    9
    This package support for iOS, Android, doesn't it?
     
  6. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
  7. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    I noticed that the 3d sound doesn't work in v2.0a.

    It can easily be fixed by fixing one line of code in USpeaker::ReceiveAudio (Line 238). Replace
    Code (csharp):
    1. playBuffer.Add( USpeakAudioClipCompressor.DecompressAudioClip( cont.encodedData, (int)cont.Samples, 1, false, settings.bandMode, RemoteGain ) );
    with
    Code (csharp):
    1. playBuffer.Add( USpeakAudioClipCompressor.DecompressAudioClip( cont.encodedData, (int)cont.Samples, 1, Is3D, settings.bandMode, RemoteGain ) );
    Instead of using the "Is3D" var that is exposed in the inspector, "false" is passed for the 3d param.
     
  8. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    I had changed the 3D sound from the beta version to the new version.
    Instead of being positional the new version just plays in 2D but calculates speaker panning depending on the position of the USpeaker relative to the camera. The idea is in most games using voice chat you want to be able to hear everyone, but it can be pretty nice to hear a directional pan if the person who's speaking is to the left of you (for example).
     
  9. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    Unfortunately we're developing not one of the most games ;) We really like the fact that you can hear people dependent on their distance :)

    But I have another issue. Currently we're re receiving warnings
    Code (csharp):
    1. Delayed playback via the optional argument of Play is deprecated. Use PlayDelayed instead!
    whenever
    Code (csharp):
    1. src.Play( delay );
    is called in "USpeakAudioManager:playClipAtPoint:"
    By replacing the method with the recommended one no audio is played anymore. Do you have suggestions how to fix that? Especially to get rid of this warning.

    Thanks a lot!
     
  10. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    How strange, I don't get that warning even though I'm on the latest version of Unity...

    In any case, the problem here is that Play( delay ) takes int samples relative to 44100hz, whereas PlayDelayed( delay ) takes a float time in seconds. I'm guessing that's why audio isn't playing anymore, because it's delaying by a huge amount.

    Try dividing by 44100.0f in PlayDelayed and see if that fixes it.
     
  11. agpunity

    agpunity

    Joined:
    Jan 28, 2013
    Posts:
    4
    I have some problem about USpeakAudioManager:playClipAtPoint().

    I have tested RecordTest scene in your uSpeak package with the "Record then send" sending mode in the USpeaker component, and when I recorded for a long time(about more than ten seconds) I can't hear the whole recorded sound(I can just hear about 2 seconds). Could you check this problem for me?

    ps: I can't find PlayDelayed() function. where is it?. I am using Unity 4.0.1f2.

    thanks.
     
  12. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    PlayDelayed should be part of the Unity audio API, perhaps it's part of 4.1?

    In any case, something kind of strange happened: I tried the package, and indeed was running into the two second limit.
    I had a branch version which I imported to the main branch (includes some things like lower memory usage and leak fixes), and oddly enough seems to have fixed the issue. I was able to record, and play back, for at least 15 seconds. Not sure what I did to fix it, but I suppose it's fixed o_O
    Such is the life of a programmer.

    I will be submitting the fix to the asset store. I also recognize the need for using this in client-server setups like Photon and Smartfox, so the update will also be including a new set of classes to help aid integration with these solutions.
     
  13. agpunity

    agpunity

    Joined:
    Jan 28, 2013
    Posts:
    4
    Thank you for your response:D.

    I'd like to reduce the size of encoded sound with Speex codec.
    The result of my test is as follows.

    [TABLE="width: 500, align: left"]
    [TR]
    [TD]codec[/TD]
    [TD]bandwidth[/TD]
    [TD]duration[/TD]
    [TD]size[/TD]
    [/TR]
    [TR]
    [TD]Speex[/TD]
    [TD]8k[/TD]
    [TD]15sec[/TD]
    [TD]45KB[/TD]
    [/TR]
    [TR]
    [TD]MuLaw[/TD]
    [TD]8k[/TD]
    [TD]15sec[/TD]
    [TD]58KB[/TD]
    [/TR]
    [TR]
    [TD]ADPCM[/TD]
    [TD]8k[/TD]
    [TD]15sec[/TD]
    [TD]63KB[/TD]
    [/TR]
    [TR]
    [TD]Raw[/TD]
    [TD]8k[/TD]
    [TD]15sec[/TD]
    [TD]129KB[/TD]
    [/TR]
    [/TABLE]












    My goal is about 40 to 50KB per 1 minute and this is the result of the mobile voice chat program I am using.
    I checked Speex codec's test result and I think it's possible to reach my goal with Speex codec.

    How can I modify some options of Speex codec in your code?
    Please help me with this problem.

    ps: When are you going to submit new version to the asset store?
     
    Last edited: Apr 19, 2013
  14. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    A small update with the additions I just mentioned has been submitted already.
    Unfortunately, as for Speex I've already played with all of the settings and set them as low as possible (encoder.quality = 1 and encoder.vbr = true), it was little more than an experiment anyway (I cannot seem to make a satisfactory implementation of Speex, which is why ADPCM is still default)
     
  15. agpunity

    agpunity

    Joined:
    Jan 28, 2013
    Posts:
    4
    I can't find new version in asset store, there is still old version submitted on March 28, 2013.
    please recheck it.
     
  16. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    It has been submitted - it is pending approval. It will be available presumably within a few days.
     
  17. marsigames

    marsigames

    Joined:
    Dec 14, 2012
    Posts:
    1
    Here we have delay for about 1 sec. I'm not sure if I'm doing something wrong or it's a problem with the plugin.

    Today I tried a newer version of the plugin and now I get this error:
    Assets/Various/MoPho Games/uSpeak/Scripts/Core/USpeakAudioManager.cs(62,29): error CS1061: Type `UnityEngine.AudioSource' does not contain a definition for `PlayDelayed' and no extension method `PlayDelayed' of type `UnityEngine.AudioSource' could be found (are you missing a using directive or an assembly reference?)

    could you please help?
     
  18. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    If there's no PlayDelayed function that must mean you are using a version prior to 4.x?
    If so it's an easy fix... replace with Play, cast parameter to int, and multiply by 44100. The difference is that Play takes samples, PlayDelayed takes seconds. In Unity 4 the Play( delay ) is deprecated which is why I replaced with PlayDelayed.

    As for the delay... your audio will be delayed by approximately the play buffer length (you can see the default value is 1 in the inspector). This is unfortunately necessary, as it has to buffer up audio in order to ensure smooth playback in jittery network conditions. Try reducing it to 0.2 or so.
     
  19. Joyrider

    Joyrider

    Joined:
    Aug 31, 2012
    Posts:
    27
    Hi,

    I just bought and downloaded the lastest version of uSpeak and am just having my first glance at it but...
    I see the NetworkUSpeaker prefab has 3 components with missing scripts...
    I'm guessing this shouldn't be this way, or should it?
     
  20. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    That should not be the case, correct.
    To solve this, locate the following three scripts and drag them onto the 'Missing' slots in order:
    USpeaker.cs
    DefaultTalkController.cs
    UnityNetworkUSpeakSender.cs
     
  21. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    Hi,

    I'm using uSpeak with 3d sounds. Is it possible to adjust the drop-off of the volume manually? I'm also looking for an option to set the sound volume to 0 if a certain distance is exceeded.
     
  22. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Hi Kathrin.
    An update should have just been pushed to the asset store, which includes the SpeakerVolume property on USpeaker.
    You can set this based on distance, clamped to 0 if far enough away.
     
  23. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    I downloaded the latest version from the AssetStore but didn't find a SpeakerVolume property. In which version is it integrated?
     
  24. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Oh, I appear to be completely wrong - evidently I never submitted that version :oops:
    Submitted now. If you PM me with your invoice and email I can send you the new version immediately.
     
  25. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    Hi,

    I have another issue that I cannot get rid of: the transmitted speech sounds "interrupted". It is like a "stuttered" speech. I noticed that the frequency of interruptions gets reduced by increasing the "PlayBufferLength".
    I'm currently using a PlayBufferLength of 0.1 in order to reduce the delay until the speech is received by the client. But even with the default of 1 the issue is present. Any ideas how to solve that?

    Thanks!

    Kathrin
     
  26. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Which codec are you using? Additionally, can you send an audio sample for me to hear?
     
  27. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    I'm using MuLawCodec. I'll send you a sample file via private message.
     
  28. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    Hi,

    I just read in your first post that there are 4 different kinds of voice activation integrated:

    "Supports four kinds of voice trigger - push to talk, toggle talk, volume activated, and always on."

    How can I use "volume activated" and "always on"?
     
  29. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Firstly, I'd like to redirect you to the new thread - http://forum.unity3d.com/threads/15...hat-Voice-Chat-For-All-Your-Multiplayer-Games
    I'd like future discussion to take place there.

    In the current version of USpeak, Always On is enabled by simply removing any Talk Controllers. It uses the default behavior of always sending.
    To enable volume activation, simply tick the Volume Activated checkbox. Currently the threshold is hardcoded, in the future I will expose this as a slider in the inspector.
     
  30. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    Ok, the next time I'll use the other thread for questions, thanks!

    I tried using "volume activation", but I noticed a delay of about ~5 seconds until the sound is played at the client (I already wrote an email with a similar issue). Any ideas why this happens? I removed the DefaultTalkController and checked "volume activated" for using it. (I also reverted code changes that I implemented previously.)

    Using "always on" works nicely! No delay, good quality! But it would be nice for use to use "volume activation" in order to keep the bandwidth low.
     
  31. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Very strange - did you try the Local-only demo and see if the delay is present there as well?
    (also coming soon is an update that adds a configurable threshold volume - if you want the update, send me your email and I'll add you to the mailing list)

    EDIT: Hm, I can't seem to replicate this, even with networking turned on. I have my OUYA (go, microconsoles!) loaded up with a demo build of the included Unity Networking scene. Mic is plugged into my PC, as I speak into the mic even with Volume Activated enabled, and no Talk Controller present (always send), I get a lag of about perhaps 3/4 of a second before voice is played back.

    EDIT 2: I've got a question... what's your Send Rate set to?
     
    Last edited: Jul 25, 2013
  32. Kathrin-Jennifer

    Kathrin-Jennifer

    Joined:
    Mar 29, 2010
    Posts:
    128
    Strange thing, I failed to reproduce it even in my application... might have been something else that went wrong yesterday. Thanks for your effort!

    My send rate is set to 15.
     
  33. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Glad to hear it's resolved. I asked about send rate because it was a possible a very low sendrate could cause the issue, but after some testing it would have to be VERY low in order to cause a delay on the order of 5-10 seconds. Certainly not a sendrate of 15.

    Is it possible that your internet connection was acting up yesterday I suppose? On occasion my internet is pretty good, but two days ago I got the worst lag I've ever experienced (Pingtest rated my internet as F when it usually gives it an A, and I got a packet loss of a whopping 20%) so such things do happen.
     
  34. Tahmay

    Tahmay

    Joined:
    Apr 10, 2013
    Posts:
    1
    I actually have the same issue with my game. We have had USpeak setup for months and we haven't updated or touched it. All of a sudden we get a random warbling of the voice. Sendrate = 16 and we use the MuLawCodec with Wide Bandwidth. Everything was perfect and now when someone talks it has this stuttering EX: someone talking sounds like this terrible description: "Hello my *sprch" na *sprch* me " It goes low and you can hear a vibration kind of noise that warbles the voice. It's dreadful because it was perfect before and we hadn't changed anything.
     
  35. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Firstly, I'd like to redirect you to the new thread - http://forum.unity3d.com/threads/15...hat-Voice-Chat-For-All-Your-Multiplayer-Games
    Please move future discussions there.

    Secondly, please:

    1.) Download the latest update from the asset store. Should be version 3.0. If you did not receive USpeak from the Asset Store, please send me your email and I will add you to the update mailing list and send you the latest version.
    2.) Give the Local demo a try. If the warbling is not present, then most likely it is an issue with your internet connection. If it does exist even in the Local demo however, please let me know and if possible send me the exact settings used (codec, bandmode, sendrate, etc) so I might be able to reproduce it.
     
  36. Pixelblock Games

    Pixelblock Games

    Joined:
    Oct 25, 2012
    Posts:
    20

    Hello :D So I just bought this Asset, and I;m trying to implement it into my game using Photon. I get this error when I start up the game:

    USpeaker requires a component which implements the ISpeechDataHandler interface
    UnityEngine.Debug:LogError(Object)
    <Start>c__IteratorC:MoveNext() (at Assets/uSpeak/Scripts/Components/USpeaker.cs:507)

    I understand what this error is saying, i just don't know how to fix it becuase I can't really code in C#. Any ideas?
     
  37. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    What's your speech data handler class look like?
     
  38. xX0utca5tXx

    xX0utca5tXx

    Joined:
    Nov 5, 2012
    Posts:
    4
    Hey,

    I'm trying to get uSpeak to work in my Unity project with SmartFox. it is fully capable of sending voice packets between two people and the audio is smooth but any more than two and some serious lag starts happening. this is due to a sort of packet overload on the receiver.

    I'm trying to figure out how to shorten the send rate of the packets. when I change the send rate to 1/sec in the uSpeaker options I don't get any audio. Is this because it is only send 1/16th of the seconds worth of audio? how can I go about lengthening the audio sample being sent?

    thanks in advance,

    Matt
     
  39. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Here's something to try. Could you swap out for any other codec besides Speex and see if it fixes the problem?
     
  40. Cheshire Cat

    Cheshire Cat

    Joined:
    Sep 18, 2012
    Posts:
    39
    Hi PhobicGunner,
    I'm trying to get USpeak Voice Chat to work with AppWarp game engine. Generally, the worklfow is pretty much simple: I want to record users voice, retrieve the recorded and compressed data as a byte[] (using Uspeak API),send it with AppWarp API which allows max 1K data size, receive the encoded data on the receiver's side as a byte[] (AppWarp API), decode and play it (Uspeak API).
    In other words, I'd like to use your framework as a recording and encoding/decoding one only, with no network part at all.
    Is that possible? At this stage I do not understand how to get recorded data, encoded and compressed, once it is recorded. Looked at USpeakAudioClipCompressor.CompressAudioData() function, but it looks like as a kind of internal API. And USpeakOnSerializeAudio is supposed to get decompressed audio, isn't it?
    10x in advance
     
    Last edited: Apr 2, 2014
  41. xX0utca5tXx

    xX0utca5tXx

    Joined:
    Nov 5, 2012
    Posts:
    4
    Yes I have tried using Speex, ADPCM, and MuLaw and between the three Speex has the best audio quality with no problems between two people but buggy if more than two people. there is a seperate audio buffer for every player in the scene, the problem seems to be too many packets at once.
     
  42. PhobicGunner

    PhobicGunner

    Joined:
    Jun 28, 2011
    Posts:
    1,813
    Yes Speex is quality-wise the best of the three. BUT what I wanted to know is if the jitter/choppiness problem goes away with the other two codecs? If not, then I can safely rule out a few possible causes.
     
  43. xX0utca5tXx

    xX0utca5tXx

    Joined:
    Nov 5, 2012
    Posts:
    4
    no, the jitter does not go away with mulaw being the worst adcpm jittering if multiple people are talking at once.
     
  44. J_Troher

    J_Troher

    Joined:
    Dec 2, 2013
    Posts:
    41
    So what happened to Uspeak is it not on the store anymore? I can't seem to find it.
     
  45. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    4,853
    It was replaced with DFVoice.
     
  46. J_Troher

    J_Troher

    Joined:
    Dec 2, 2013
    Posts:
    41
    Good looking out! Did they ever manage to get webcam streaming working?
     
  47. LuckyHRF

    LuckyHRF

    Joined:
    May 17, 2013
    Posts:
    6
    I also want to have a test package.Thank you to help!