uSpeak Voice Chat - work in progress

Yeah, haven't found any serious bugs. I must write something more involved with it now

 

Is this fully managed? (e.g. can we run it in the webplayer fine?) and do you have a source licensing option?

 

This is for the most part a source distribution (a few DLLs in there), and the whole thing is fully managed. It uses no classes or namespaces which are blacklisted in the webplayer, so it should run just fine.

 

Well it took months of procrastination, but I finally got it accepted!
http://u3d.as/content/mo-pho-games/u-speak-voice-chat/3sR
Voice chat is now live! WOOT!

 

This package support for iOS, Android, doesn't it?

 

Yes, it does. I've had reports that it in fact works very well on mobile devices.
Actually, one of these days I need to update the main page. USpeak has gone through a complete redesign from the ground-up, and if you plan on buying I suggest you take a look at the updated feature set here: http://forum.unity3d.com/threads/15...hat-Voice-Chat-For-All-Your-Multiplayer-Games

 

I noticed that the 3d sound doesn't work in v2.0a.

It can easily be fixed by fixing one line of code in USpeaker::ReceiveAudio (Line 238). Replace

Code (csharp):

1. playBuffer.Add( USpeakAudioClipCompressor.DecompressAudioClip( cont.encodedData, (int)cont.Samples, 1, false, settings.bandMode, RemoteGain ) );

with

Code (csharp):

1. playBuffer.Add( USpeakAudioClipCompressor.DecompressAudioClip( cont.encodedData, (int)cont.Samples, 1, Is3D, settings.bandMode, RemoteGain ) );

Instead of using the "Is3D" var that is exposed in the inspector, "false" is passed for the 3d param.

 

I had changed the 3D sound from the beta version to the new version.
Instead of being positional the new version just plays in 2D but calculates speaker panning depending on the position of the USpeaker relative to the camera. The idea is in most games using voice chat you want to be able to hear everyone, but it can be pretty nice to hear a directional pan if the person who's speaking is to the left of you (for example).

 

Unfortunately we're developing not one of the most games We really like the fact that you can hear people dependent on their distance

But I have another issue. Currently we're re receiving warnings

Code (csharp):

1. Delayed playback via the optional argument of Play is deprecated. Use PlayDelayed instead!

whenever

Code (csharp):

1. src.Play( delay );

is called in "USpeakAudioManagerlayClipAtPoint:"
By replacing the method with the recommended one no audio is played anymore. Do you have suggestions how to fix that? Especially to get rid of this warning.

Thanks a lot!

 

How strange, I don't get that warning even though I'm on the latest version of Unity...

In any case, the problem here is that Play( delay ) takes int samples relative to 44100hz, whereas PlayDelayed( delay ) takes a float time in seconds. I'm guessing that's why audio isn't playing anymore, because it's delaying by a huge amount.

Try dividing by 44100.0f in PlayDelayed and see if that fixes it.

 

I have some problem about USpeakAudioManagerlayClipAtPoint().

I have tested RecordTest scene in your uSpeak package with the "Record then send" sending mode in the USpeaker component, and when I recorded for a long time(about more than ten seconds) I can't hear the whole recorded sound(I can just hear about 2 seconds). Could you check this problem for me?

ps: I can't find PlayDelayed() function. where is it?. I am using Unity 4.0.1f2.

thanks.

 

PlayDelayed should be part of the Unity audio API, perhaps it's part of 4.1?

In any case, something kind of strange happened: I tried the package, and indeed was running into the two second limit.
I had a branch version which I imported to the main branch (includes some things like lower memory usage and leak fixes), and oddly enough seems to have fixed the issue. I was able to record, and play back, for at least 15 seconds. Not sure what I did to fix it, but I suppose it's fixed
Such is the life of a programmer.

I will be submitting the fix to the asset store. I also recognize the need for using this in client-server setups like Photon and Smartfox, so the update will also be including a new set of classes to help aid integration with these solutions.

 

Thank you for your response.

I'd like to reduce the size of encoded sound with Speex codec.
The result of my test is as follows.

[TABLE="width: 500, align: left"]
[TR]
[TD]codec[/TD]
[TD]bandwidth[/TD]
[TD]duration[/TD]
[TD]size[/TD]
[/TR]
[TR]
[TD]Speex[/TD]
[TD]8k[/TD]
[TD]15sec[/TD]
[TD]45KB[/TD]
[/TR]
[TR]
[TD]MuLaw[/TD]
[TD]8k[/TD]
[TD]15sec[/TD]
[TD]58KB[/TD]
[/TR]
[TR]
[TD]ADPCM[/TD]
[TD]8k[/TD]
[TD]15sec[/TD]
[TD]63KB[/TD]
[/TR]
[TR]
[TD]Raw[/TD]
[TD]8k[/TD]
[TD]15sec[/TD]
[TD]129KB[/TD]
[/TR]
[/TABLE]












My goal is about 40 to 50KB per 1 minute and this is the result of the mobile voice chat program I am using.
I checked Speex codec's test result and I think it's possible to reach my goal with Speex codec.

How can I modify some options of Speex codec in your code?
Please help me with this problem.

ps: When are you going to submit new version to the asset store?

 

A small update with the additions I just mentioned has been submitted already.
Unfortunately, as for Speex I've already played with all of the settings and set them as low as possible (encoder.quality = 1 and encoder.vbr = true), it was little more than an experiment anyway (I cannot seem to make a satisfactory implementation of Speex, which is why ADPCM is still default)

 

I can't find new version in asset store, there is still old version submitted on March 28, 2013.
please recheck it.

 

It has been submitted - it is pending approval. It will be available presumably within a few days.

 

Here we have delay for about 1 sec. I'm not sure if I'm doing something wrong or it's a problem with the plugin.

Today I tried a newer version of the plugin and now I get this error:
Assets/Various/MoPho Games/uSpeak/Scripts/Core/USpeakAudioManager.cs(62,29): error CS1061: Type `UnityEngine.AudioSource' does not contain a definition for `PlayDelayed' and no extension method `PlayDelayed' of type `UnityEngine.AudioSource' could be found (are you missing a using directive or an assembly reference?)

could you please help?

 

If there's no PlayDelayed function that must mean you are using a version prior to 4.x?
If so it's an easy fix... replace with Play, cast parameter to int, and multiply by 44100. The difference is that Play takes samples, PlayDelayed takes seconds. In Unity 4 the Play( delay ) is deprecated which is why I replaced with PlayDelayed.

As for the delay... your audio will be delayed by approximately the play buffer length (you can see the default value is 1 in the inspector). This is unfortunately necessary, as it has to buffer up audio in order to ensure smooth playback in jittery network conditions. Try reducing it to 0.2 or so.

 

Hi,

I just bought and downloaded the lastest version of uSpeak and am just having my first glance at it but...
I see the NetworkUSpeaker prefab has 3 components with missing scripts...
I'm guessing this shouldn't be this way, or should it?

 

That should not be the case, correct.
To solve this, locate the following three scripts and drag them onto the 'Missing' slots in order:
USpeaker.cs
DefaultTalkController.cs
UnityNetworkUSpeakSender.cs

 

Hi,

I'm using uSpeak with 3d sounds. Is it possible to adjust the drop-off of the volume manually? I'm also looking for an option to set the sound volume to 0 if a certain distance is exceeded.

 

Hi Kathrin.
An update should have just been pushed to the asset store, which includes the SpeakerVolume property on USpeaker.
You can set this based on distance, clamped to 0 if far enough away.

 

I downloaded the latest version from the AssetStore but didn't find a SpeakerVolume property. In which version is it integrated?

 

Oh, I appear to be completely wrong - evidently I never submitted that version
Submitted now. If you PM me with your invoice and email I can send you the new version immediately.

 

Hi,

I have another issue that I cannot get rid of: the transmitted speech sounds "interrupted". It is like a "stuttered" speech. I noticed that the frequency of interruptions gets reduced by increasing the "PlayBufferLength".
I'm currently using a PlayBufferLength of 0.1 in order to reduce the delay until the speech is received by the client. But even with the default of 1 the issue is present. Any ideas how to solve that?

Thanks!

Kathrin

 

Which codec are you using? Additionally, can you send an audio sample for me to hear?

 

I'm using MuLawCodec. I'll send you a sample file via private message.

 

Hi,

I just read in your first post that there are 4 different kinds of voice activation integrated:

"Supports four kinds of voice trigger - push to talk, toggle talk, volume activated, and always on."

How can I use "volume activated" and "always on"?

 

Firstly, I'd like to redirect you to the new thread - http://forum.unity3d.com/threads/15...hat-Voice-Chat-For-All-Your-Multiplayer-Games
I'd like future discussion to take place there.

In the current version of USpeak, Always On is enabled by simply removing any Talk Controllers. It uses the default behavior of always sending.
To enable volume activation, simply tick the Volume Activated checkbox. Currently the threshold is hardcoded, in the future I will expose this as a slider in the inspector.

 

Ok, the next time I'll use the other thread for questions, thanks!

I tried using "volume activation", but I noticed a delay of about ~5 seconds until the sound is played at the client (I already wrote an email with a similar issue). Any ideas why this happens? I removed the DefaultTalkController and checked "volume activated" for using it. (I also reverted code changes that I implemented previously.)

Using "always on" works nicely! No delay, good quality! But it would be nice for use to use "volume activation" in order to keep the bandwidth low.

 

Very strange - did you try the Local-only demo and see if the delay is present there as well?
(also coming soon is an update that adds a configurable threshold volume - if you want the update, send me your email and I'll add you to the mailing list)

EDIT: Hm, I can't seem to replicate this, even with networking turned on. I have my OUYA (go, microconsoles!) loaded up with a demo build of the included Unity Networking scene. Mic is plugged into my PC, as I speak into the mic even with Volume Activated enabled, and no Talk Controller present (always send), I get a lag of about perhaps 3/4 of a second before voice is played back.

EDIT 2: I've got a question... what's your Send Rate set to?

 

Strange thing, I failed to reproduce it even in my application... might have been something else that went wrong yesterday. Thanks for your effort!

My send rate is set to 15.

 

Glad to hear it's resolved. I asked about send rate because it was a possible a very low sendrate could cause the issue, but after some testing it would have to be VERY low in order to cause a delay on the order of 5-10 seconds. Certainly not a sendrate of 15.

Is it possible that your internet connection was acting up yesterday I suppose? On occasion my internet is pretty good, but two days ago I got the worst lag I've ever experienced (Pingtest rated my internet as F when it usually gives it an A, and I got a packet loss of a whopping 20%) so such things do happen.

 

I actually have the same issue with my game. We have had USpeak setup for months and we haven't updated or touched it. All of a sudden we get a random warbling of the voice. Sendrate = 16 and we use the MuLawCodec with Wide Bandwidth. Everything was perfect and now when someone talks it has this stuttering EX: someone talking sounds like this terrible description: "Hello my *sprch" na *sprch* me " It goes low and you can hear a vibration kind of noise that warbles the voice. It's dreadful because it was perfect before and we hadn't changed anything.

 

Firstly, I'd like to redirect you to the new thread - http://forum.unity3d.com/threads/15...hat-Voice-Chat-For-All-Your-Multiplayer-Games
Please move future discussions there.

Secondly, please:

1.) Download the latest update from the asset store. Should be version 3.0. If you did not receive USpeak from the Asset Store, please send me your email and I will add you to the update mailing list and send you the latest version.
2.) Give the Local demo a try. If the warbling is not present, then most likely it is an issue with your internet connection. If it does exist even in the Local demo however, please let me know and if possible send me the exact settings used (codec, bandmode, sendrate, etc) so I might be able to reproduce it.

 

Hello So I just bought this Asset, and I;m trying to implement it into my game using Photon. I get this error when I start up the game:

USpeaker requires a component which implements the ISpeechDataHandler interface
UnityEngine.Debug:LogError(Object)
<Start>c__IteratorC:MoveNext() (at Assets/uSpeak/Scripts/Components/USpeaker.cs:507)

I understand what this error is saying, i just don't know how to fix it becuase I can't really code in C#. Any ideas?

 

What's your speech data handler class look like?

 

Hey,

I'm trying to get uSpeak to work in my Unity project with SmartFox. it is fully capable of sending voice packets between two people and the audio is smooth but any more than two and some serious lag starts happening. this is due to a sort of packet overload on the receiver.

I'm trying to figure out how to shorten the send rate of the packets. when I change the send rate to 1/sec in the uSpeaker options I don't get any audio. Is this because it is only send 1/16th of the seconds worth of audio? how can I go about lengthening the audio sample being sent?

thanks in advance,

Matt

 

Here's something to try. Could you swap out for any other codec besides Speex and see if it fixes the problem?

 

Hi PhobicGunner,
I'm trying to get USpeak Voice Chat to work with AppWarp game engine. Generally, the worklfow is pretty much simple: I want to record users voice, retrieve the recorded and compressed data as a byte[] (using Uspeak API),send it with AppWarp API which allows max 1K data size, receive the encoded data on the receiver's side as a byte[] (AppWarp API), decode and play it (Uspeak API).
In other words, I'd like to use your framework as a recording and encoding/decoding one only, with no network part at all.
Is that possible? At this stage I do not understand how to get recorded data, encoded and compressed, once it is recorded. Looked at USpeakAudioClipCompressor.CompressAudioData() function, but it looks like as a kind of internal API. And USpeakOnSerializeAudio is supposed to get decompressed audio, isn't it?
10x in advance

 

Yes I have tried using Speex, ADPCM, and MuLaw and between the three Speex has the best audio quality with no problems between two people but buggy if more than two people. there is a seperate audio buffer for every player in the scene, the problem seems to be too many packets at once.

 

Yes Speex is quality-wise the best of the three. BUT what I wanted to know is if the jitter/choppiness problem goes away with the other two codecs? If not, then I can safely rule out a few possible causes.

 

no, the jitter does not go away with mulaw being the worst adcpm jittering if multiple people are talking at once.

 

So what happened to Uspeak is it not on the store anymore? I can't seem to find it.

 

It was replaced with DFVoice.

 

Good looking out! Did they ever manage to get webcam streaming working?

 

I also want to have a test package.Thank you to help!