[RELEASED] USpeak Voice Chat - Voice Chat For All Your Multiplayer Games

Hello,

I'm using USpeak with TNet and I'm trying to disable the echo of the speaker's voice in his own ear. I have the USpeaker set with "Send while recording" and ShouldSend() returns true to keep the mic open. Basically, it appears that any time speaker 2 is transmitting to speaker 1, it also transmits what it received from speaker 1. This happens even when speaker 2's headphones are silent, so it's not audio feedback.

Do you know what's going on here?
Thanks

 

Is "Debug Playback" enabled?

 

No. It's false.

 

What input device is Speaker 2 using?

EDIT: Scratch that.
Stick a debug log in your RFC and log which TNPlayer the RFC was called from.

 

Hmm, not sure how to determine where it was called from. Printing the tno.ownerID in the RFC call and function shows both clients sending them.

I've made a small test project to demonstrate it. Do you have an email I can send a link to?

Thanks

 

I sent a private message to you with a demo project on Friday. Did you get it?

Basically, whenever both mics are open, the listener echos what is said in the sender's mic. This happens even with volume off on the listener's headphones, so it's not picking it up via the mic.

 

I did, and sent you back a PM, did you get it?
I can't see to reproduce this, even with the exact project you sent me.
Just as a sanity check how close are your two machines? Maybe disable the microphone of one just in case?

 

Sorry, I didn't see the message till now.

It really is possible that it's the mic picking it up, but in testing it was really like we were speaking directly into our own ears. I'll try to set up an isolated test to see what happens.

Thanks!

 

Well, it looks like it was a hardware/user issue. The input on my mic wasn't 100% in the socket, so it connected the mic ring on the connector to one of the headphone lines causing the audio to feedback into the mic line. Kind of surprised it worked at all.

Thanks for your help. Sorry to waste your time.

 

Hey Phobic!

Great product and easy TNet integration, however the audio quality seems to suffer a lot when more than one person is talking. Is there a way to mitigate this?

Thanks

 

Is all of your voice data being routed through a single USpeaker object (game object with USpeaker component)? If so, try using multiple objects (1 per player).

 

Hey phobic, was wondering if you could help me with this.

I'm very new to USpeaker, but understand some C# coding and what not..

The problem is, the quality of the voice chatting is very bad, its sharp at times, loud, and cuts in and out so much, you can't really hear what the person is saying.

I'm not sure if I can swap out codecs or what. I'm not sure how USpeak is built, so I'm not to familiar with its setup.
If you could get back to me soon, that'd be great!

Forgot to mention, I'm using this in my Photon multiplayer game.

 

Which codec are you using? (should be a dropdown in the USpeaker inspector)
Does it happen with one person speaking, or multiple people speaking at once?
Is Volume Activated enabled?

 

Every player has their own USpeaker player. I should also note there are ~20 in a channel when this occurs. I've just added a distance threshold for voice data being sent, as the 3D voice drop-off seemed to never hit 0.

An additional question, what is the best way to equalize audio? I'm using 3D audio but you can tell different users have way different mic input levels. If they are right next to you, one will be soft while the other is loud.

Any pointers?

Thanks

 

Changing the encoding to MuLaw fixed the multiple people talking at once issue!

Thanks

 

That's a very good hint.
I might need to re-arrange how codecs work, especially Speex. It could be that Speex is designed to handle one and only one audio stream, and so if you feed it multiple simultaneous streams it could start sounding strange...
I'll have to revisit this issue when I have time.

 

Hello! Help me pls.
I bought your plugin Uspeak - Tnet edition and I have two questions.
1) I have a very big world in the game and players must hear each other only if they stand close. Please tell me how to do this? (loudness dependence on the distance)
2) players must be able to communicate with each other via In-game pocket-radios. at this point their's talking should be isolated from all other players talk and should not depend on the distance between the players. Please tell me how to do it?

=====================
not waiting for an answer, I finally figured put prefab speaker for each player into his game character, the number one problem was solved. Players now can hear each other only when they are close to each other.
The second problem is still not resolved

When a player uses the phone in the game and calling a friend who might be far away. how to establish a connection between them at the time of phone-talking?

 

Hey PhobicGunner!

First of all, congrats for this great asset!
It was easy to get uSpeak working in my project, but I'm wondering if you could help me with a problem I'm dealing in the last days.

I want to enable users to record some speech (saved in a server, in the ReceiveAudio method). To accomplish that, using the speex codec, I am trying to use the OggSpeexWriter from the Nspeex.dll. (http://nspeex.codeplex.com/SourceControl/latest#main/NSpeex/OggSpeexWriter.cs)

I'm sorry to present you such a detailed doubt, but I'm struggling with this for some days.. and perhaps I'm making some basic mistake.

For that, the first time I receive audio, I initialize it:

Code (csharp):

1.  
2. Stream newStream = new FileStream("cenas2.ogg", FileMode.OpenOrCreate);
3. myWriter = new OggSpeexWriter (1, recFreq, 1, 1, false);  // mode, sampleRate, channels, nFrames and vbr
4. myWriter.Open (newStream);
5.  

Then, when iterating in the While(offset <data.Length), among the other code to play the audio, I try to write to the file:

Code (csharp):

1.  
2.  
3. //to deal with the first bytes, which have the length
4. byte[] len_bytes = USpeakPoolUtils.GetByte( 4 );
5. System.Array.Copy ( cont.encodedData, len_bytes, 4 );
6. int dataLength = System.BitConverter.ToInt32( len_bytes, 0 );
7.  
8. USpeakPoolUtils.Return( len_bytes );
9.  
10. //To get the bytes to write to the .ogg file
11. byte[] actual_bytes = USpeakPoolUtils.GetByte( cont.encodedData.Length - 4 );
12. System.Buffer.BlockCopy( cont.encodedData, 4, actual_bytes, 0, cont.encodedData.Length - 4 );
13. myWriter.WritePacket(actual_bytes, 0, cont.encodedData.Length - 4);
14.  

And then, when I stop receiving data, I write the Header:

myWriter.WriteHeader(""); //Not using Comment
myWriter.Close ();

Do you spot some possible mistakes? I have tried some variations, but none of them work (it writes to the file, but I can't play it).

Thanks a lot,
João Guerreiro

 

As far as I know, the Ogg speex writer doesn't actually write out a playable Ogg file. What it writes out is speex data, wrapped in an Ogg container format.

The question is, do you intend the recorded speech to be playable outside the game, or inside the game?
If inside the game, you could just take the raw byte array you received and directly write that as-is to a file, then later retrieve the raw bytes and pass them to ReceiveAudio (this solution wouldn't even require modifying USpeak - you could do it directly in your own network code)
If outside the game, the only solution I can think of is to write out a WAV file (I don't think there's any good solution for C# for most popular codecs like MP3 or Ogg Vorbis, but WAV is pretty much just a raw dump of the samples plus a header)

 

We're implementing this in a mobile game, and have a couple of questions:

1) Maybe this is an idiot thing to ask...but does this require a headset? It works great with them, but using the iPad microphone results in lots of feedback loop/echos (player 1's voice played on player 2's device and being picked up by the latter microphone).

2) On iPad 2, performance is pretty bad -- 20ms for ADPCM or MuLaw Codec (which is 2/3 of maximum ms on that device). Can we optimize that?

3) Running it in the editor, if we change microphone inputs (plug in a headset, or unplug it), that seems to kill all audio transmissions. Do we need to add something to let players choose the new microphone?

 

1.) There's no echo suppression built in (actually, if you've played Xbox LIVE you'll often hear echos over voice chat for the same reason - players have their audio outputting through their TV speakers, and so their microphone picks up the game audio and re-transmits it), headphones are probably ideal.
2.) I've never encountered such performance. Do you have a profiler screenshot? ADPCM in particular should be lightning fast - what kind of performance do you get with MuLaw?
3.) There is a function available which allows you to switch the microphone device. Presumably, you'd use this from an options screen to let the player choose their device (so they could plug in their headset, go to the options menu, and choose their headset from the device list).

 

thanks for the quick reply! Regarding #2: Turns out half of that 20ms was debug stuff, and much of the rest of it is photon sending the audio data. Any suggestions for optimizing the latter? How many ms *should* an iPad 2 require to send voice data?

 

How many times per second are you sending voice data? What's your bandwidth mode?

 

Hello, PhobicGunner, and thanks for your response. I'm the developer working on gecko's project.

At the moment, we haven't changed any of the default settings, besides the codec and turning "Debug Playback" off. So "Wide - 16kHz" bandwidth mode, and 16/sec.

 

ADPCM + 8Hz + 8/sec seems to bring the cost down to 3ms or so (and that's when I continuously babble into the microphone). So looks like everything's good. Do you think this is what we should expect?

 

3ms is definitely closer to where it is on PC (I had a bit of back and forth with Facepunch Studios on performance, and we managed to get it to 1-2ms on PC with the default settings - although that was also with uLink, which could potentially be better optimized than Photon)

 

This is odd...

I'm trying to test all the combinations of codec, bandwidth, and send rate by writing a separate GUI script that allows changing the respective values at runtime (that is, USpeaker.Codec, USpeaker.BandwidthMode, and USpeaker.SendRate). However, once I've used it to access the values (not even necessarily change them), the CPU costs of talking suddenly jump to around 80ms on the quicker codecs. This continues forever, even if I close the GUI again, so it doesn't access anything. Here is the script - I've added it to the same prefab that has the USpeaker and sender:

Code (csharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class USpeakerSettingsController : Photon.MonoBehaviour {
5.  
6.     private USpeaker uSpeaker;
7.     private bool open = false;
8.  
9.     public void Awake(){
10.         uSpeaker = GetComponent<USpeaker>();
11.         if (!photonView.isMine){
12.             return;
13.         }
14.         useGUILayout = false;
15.     }
16.  
17.     public void OnGUI(){
18.         if (!photonView.isMine){
19.             return;
20.         }
21.         GUI.color = Color.cyan;
22.         if (!open){
23.             if (GUI.Button(new Rect(Screen.width - 200.0f, 90.0f, 170.0f, 40.0f), "Open USpeaker GUI")){
24.                 open = true;
25.                 useGUILayout = true;
26.             }
27.         } else {
28.             GUILayout.BeginArea(new Rect(30.0f, 100.0f, Screen.width - 60.0f, Screen.height - 60.0f));
29.             GUILayout.BeginVertical();
30.             GUILayout.Label("Codec");
31.             GUILayout.BeginHorizontal();
32.             if (GUILayout.Toggle(uSpeaker.Codec == 0, "Speedx")  uSpeaker.Codec != 0){
33.                 uSpeaker.Codec = 0;
34.             }
35.             if (GUILayout.Toggle(uSpeaker.Codec == 1, "ADPCM")  uSpeaker.Codec != 1){
36.                 uSpeaker.Codec = 1;
37.             }
38.             if (GUILayout.Toggle(uSpeaker.Codec == 2, "MuLaw")  uSpeaker.Codec != 2){
39.                 uSpeaker.Codec = 2;
40.             }
41.             GUILayout.EndHorizontal();
42.             GUILayout.Label("Bandwidth mode");
43.             GUILayout.BeginHorizontal();
44.             if (GUILayout.Toggle(uSpeaker.BandwidthMode == BandMode.Narrow, "8kHz")  uSpeaker.BandwidthMode != BandMode.Narrow){
45.                 uSpeaker.BandwidthMode = BandMode.Narrow;
46.             }
47.             if (GUILayout.Toggle(uSpeaker.BandwidthMode == BandMode.Wide, "16kHz")  uSpeaker.BandwidthMode != BandMode.Wide){
48.                 uSpeaker.BandwidthMode = BandMode.Wide;
49.             }
50.             if (GUILayout.Toggle(uSpeaker.BandwidthMode == BandMode.UltraWide, "32kHz")  uSpeaker.BandwidthMode != BandMode.UltraWide){
51.                 uSpeaker.BandwidthMode = BandMode.UltraWide;
52.             }
53.             GUILayout.EndHorizontal();
54.             GUILayout.Label("Send rate: " + uSpeaker.SendRate);
55.             float newSendRate = Mathf.Round(GUILayout.HorizontalSlider(uSpeaker.SendRate, 0.1f, 32.0f));
56.             if (newSendRate != uSpeaker.SendRate){
57.                 uSpeaker.SendRate = newSendRate;
58.             }
59.             GUILayout.EndVertical();
60.             GUILayout.EndArea();
61.             if (GUI.Button(new Rect(Screen.width - 200.0f, 90.0f, 170.0f, 40.0f), "Close USpeaker GUI")){
62.                 open = false;
63.                 useGUILayout = false;
64.             }
65.         }
66.     }
67. }

 

Here's a crazy idea... try deleting everything but the calls to photonView.isMine. I wonder if that's the slowdown here.

 

I'm sorry, I don't understand. If I delete almost everything, then I have no script...

The 80ms spike happens when I talk after using my script, not while I'm using the script. So it's not the GUI doing this.

 

OK, but it only happens when this script is attached, correct? So it must be caused by *something* in the script. It's got to be in the OnGUI function since that's the only function that runs more than once.
First, try commenting out the entire script and see if the slowdown still happens. It shouldn't, but check and be sure.
Then uncomment the private fields and the Awake function and check again.
Then uncomment the GUI function and check again.
etc. Process of elimination, basically.

 

I've already eliminated everything after line 27, because I know the spike happens after I press the "Close USpeaker GUI" button, which switches off the "open" field. It seems to make more sense that once I've accessed the USpeaker properties, something happens in USpeaker that makes it take more CPU than before.

(If you saw my message before my quick edit when it said this script only needed to exist, then I'm sorry for that confusion. It does seem to require that it runs the main GUI part at least once.)

 

But yeah, I can try to narrow it down to a particular call of course.

 

Except now it doesn't happen anymore... I suppose it's possible that I accidentally switched to Speedx every time, though that would be weird... Thanks for your responses.

 

I noticed that changing the SendRate field at runtime programmatically has no effect - it doesn't print out "Settings changed" like if I change the codec or the bandwidth. This seems to be because USpeaker sets the field "sendt" to 1/SendRate at the Start function and never uses "SendRate" again. Is this intentional?

 

Well, it never triggers settings changed because the purpose of that function is to allow you to send a message to other players (if you change codecs, other players need to know what your codec is so they can decode your audio).
That said, I never did consider allowing it to be changed at runtime (admittedly, it's a fairly limited use case - you generally tweak it to fit your game and then leave it alone)

 

Thanks for the response, PhobicGunner.

After trying for some more time, I'm now reasonably sure that none of the properties (or at least the ones I'm interested in: codec, bandwidth, and send rate) can be updated at runtime. It does seem to update the "settings" structure under USpeaker, as well as the properties themselves, to match what I've selected on my GUI, and it does seem to send the message over the network, but it doesn't seem to affect the actual performance. That is, if I set the Speex codec and 32 to both bandwidth and send rate in the editor, I'm going to get frame times in the hundreds on my iPad 2, even if I use my GUI to switch to another codec and 8 to bandwidth and send rate. Then again, if I set the settings low in the first place, I'll get decent performance no matter what I change.

At the moment, I've just been trying to test them all without having to rebuild every time. Would probably have been quicker to just rebuild, given the problems, but I can see another reason to change the settings at runtime: We'd probably like to make the settings depend on the user's platform - poor quality for low-end devices and high quality for high-end devices. Or is this possible at the very beginning, the way it's set up not, just not once everything has been set up?

 

Except now changing the codec does seem to matter, at least when I change to Speedx. Oh well.

 

uSpeak is not available from Asset Store anymore? Will it be available anywhere else? Where to submit patches?

 

That's a message from the uSpeak for TNET thread.

If you check the asset store, all versions of uSpeak have been removed.

 

Hey guys, since uSpeak was taken off the asset store, I found it on unity magic.com it says the version is 3.0, is this the latest version?

 

It looks like this isn't on the asset store anymore. Can you point me to where I could purchase this asset?

 

USpeak has indeed been pulled from the asset store, along with all other middleware offered by MoPho' Games.
However, as part of my partnership with DaikonForge, we will be releasing a sort of spiritual successor to USpeak, called DFVoice, under the DaikonForge name.
If you have a project in mind you would like to evaluate DFVoice with, please don't hesitate to apply for the closed beta.
Details here: http://forum.unity3d.com/threads/248594-DFVoice-VoIP-Closed-Beta-Invitation

 

If I get into the closed beta, do I have the right to use it in a commercial game?

 

Yes. Beta users continue to get updates and have all rights forever, even after the closed beta is over.

 

I really was looking forward to using this to help me set up my multiplayer game to have in game audio sources emit sound from players microphones (speakers, etc.) but now it's gone. Is there any way to get a hold of this? I'm a bit lost without it.

 

Hey there, what are the chances that users who purchased uSpeak way back when can get a version of DF-Voice?

 

Sign up on the DaikonForge forums and send me (MasaakiHosoi) a PM with your invoice number.

 

Manually made your account valid. Try logging in.

 

Hi Phobic,
I am already using USpeak in a current project, but was not able to download the update to version 3.0 before you pulled it from the asset store. If I could send you the invoice for the purchase, would you be willing to send me the update?
Thanks!

Edit: I forgot to mention, I tried to register on the DF forums so I could PM you there, but I have been unable to receive the confirmation e-mail. My account there is the same user name.

 

Your account has been manually validated.