[Released] Dissonance: Unity Voice Chat

I found the root cause, we are calling DontDestroyOnLoad for DissonanceSetup game object to make sure it will be transferred to other scenes. Dissonance starts working fine on reconnect when I removed this instruction. So how we should properly configure Dissonance for such multi scenes case?

 

Hi ExyteHQ, I'm on Christmas holiday at the moment. I'll have a proper look at this in the new year

From a very quick skim it looks like something isn't being torn down properly when the disconnection happens, when you reconnect the old state breaks the initialisation of a new audio capture system and everything falls apart from there. The error "Error: Cannot associate preprocessor with Playback filter - one already exists!" indicates that a bit of state (the playback filter) has been setup and was never torn down.

Another thing I spotted is that it also looks like something is causing

DissonanceComms.Start

to be called after the disconnection (maybe when you reconnect) and I guess this might cause the initialisation to run twice which would cause the same error. Is it possible you have two instances of DissonanceComms at once?

Edit: Sorry, I just saw your second post there. Could don't destroy on load combined with loading scenes be creating two instances?

 

Just bought the asset and can't get it to work. What's the support email?

I just want to use proximity chat but can't hear anything. The "Tracking" variable of Hlapi Player is always false unlike in the demo, which will be true after connecting to server.

 

Hi Dreamer, You can email me for support at martin@placeholder-software.co.uk. I'm on holiday at the moment so I may not reply until the new year

`Tracking` being false by itself shouldn't completely mute voice (it would just disable positional chat until tracking is initialised). There must be something deeper which is breaking voice (e.g. no active broadcast trigger, no active receipt trigger, networking broken somehow). Have a look at the DissonanceComms component, it shows a load of useful information (e.g. what channels you are sending on and what channels you are listening on).

 

Hi Nyarlathothep, DissonanceComms has information as below:




I didn't add receipt trigger at all. Because I tested in the demo and deleted all the receipt triggers, room A, room B just to test the proximity chat. And it works just fine without receipt trigger.

I mannualy add 5 components to player objects (both local player and remote players) after they are network spawned:
1. Hlapi Player, 2. Voice Broadcast Trigger(Set to Self), 3. A shpere collider trigger, 4. trigger visualizer (I can see the circle after player connects), 5. speaker indication (It doesn't work no matter how close I stand in the circle)

When I speak, I can see mic amplitude value change. But can't hear and speaker indication doesn't pop up like in the demo.

I'm new to this plugin. Tried many times and still can't get it to work.

 

Just in case you're not sure I'll clarify exactly how triggers/channels work:

There are two types of channel: Room and Player. If you broadcast voice to a room (e.g. using a broadcast trigger with channel type: room) no one will hear it unless they are listening to that room (e.g. using a receipt trigger). If you broadcast voice directly to a player (e.g. using a broadcast trigger with channel type: player) that player will hear you even if they don't have a receipt trigger set up.

Using Channel Type: Self is a special case of Channel Type: Player which automatically fills in the player ID for you based on a sibling player tracking script. That explains why you can't hear anything - the broadcast trigger effectively isn't doing anything because tracking isn't initialised and so it doesn't know what ID to use.

The root cause is that your player does not have Local Player Authority set to True. I just briefly tested this and setting it to False causes the the tracking script gets confused about things (it detected the local player as Remote, and Tracking never became true). If you open an issue on the tracker (https://github.com/Placeholder-Software/Dissonance/issues) I'll in investigate it properly once I'm back at work

As a workaround writing your own player tracking script is pretty simple - you just need to implement the IDissonancePlayer interface, return the information it asks for and call comms.TrackPlayerPosition(this) and comms.StopTracking(this) when you want to start/stop tracking a player. The only complex part is syncing the ID across to all of the clients so every client know which ID is represented by which tracker - the current script uses RPC/Command attributes to sync the ID and I guess one of those is being broken by not having the necessary authority.

 

It's good know what is the root cause so I have some clues.
Local Player Authority is set to be false for all server authrotative online games. So it's all controlled by server and difficult to be hacked.
I think it's a important feature that should be included in this nice plugin. I will open an issue as you mentioned.
Meanwhile, I will look into it. Hopefully, you will come with a fix soon. Thx.

 

@Dreamer I'm back off holiday now so once I've cleared my backlog of support emails I'll be looking at the HlapiPlayer tracker and whether I can make it work without Local Player Authority. Have you had any luck doing it yourself?

@ExyteHQ do you still need help with your issue or did you manage to resolve it?

 

I moved on other parts of project.
As I'm not familiar with the plugin, it will cost me a lot more time to dig into it and it's difficult for me. I'm waiting for a fix. And you are waiting for me these days?

 

Hi Dreamer, I actually developed a fix 4 days ago but I forgot to update you. I'm really sorry about that!

I've posted a couple of links on your issue tracker issue where you can downloaded the modified files. Just overwrite the files in your project with those to apply the fix. This fix should be included in the next Dissonance release (probably within a couple of weeks).

 

Still doesn't work. And it shows No player id or Tracking for other remote players.

 

Hi Dreamer, sorry about that. I went back and had a closer look and spotted something that I'd missed - it worked if you were looking at it from the perspective of a client, but not from the perspective of a mixed client/server! I've posted another link in the issue tracker to overwrite the HlapiPlayer.cs with some additional fixes.

 

It works now if HlapiPlayer and VoiceBroadcast Triggers are predefined on the player prefab. (It didn't work before your last fix)

But it doesn't work if these components are added by script in game.
Because not all the players object in game require voice-chat components for my project. For each player object in game that requires voice-chat, these components are added in player's initialization script after player is network spawned on Server, local client, remote clients.

By adding these components by script in game, voice-chat doesn't work. And HlapiPlayer now shows type "Unknown", tracking is false, no player id.

 

The script uses two methods from HLAPI to start itself up:

- OnStartClient
- OnStartLocalPlayer

So if you add your script later those methods never get called because the startup events have already happened before the script even exists. I experimented with using isLocalPlayer instead to avoid needing these method calls but that wasn't very reliable (it looks like isLocalPlayer doesn't get set straight away). Unfortunately I don't know of any options for detecting if this is the local/remote player besides using isLocalPlayer property or the OnStartLocalPlayer method.

I would suggest simply calling that methods yourself when the script is added.

Another option would be to have a player tracker attached to every player so it gets initialised automatically and then adding/enabling the VoiceBroadcastTrigger component later.

 

To let OnStartClient, OnStartLocalPlayer work, I now put the AddComponents in Awake. Now it works, thanks to your explanation.

1. There is still one issue. If UseColliderTrigger is set true, I can't hear anything. And the "Trigger Visualizer" won't be highlighted like in demo when I'm in remote player's circle. I want this to be true, so it won't send voice over network the whole time but only send when it's necessary to save resources. (Because it's 60-100 players per match)

2. To maxmize the performance of online game, I tend to keep the components on objects as less as possible.
Can you tell me to make proximity chat work in my topdown shooter game, which is required and which is not required on Client Local Player Object, Client Remote player objects, Dedicated Server?

Hlapi Player (Does server require it?)
Broadcast Trigger
Sphere Collider
Trigger Visualizer (I won't add it in final game, so don't worry about this one)
Speaker Indicator (I want to show it in final game, so I know who talks. Server doesn't require it, right? And how about local player object?)

 

• HlapiPlayer - required on clients, but not on the server. Server neither sends of receives audio so it doesn't matter at all.
• Broadcast Trigger - required on all clients attached to all local copies of remote players.
• Sphere Collider - Same as Broadcast Trigger.
• Speaker Indicator - Same as Broadcast Trigger.

Generally everything in Dissonance is done on the client side. So if we look at this from the perspective of a client (Alice) trying to talk to another client (Bob):

Alice walks around the environment, she can see a game object which is her local representation of Bob. When she walks near the Bob gameobject she triggers the Sphere Collider (locally), this checks if the triggering gameobject has a HlapiPlayer component tracking the local player, if it does it activates the Broadcast Trigger (locally) and starts sending her voice to Bob.

Now the other way around. When Bob hears Alice speaking the local Speaker Indicator component attached to the Alice gameobject first looks at the HlapiPlayer object (to discover which remote speaker this object represents) and then looks at the audio passing through the local audio decoding pipeline assigned to that player and pulses the orb as the amplitude varies.

Hopefully that makes it clear why the components need to go where they do

> If UseColliderTrigger is set true...

According to the above info is your trigger setup correct? Is the tracker tracking? If you just log when the trigger is triggered does it happen when you expect? Keep in mind that the trigger only triggers Dissonance broadcast if the triggering object has a HlapiPlayer component tracking the Local player.

 

1. After investigation, I found why it's not triggered. My players' collision layer set to not to be collided with each other for my gameplay. If I set collision layer to be true, it works. What I can do to make it work and avoid to change gameplay?

2. When I don't add HlapiPlayer on server. It will trigger and error on clients and I can't enter the game. (It won't happen if I add HlapiPlayer on server)

IndexOutOfRangeException: NetworkReader:ReadByte out of range:NetBuf sz:45 pos:45
UnityEngine.Networking.NetBuffer.ReadByte () (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkBuffer.cs:35)
UnityEngine.Networking.NetworkReader.ReadUInt16 () (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkReader.cs:179)
UnityEngine.Networking.NetworkReader.ReadString () (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkReader.cs:290)
Dissonance.Integrations.UNet_HLAPI.HlapiPlayer.OnDeserialize (UnityEngine.Networking.NetworkReader reader, Boolean initialState)
UnityEngine.Networking.NetworkIdentity.OnUpdateVars (UnityEngine.Networking.NetworkReader reader, Boolean initialState) (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkIdentity.cs:751)
UnityEngine.Networking.ClientScene.ApplySpawnPayload (UnityEngine.Networking.NetworkIdentity uv, Vector3 position, System.Byte[] payload, NetworkInstanceId netId, UnityEngine.GameObject newGameObject) (at C:/buildslave/unity/build/Extensions/Networking/Runtime/ClientScene.cs:491)
UnityEngine.Networking.ClientScene.OnObjectSpawn (UnityEngine.Networking.NetworkMessage netMsg) (at C:/buildslave/unity/build/Extensions/Networking/Runtime/ClientScene.cs:552)
UnityEngine.Networking.NetworkConnection.HandleReader (UnityEngine.Networking.NetworkReader reader, Int32 receivedSize, Int32 channelId) (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkConnection.cs:469)
UnityEngine.Networking.NetworkConnection.HandleBytes (System.Byte[] buffer, Int32 receivedSize, Int32 channelId) (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkConnection.cs:425)
UnityEngine.Networking.NetworkConnection.TransportReceive (System.Byte[] bytes, Int32 numBytes, Int32 channelId) (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkConnection.cs:576)
UnityEngine.Networking.NetworkClient.Update () (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkClient.cs:753)
UnityEngine.Networking.NetworkClient.UpdateClients () (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkClient.cs:965)
UnityEngine.Networking.NetworkIdentity.UNetStaticUpdate () (at C:/buildslave/unity/build/Extensions/Networking/Runtime/NetworkIdentity.cs:1092)

 

Sorry, looks like I was mistaken about not needing HlapiPlayer on the server. Dissonance itself doesn't need it on the server (the Dissonance server is pretty much just a packet relay, it doesn't need to know player identities). However the HLAPI system needs it on the game object so that it can relay the commands/RPCs used to setup the HlapiPlayer component across the various clients.

Can you separate out the VoiceBroadcastTrigger and collider trigger into a child gameObject and then put that on a different layer?

You'll need to make sure that the gameObject with the HlapiPlayer component can collide with the collider trigger. The VoiceBroadcastTrigger will ignore all collisions which do not involve a HlapiPlayer component.

Self broadcast will no longer work, since the VoiceBroadcastTrigger is not a sibling of the HlapiPlayer component. Instead you'll have to write a script to copy the ID from HlapiPlayer.PlayerId component to the VoiceBroadcastTrigger.PlayerId property and use ChannelType Player. Alternatively you should be able to replace this code (VoiceBroadcastTrigger Line 275):

_self = GetComponent<IDissonancePlayer>();

with:

_self = GetComponentInParent<IDissonancePlayer>();

But I haven't tested that yet.

 

I create a prefab that contains VoiceBroadcastTrigger, Collider, Indicator, Visualizer. I Instantiate prefab and set it's parent to player object on Start().

And I have tried both methods:
1. VoiceBroadcastTrigger Set to Player. Copy player name to VoiceBroadcastTrigger.PlayerId.
2. VoiceBroadcastTrigger Set To Self. And use _self = GetComponentInParent<IDissonancePlayer>();


The first method doesn't work. I can't hear any thing. Even when UseTrigger on VoiceBroadcastTrigger is off.

The second method works for me. Hope you can make this temp solution to an option for plugin. So that when everytime I update to the latest version, the script won't be over written.

 

I've already merged it into the plugin so it will work this way in all future versions

 

Dissonance 6.3.0 has been released on the asset store!
It's been a long time since the last version of Dissonance was released (over two months) and we've accumulated a lot of fixes and improvements.

There's a potential fix for the Android Audio effect Dissonance Echo Cancellation could not be found error (see here for details). We've added support for Tiny Frames for ultra low latency LAN only voice chat (useful for shared space VR). We've modified the HlapiPlayerTracker to support server authority and have also made a number of small quality-of-life changes around the VoiceBroadcastTrigger Self mode (this one's for you @Dreamer ). There's also a new IsMuted property on the VoiceBroadcastTrigger, this can be combined with VAD activation to create a UI Push-To-Talk mode for phones/tablets.

That's just the highlights! Check out the release notes for all the details as usual.

Unity Version Support
We originally developed Dissonance using Unity 5.6 (back when that was the cutting edge) and we've automatically supported every new version since then. This has made our release/support process very complicated as there are subtly different rules for binary plugins on all the different target platforms on all the different Unity versions. Dissonance 6.3.0 is the last version which will be released for all the platforms from Unity 5.6 upwards.

In the future we have decided to release new versions of Dissonance for platforms similar to what Unity themselves support. Unity have TECH and LTS releases, the only Unity supported platforms are the LTS ones for two years after release and the latest TECH release. See this blog post for more details. We will support slightly more than this:

• All supported LTS versions
• All TECH versions after the latest LTS

That means that if we were to release a new Dissonance version right now it would be submitted to the asset store for:

• 2017.4 (LTS)
• 2018.1
• 2018.2
• 2018.3 (TECH supported by Unity)

 

Dissonance 6.3.1 has been released on the asset store!
Dissonance 6.3.1 has two really cool features which have both been driven by requests from users; Mirror Networking integration and Magic Leap support! Check out the full release notes for all the details and to download the integrations.

Mirror Networking
Mirror is a high level networking library designed to be a drop in replacement for the deprecated UNet HLAPI. With this release we've included a new Dissonance integration for using Mirror that makes it easy to setup voice chat.

Mirror supports multiple "network backends" which implement the low level details of how packets are sent, by default it uses the Telepathy TCP backend. However TCP is not suitable for voice comms - if you're using Mirror with Dissonance make sure to swap the backend to one which supports Unreliable packets such as Ignorance.

Huge thanks to BigBoxVR for sponsoring development of this integration!

Magic Leap
Magic Leap One is an incredible Augmented Reality device. We've worked with Dissonance users and magic leap engineers to develop and test Dissonance support for Magic Leap One. Thankyou to everyone who was involved in helping to make this happen.

Version Support
As promised last time we've now reduced the number of Unity platforms we support. Dissonance 6.3.1 has only been submitted to the asset store for Unity 2017.4 and up.

 

The setup prefab and demo scene are missing from the HLAPI integrations in version 6.3.x ?
(I'm trying to follow the tutorial)

 

Sorry about that @Snorkel, I'm not sure how that happened. Thanks for reporting it!

I've uploaded a new integration package which should include all the necessary bits. It's available here: https://placeholder-software.co.uk/....3.1/Dissonance-UNet_HLAPI-6.3.1.unitypackage (also available on the latest release notes as usual).

 

Critical Bug In Dissonance 6.4.0
One of the changes we made in Dissonance 6.4.0 was to lazy load the native plugins. This means that platforms that don't handle voice audio (i.e. dedicated servers) never need to load those dependencies which can make managing your servers easier. However it appears that this change has encountered a bug in Unity - around one time in 10 the native dependencies simply don't load in player builds which disables all audio processing! This affects all editor versions.

We've submitted Dissonance 6.4.1 to the asset store which will fix this, once it is available make sure to update your copy of Dissonance and integration packages.

Before that is released there is a simple workaround you can apply:

1. Open DissonanceComms.cs
2. Find the `Start` method (line 205)
3. Modify the method so it starts like this:

Code (CSharp):

1. [UsedImplicitly] private void Start()
2. {
3.     // Add this code vvvvv
4.     try
5.     {
6.         TestDependencies();
7.     }
8.     catch (Exception e)
9.     {
10.         Log.Error("Dependency Error: {0}", e.Message);
11.     }
12.     // Add this code ^^^^^
13.  
14.     // rest of the method...
15.  

This forces the dependencies to load immediately and appears to work around the Unity bug.

 

Dissonance 6.4.1 Is Now Available On The Asset Store

This is a critical update! Everyone who is using Dissonance 6.4.0 should update now.

The main change in Dissonance 6.4.1 is to fix the critical issue mentioned above. Unfortunately Unity have not approved the update to the HLAPI integration package yet, so if you're using that package you will need to apply the workaround mentioned in the previous post.

We've also made some other changes to help with profiling Dissonance performance - there's now a lot of extra detail in the profiler window showing exactly where Dissonance is spending it's time. This helped us reduce the startup time of Dissonance by about 1ms by moving some of the work to another thread. We also spread the work across two frames so all startup times should be approximately halved.

 

Hello there, well im having problems for run this plugin , its for my boss that want me to install it on his proyect , but i tested and cant see that works coz i cant listen anything , i try with global chat example simple and dosnt work , i saw that the basic microphone behaviour dosnt set the microphone as default itself but i saw it shows the channel 2 of my focusrite card ... anyways in the another side have a macbookpro retina with bootcamp and it have mic and works and i cant ear on any of the players ... so im going nuts here, we use photon networking example and im here still fighting with this to check if the global chat room thing work, we have like 10 rooms and into each one we have the dissonancesetup object with the stuff , and the trigger and reciever all as default global... any guide ? im searching but cant understand what im doing wrong...

 

Hi Wothanar

Have you tried simply running Dissonance in an empty project with the Photon demo scene (included in the free photon+Dissonance package)? That should be very simple to setup and will help me debug your problem

 

yea, and it works but it only send mic device audio from only one client , and the another one cant speech just listen, why this? also it only appear the green circle over one player why this? and why it dosnt work on my own game i put the stuff it should run the global voice chat stuff...

 

I assume both clients are using the same build? Could you look at the inspector (for the client which cannot speak) for the BasicMicrophoneCapture component (next to DissonanceComms) when the game is running - does it show an audio level indicator wobbling about (indicating that it is recording audio) when you speak?

Which green circle are you referring to? The demo has a blue ball which pulses when someone speaks, so if you mean that then it won't appear over one player due to the speech issue you mentioned above.

Does it not work at all in your own game, or is it the same issue with one client unable to speak?

 

the proyect its very big , and its hard to me test in the same proyect since its seems the problem can be the microphone that i have in the desktop and the echoes it produce here, one that its plug in via focus rite external driver, the mic works fine but i mean your demo dosnt set it at default, it reconize it since it display the dropbox and it says the line 2 of the focus rite where the mic its but, i set it via editor there and i didnt seems to be working so , people with this mic connection system probably will experience this kind of bug? ,what about made a simple global demo for test with out players moving just photon for clients can speak when joins a room globally in a room like a voice chat kind stuff? it can be done? thank for your support and sorry about trouble you

 

And forget to mention that , in game didnt worked at all , and it took long time me to build proyect on the one i need to put it and well i will give another try but i think i will need a new cheap pc mic for this tests...

 

That's why I suggested testing out in a new project which just has Dissonance and Photon imported - we can fix some of your problems more easily in a simpler project using the demo scene which I have built.

Dissonance doesn't choose a microphone - you can either specify one (with the dropdown in the editor) or if you don't specify a one it uses whatever the system default is (it's handled by Microphone.Start). If you're on windows check Sounds -> Recording and see which device is set as the default (it will have a green tick like in the attached screenshot)

That should be very easy to set up (there's a quick start guide covering most of this here):

1. Create new project
2. Import from the store: Dissonance, Dissonance For Photon, Photon Unity Networking
3. Set up your photon API keys/regions etc
4. Create a new scene called chat_scene
5. Drop in the Assets/Dissonance/Integrations/PhotonUnityNetworking/DissonanceSetup prefab
6. Create a VoiceBroadcastTrigger (to send voice) set it up with:
   • Chat Room: Global
   • Activation Mode: Voice Activation
7. Create VoiceReceiptTrigger (to receive voice) set it up with:
   • Chat Room: Global

Now you just need to create/join a photon network session and launch this scene. It's voice activated so anything you say should be transmitted.

 

https://gyazo.com/59109c0f1c34e931125e88c7f13decf3 this is the green circle you made in your demo that what im talking about, and i have a new default mic headphones 2 pcs, we are testing this in a new empty proyect the one you send us as demo. again the same stuff

https://gyazo.com/00a81c9250966e7c215f56781cd79be4

Only one character its talking only one blue bubble and i cant see this work on my side my microphone dosnt work on the another client he cant listen to me but i cant listem to her and i have all right i cant see the mic working on the pc its direct connected trought mother board so please check this and tell me whats wrong here?

 

THIS IS EXACTLY WHAT IVE DONE IN OUR PROYECT FOR TEST IN OUR GAME WHERE WE NEED IT....
But dosnt seems to work at all... no red errors... how can i debug this or fix it please help !

 

Ah I see, that appears around every player except yourself. It indicates the area around that player which will activate their proximity chat (i.e. if you walk into it, you can hear them).

Again the blue bubble indicates if you are hearing speech from that person - you'll never see it over your own character (because you never hear your own speech).

When the ball is pulsing (as shown in your gif) did you hear voice then?

It seems like the main problem here is a microphone issue. Could you try the code shown here in a blank scene - this should just play your microphone through your speakers. If that does not work the issue is at a level below Dissonance.

I'm trying to help but I don't have much to go on at the moment!

 

after some tweaks on both operative systems seems to be working the last test.... i will do the game test now and test if it works there... since its all right with your demo but we need this working on our proyect and out of the box

 

it worked on game , thank for your help have fun man!.

 

Fantastic, glad I could help

 

Requesting Android permissions microphone, you can now use Permisson instead of start/stop recording:

Code (CSharp):

1.  
2. if (!Permission.HasUserAuthorizedPermission(Permission.Microphone))
3. {
4.     Permission.RequestUserPermission(Permission.Microphone);
5. }
6.  

 

Thanks for the heads up! I'll add a note to the Dissonance documentation about that right now and I'll see about somehow integrating it into Dissonance for the next release.

 

Hi rosssssss, You need to add an input axis with the correct name (GlobalChat) and set it to whatever key you want to be the Push-To-Talk key. Here are the settings I use:

 

Ah thanks - had just seen the warning info in Broadcast-Trigger and deleted my post... but that's useful anyway

 

Hello I'm trying to follow this https://dissonance.readthedocs.io/en/latest/Tutorials/Position-Tracking/index.html - for positional audio and I'm using Forge but I don't have a ForgePlayer.cs in my import as it seems to suggest I should. The demo scene appears to do this a different way, setting up and RPC which transmits the playerId. Ummm should I have that file or are those docs out of date? Thanks!
(p.s. what i'm actually trying to do is identify which player is speaking so that i can do lip sync (not salsa for now.) )

 

Sorry those docs are out of date, I'll go and update them now.

We no longer support the old Forge Networking. For Forge Networking Remastered there is no built in ForgePlayer, you need to implement it yourself - this is because you'll probably need to modify the contract on your player gameObject to synchronise the player tracking info and obviously we can't ship a prebuilt version of that.

Check out the docs on Custom Position Tracking for help on how to do that. it's pretty simple - you just need to make sure that the Dissonance player ID is sychronised across all players in the session (so Dissonance knows which object represents which speaker).

 

Hello again! I've managed to cause another problem here. I've been trying to get lip syncing working - not using Salsa. The way i attempted to go about it is: When the SetPlayerId RPC comes through on a network player, I then look for that player under the DissonanceComms object. Upon finding the player gameobject (with the VoicePlayback and AudioSource components on it) I added a component to it which implemented OnAudioFilterRead(...) and then sent that info on to my lip-sync code. This worked perfectly for lip-syncing but muted all the audio. I can still see the audio coming through (on VoicePlayback and SamplePlayback) but don't hear anythiing and there are loads of warning messages.
I guess there's a better way to do this... Any clues? I'm using the Oculus lip sync library and it seems to need that OnAudioFilterRead info from the AudioSource... Thanks

 

That sounds like a reasonable approach to lip syncing. What are the warnings you're receiving?

 

I think the problem might have been related to the fact that when you have multiple OnAudioFilterRead on the same AudioSource the second one gets the result of whatever processing the first one has done on the data... i changed it to use a copy of the data for the lip-reading and it started working

I think the warnings were along these lines...
[Dissonance:Recording] (14:24:47.246) BasicMicrophoneCapture: Insufficient buffer space, requested 33957, clamped to 16383 (dropping samples)

[Dissonance:Recording] (14:24:47.248) BasePreprocessingPipeline: Lost 255 samples in the preprocessor (buffer full), injecting silence to compensate

 

Ah yes that's definitely something you need to be careful with.

This means that the microphone hardware delivered xxx samples within a single frame which is more than the buffer size (yyy). It depends on exactly what the sample rate is (which depends on your mic hardware), but if we assume 48kHz that's 700ms of audio delivered in one frame!

This is another artifact of the problem above. A large amount of audio is delivered all at once, overflowing the preprocessor input buffer so some audio is discarded to make space.

Both of these problems are related to bad frame rates which cause a lot of audio to be delivered all at once (well beyond the bounds of sensible real-time audio). If this only happens once or twice it's not a problem (often there can be big frame rate stutters when you're loading the scene).

Have these warnings been resolved?

 

Hello again. Thanks yes, i think actually that the lip syncing stuff is now all good! It's working very well when the audio is working.
I have one other issue that has cropped up in the meantime though. It seems like the audio doesn't work at all just with one particular machine. When I host and connect to his machine there's silence in both directions (though the rest of the networking works fine.) (We're running windows exes) but if I connect to other machines it all works.
He attempted to install the redist but the system said that he already had it on there.
When he runs the exe he gets a load of warnings from Dissonance - at first they say "Decoder Pipeline: Failed to write an encoded audio packet into the input transfer buffer" and then after a while they say "Encoded Audio Error: Voice Error: Encoded Audio heap is getting very large (140 items.)"... any clues? Is it that maybe he has the wrong version of the redist? thanks.