Problem restarting audio stream

URS 3.1.0-exp.3
Hi,

I'm modifying the URS Example scene WebBrowserInput for my needs - one difference though is that I'm using SingleConnection instead of Broadcast. I've got things working well overall. But am having a problem with audio only working during the first connection between Sender and Receiver.

I'm setting things up so the user can use the Receiver (web app) to disconnect, either deliberately via a UI button or by refreshing the browser, and the Sender (unity app) will detect it, delete the SingleConnection connection, and then initiate it again so that the Receiver can reconnect.

On the Receiver side, the disconnection is handled via:

Code (CSharp):

1. //Disconnect peer connection and signaling
2.   async hangUp(connectionId) {
3.     if (this.signaling == null) {
4.       return;
5.     }
6.  
7.     this.pc.close();
8.     this.pc = null;
9.     Logger.log(`delete connection ${connectionId}`);
10.     await this.signaling.deleteConnection(connectionId);
11.     await this.stop();
12.   }
13.  
14.   async stop() {
15.     if (this.signaling) {
16.       await this.signaling.stop();
17.       this.signaling = null;
18.     }
19.   }

On the Sender side, the disconnection is done via SingleConnectioneleteConnection() (either via UI event, or a data channel close event is received). I'm letting the RenderSteaming component keep running (I tried stop/starting that too but that borked everything).

I'm using AudioStreamSender (in case there's a different way I don't know about), and it works when I first run the Sender (from Unity Editor) and connect the Receiver. But if I disconnect, either from Sender or Receiver, and then reconnect w/out stopping and starting the unity editor, the audio doesn't work (the video and data channel streams do work).

I've looked at what I can in the code and from what I can see it looks like things are still connected even when the audio isn't playing in the Receiver:

On Sender side:
- audioStreamSender.Track.ReadyState is 'Live'
- audioStreamSender.Track.Enabled is 'true'
- the AudioStreamSender componenent shows the sound meter UI in the Inspector, and it moves up/down like it does when the audio does play properly

On Receiver side (in video-player.js):
- during connection setup, the Peer.trackevent event is received and a track of type 'audio' is added.
- after 'audio' track is added and I interrupt the code, it has property values 'enabled: true', 'muted: false', and 'readyState: live'

Does anyone have any suggestions as to how to keep debugging this??

 

Thanks for the reply. It is only audio. The video and data channels work properly when I reconnect in the way I describe above, but the audio does not. The audio only works the first time I connect after restarting my app in unity editor.

Also is there documentation somewhere that describes the nature of the various connections in more detail than the overviews that are already given?

 

my demo can not send audio to web, may web not play audio. do you have this problem.

 

Thanks for this! I'm just getting back to this project now. I'm looking at the fix you've linked here - is this the only relevant part of the PR for this issue I described with the audio? This fix changes only the ReceiverSample.cs file. My receiver though is derived from the video-player.js file in WebApp videoplayer code: https://github.com/Unity-Technologi.../client/public/videoplayer/js/video-player.js

The code that sets up the audio stream does it in reoponse to a callback from the RTCPeerConnection instance that's contained within class Peer:

Code (CSharp):

1.    
2. this.pc = new Peer(this.connectionId, true);
3.     this.pc.addEventListener('disconnect', () => {
4.       Logger.log('video-player pc disconnect handler');
5.       _this.mainProcessPeerDisconnect();
6.     });
7.     this.pc.addEventListener('trackevent', (e) => {
8.       const data = e.detail;
9.       Logger.log('* * * video-player trackevent received: ' + e, true);
10.       if (data.track.kind == 'video') {
11.         _this.videoTrackList.push(data.track);
12.       }
13.       if (data.track.kind == 'audio') {
14.         Logger.log("--- setupConnection: trackevent adding type 'audio'", true);
15.         _this.localStream.addTrack(data.track);
16.         let audioTracks = _this.localStream.getAudioTracks();
17.         Logger.log("--- setupConnection: trackevent adding type 'audio'. audioTracks length: " + audioTracks.length);
18.       }
19.       // NOTE this gets called cuz maxVideoTrackLength == 1
20.       if (_this.videoTrackList.length == _this.maxVideoTrackLength) {
21.         _this.switchVideo(_this.videoTrackIndex);
22.       }
23.     });
24.  
25. <snip>

Do you have any suggestions regarding this and getting the audio to reconnect properly?
Thanks

 

I'm modding the 'videoplayer' sub-project from the URS WebApp code: https://github.com/Unity-Technologi...tree/develop/WebApp/client/public/videoplayer

But I realize now I haven't tested it with the original code, only with my modded code. So if it can't be debugged from the original WebApp/videoplayer code, it'll be an issue with my changes. If you don't have someone who can test the original WebApp/videoplayer code, I can do that but not until next week at the earliest.

Thanks again