Question Unity WebRTC Add MediaStream/Track at runtime

Hi all,
We are developing a solution that provides a stream based on a Camera in Unity. The client peer should be able to request custom resolutions in order to utilize the available screen estate of the client, and the server should respond by replacing the existing track with a new track with the new resolution.

The client pass a request of a new resolution to Unity WebRTC, either via the Signaling Server OR via a WebRTC Data Channel. Unity WebRTC picks up this message, and creates a new track with new resolution, to replace the old track. We've got two problems that relates to this:

As expected, calling RTCPeerConnection.AddTrack(...) triggers OnNegotiationNeeded. When OnNegotiationNeeded is triggered, CreateOffer() is used to create an offer with the new track, and it is passed further along to SetLocalDescription(...). The SetLocalDescription method never returns (neither IsDone nor IsError), but at the same time, just as the call is being made, OnIceConnectionChange is triggered with RTCIceConnectionState.Disconnected and thus interrupting the flow. (The expected next step after SetLocalDescription would be to pass the offer along to the client using the Singaling Server)

Are there any known limitations to adding additional tracks? I see no apparent reason why OnIceConnectionChange is triggered at this stage.

Second, if we do a RTCPeerConnection.RemoveTrack(), Unitycrashes when running from inside the dev environment.

Any help or suggestions that might relate to these problems are appreciated!

Regards,
Anders

 

Hi, sorry for the late reply, been some busy days wrapping up the year.

I've recreated the issue based on LegacyRP sample running of version 2.2.1-preview, running on Unity 2019.4.16f1
It seems that I no longer get the crash (I was running on an earlier build of Unity when I had that one),
but the other error is still persistent.

Please find attached the following modified files:
* SimpleCameraController.cs
- Updated FixedUpdate() - A simple hack, capturing client typing 1 (AddTrack) or 2 (RemoveTrack)
- Two methods, SampleRemoveTrack() and SampleAddTrack() - Triggering RenderStreaming methods
* RenderStreaming.cs
- New methods, TriggerRemoveTrack() and TriggerAddTrack()
- Modified OnOffer - Only adding two cameras initially, added OnNegotiationNeeded

In addition to the files, the scene must be modified:
* Create a camera to use for the replacement track - Duplicate Render Streaming Camera

I also found another issue: When creating an offer SDP, the name of the camera is used. If the camera name contains one or more spaces, this will cause the SDP to be invalid. I.e. The RenderStreamingCameraX Objects must be named without space in the name.

How to reproduce:
1. Start the Unity sample webserver
2. Play the scene from the Unity Editor
3. Launch a browser, navigate to the webserver, Connect to the Unity stream
4. Click '1' in the browser window
- Observe that OnNegotiationNeeded is triggered,
- Once the code reach SetLocalDescription, OnIceConnectionChange is triggered and the connection is Disconnected

I see that the sample lacks handling of SendOffer (HttpSignaling.cs) - But the code currently fails before/independent of this.

Please let me know if you need any further input from me!

Best regards,
Anders

 

I've tried upgrading to 2.2.2-preview version of the package, but unfortunately I am still getting the same issue with the connection getting disconnected upon adding a new track

Regards,
Anders