Official Unity Render Streaming Introduction & FAQ

Thanks for the feedback, looking forward into that version 2.3

 

I get a similar error only when using Safari (14 on macOS 10.15.7) while FF and Chrome are working fine. Mine occurs when setting the local answer:

Failed to set local answer sdp: Failed to set local video description recv parameters for m-section with mid='1'.,​

 

Hello, I want to apply Renderstreaming to my own game project, But I have some problems.
My game project has multiple game scenes, each with a separate camera, This means I have to change the video track every time I switch scenes.
How do I implement this?

Thanks in advance.

 

VideoStreamTrack can now be created from RenderTexture, so if the scene changes, I think it can be done by rendering the camera image of that scene to the RenderTexture that generated VideoStreamTrack.

 

Hi, I am looking for an advice.

Since the websocket-sharp library used by RenderStreaming does not support adding headers(which I need for cookie authentication) and also has some protocol issues with TLS 1.3

I opted for this Socket library (https://github.com/endel/NativeWebSocket)

So far I have been able to establish a connection to our websocket server and also exchange STOMP messages. I wanted to know that how long of an effort is it to replace the websocket-sharp library in the RenderStreaming project with the one mentioned above(NativeWebSocket)? And is it even possible to do so? I have looked at the source code but since I don't have the render-streaming's domain knowledge I am not sure if diving into the code and replacing the socket library would end up as a success

 

I noticed that when creating a new VideoStreamTrack with the constructor that takes a label and a RenderTexture, it only works when the RenderTexture is a targetTexture for some camera.

For example, if I have rendertex1 which is a target texture for some camera, and I also create rendertex2 and call
Graphics.Blit(rendertex1, rendertex2) every frame, rendertex2 gets properly updated in the editor, but when i try to create a new VideoStreamTrack("test", rendertex2), the stream connects, but is black. I noticed that it worked with just regular texture assets, but these can't be Blitted into.

Is there a plan to support this?

 

Hi All

I am trying the web Streaming. I am using Streaming Template ,webrtc 2.0 version. When i run the project it shows the error .

If i run webrtc.exe it shows the ipaddress.... But when the unity project it shows the error.

How can i solve this error.

 

Hello. I have an issue with UI. In the editor, the UI works fine both in the game, and on the stream. However, in a build the UI doesn't show in any of the places. It doesn't seem to get rendered to the render texture even. I am using LegacyRPRenderTextureBlitter

 

Thank you for your answer, I have successfully solved my problem.

 

Hello developers,
I would like to read input from a remote XRController (eg. Oculus motion controller), as it is made for the gamepad in SimpleCameraController.cs and here.
Can anyone give me some ints?

 

Hello, I would like to know what audio encoding methods are supported by RenderStreaming and whether Settings of audio encoding parameters (such as sampling rate) are supported?

 

Hello,
I try to use the RTCRtpSender Get/SetParameters logic to set:

• RTCRtpEncodingParameters - Frame Rate
• RTCRtpEncodingParameters - Min Bit Rate
• RTCRtpEncodingParameters - Max Bit Rate

But I find that it sometimes fails, and the frame rate I see through Chrome :// webrtC-Internal is not the same as what I set.
Do you know what caused this?

 

We are still using the older SDP munging technique in our code. We have the configuration to turn it on and off so it gives us better control. I know it was removed in the newer versions of the RenderStreaming package but, it is nice to have options.

Code (CSharp):

1. var startKbits = Convert.ToInt32(Configuration.Instance.WebRTCStartBitRateMegabit.Value * 1024f);
2. var maxKbits = Convert.ToInt32(Configuration.Instance.WebRTCMaxBitRateMegabit.Value * 1024f);
3.  
4. var pattern = @"(a=fmtp:\d+ .*level-asymmetry-allowed=.*)\r\n";
5. var newSDP = Regex.Replace(originalSDP, pattern, "$1;x-google-start-bitrate=" + startKbits + ";x-google-max-bitrate=" + maxKbits + "\r\n");

 

Looks like calling AddICECandidate is crashing unity? Maybe my ice candidate is no good but a crash seems excessive, it dies right at this line.

 

Thanks for your reply.

Hello, I would like to know whether you have tried to use SDP to control the audio coding parameters. I tried to use SDP to modify the opUS sampling rate, code rate and sound channel number, but it was invalid.

Is it the renderStreaming itself that is not supported or my problem

 

Hello,

I have a unity app for IOS and Android, and I was wondering if this render streamer works on those two platforms?

Thanks

 

For DataChannel configuration, when making RTCDataChannelInit config, and passing it in by reference I get an error that says

ArgumentException: RTCDataChannelInit object is incorrect.

Code (CSharp):

1. var conf = new RTCDataChannelInit(false)
2.         {
3.             id = 231,
4.             maxRetransmits = 1,
5.             maxRetransmitTime = 1,
6.             negotiated = false,
7.             ordered = false,
8.             protocol = ""
9.         };
10. _defaultDataChannel = _peerConnection.CreateDataChannel("data", ref conf);
11.  

using the default config works:

var conf = new RTCDataChannelInit(true)

But also this is slightly outdated I think. reliable looks like it was depricated and names are a bit off assuming this reference is correct: https://developer.mozilla.org/en-US/docs/Web/API/RTCDataChannel

 

It's December 2020, and all links of the first post has broken
I just made this 5 minutes basic demo guide of Unity Render Streaming with URP, super simple:

 

Are there any estimates on when we can expect this package to support Unity 2020?

 

The package does support Unity 2020, if the issue is you are not seeing the package, take a look at the documentation and it will show the handling in the package manager is a bit different from 2019.4 https://docs.unity3d.com/Packages/com.unity.webrtc@2.2/manual/install.html

 

Hi,

I am currently testing Unity Render Streaming and I am wondering if we can receive a stream inside Unity.

Right now, you can stream from Unity to a web Page. But is it possible to stream from Unity to Unity?

I would think it's possible, Would it be possible to define the videoSource to be a Unity Render Stream ? (https://docs.unity3d.com/Manual/Video.html).

thanks,

Jean

 

Yes, you can try it.
https://github.com/Unity-Technologi...p/Samples~/VideoReceive/VideoReceiveSample.cs

 

Hi,

Cool, I'll try that, thanks

Bye,

Jean

 

Hello

I'm experimenting with getting a sdp from another peer.

Assume you're given an offer sdp:

Code (csharp):

1. v=0\r\n
2. o=- 535940652912167540 2 IN IP4 127.0.0.1\r\n
3. s=-\r\n
4. t=0 0\r\n
5. a=group:BUNDLE 0\r\n
6. a=msid-semantic: WMS\r\n
7. m=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\n
8. c=IN IP4 0.0.0.0\r\n
9. a=ice-ufrag:scMp\r\n
10. a=ice-pwd:IqZuYs3gcfcEqnazoQQzeP/x\r\n
11. a=ice-options:trickle\r\n
12. a=fingerprint:sha-256 C4:4C:CE:C2:1E:F6:99:BB:7C:24:6D:31:8A:51:31:EB:A5:14:FD:0E:4F:2C:A7:6C:91:3E:E1:48:0E:97:E1:BA\r\n
13. a=setup:actpass\r\n
14. a=mid:0\r\n
15. a=sctp-port:5000\r\n
16. a=max-message-size:262144\r\n

How do you set this as a remote description? I've been trying this.

Code (csharp):

1. // get sdp from above
2. string sdp = "v=0\r\"
3.  
4. RTCSessionDescription offer = new RTCSessionDescription{};
5. offer.sdp = sdp
6.  
7. var op2 = pc2.SetRemoteDescription(ref offer);
8. yield return op2
9.  

But I'm getting this error: RTCErrorException: Failed to create fingerprint from the digest.

Am I on the right track? I'm basically having trouble setting a remote description from a sdp string.

 

Answered my question. The string has to be like:

Code (csharp):

1. string sdp = "v=0\r\no=- 535940652912167540 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:scMp\r\na=ice-pwd:IqZuYs3gcfcEqnazoQQzeP/x\r\na=ice-options:trickle\r\na=fingerprint:sha-256 C4:4C:CE:C2:1E:F6:99:BB:7C:24:6D:31:8A:51:31:EB:A5:14:FD:0E:4F:2C:A7:6C:91:3E:E1:48:0E:97:E1:BA\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n";

 

Is there a way to send phone's gyro data to unity to control the camera?

 

when i create ice from websocket like this
var initInfo = new RTCIceCandidateInit();
initInfo.candidate = ice;
initInfo.sdpMid = sdpMid;
initInfo.sdpMLineIndex = sdpMlineIndex;
RTCIceCandidate candidate = null;
try
{
candidate = new RTCIceCandidate(initInfo);
}
catch (Exception ex)
{
Debug.LogError($"{ex}");
}
i will get a error like this,
System.ArgumentException: Value does not fall within the expected range.
at Unity.WebRTC.RTCIceCandidate..ctor (Unity.WebRTC.RTCIceCandidateInit candidateInfo) [0x00031] in D:\Project\Rtc\Library\PackageCache\com.unity.webrtc@2.3.0-preview\Runtime\Scripts\RTCIceCandidate.cs:251
at STRtcConnection.OnRemoteIce (System.String remoteId, System.String label, System.String ice) [0x00076] in D:\Project\Rtc\Assets\STRtc\STRtcImpl\STBase\STRtcConnection.cs:285
UnityEngine.Debug:LogError(Object)

from the error log ...can't get any idea ,help..

 

the old version 2.2.1 is ok,but don't know why

 

@kazuki_unity729 Why didn't you answer to this guy? If you can, I would be curious to know it too

 

Unreal's PixelStreaming is also not production-ready by the way. They are still in 'beta' themselves with their own set of issues.

 

i see there is app2.js in the project, which seems like demostrating how html side sending video to Unity. I would like to ask how should i set it up? i have uncomment the app2 section of index.html. the html ui is working, and it can enable webcam video when i access localhost. but i can't achieve anything farther. the best i can do is triggering an error in server side

Code (CSharp):

1. TypeError: Cannot read property '0' of undefined
2.     at ........\WebApp\build\signaling.js:132:30

 

I cannot access my server on the public network now. How can I expose my server to the public network

 

@licoder @kazuki_unity729
I have same issue.
I used only Unity WebRTC package.(not use Unity Render streaming package)

my params
candidate: "Cand[:xxxxxxxxx:x:udp:xxxxxxxxxx:xxx.xxx.xxx.xxx:xxxxx:local::x:yyyyy::x:xx:x]"
sdpMid: "0"
sdpMLineIndex:0

The format of ICE Candidate is
"candidate: 4234997325 1 udp 2043278322 192.168.0.56 6502 typ srflx raddr 192.168.2.77 rport 32768 generation 0"
I imagined something like that, but it was completely different.

 

Kazuki,
I see you have posted you are working on the merge in for the fix of [BUG] Visual artifacts when using hardware encoding #205
It looks like this fix won't be in until version 2.5 (April) according to the road map. Is there a branch with those changes available? We have been looking into addressing this bug ourselves and would like to see what solution you came up with and even potentially merge it early into our own code.

Thank you

 

Hello,

I am using the Unity WebRTC library (not RenderStreaming) on iOS, and was experiencing some issues when signaling. Everything works fine in the editor and desktop platforms, but when run on an iOS device, the ICE connection state never changes to Connected/Completed. I have verified that the ICE candidates are successfully generated and sent to the remote peer, and that remote candidates are properly received by the local peer.
Do you have any ideas on what this could be? Once again, this only happens on iOS, and I am trying to receive video and create a data channel.

Thanks!

 

Is there a method that takes takes a string and converts it to an RTCIceCandidate / RTCSessionDescription or do we have to make our own? (Same goes for the other way- turn the class into a string)

 

@kazuki_unity729

My problem is that the code I wrote myself (Simple.cs attached to my last post)
The value of the candidate property of iceCandidate passed by OnIceCandidate is
"Cand [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]"

When I tried the Unity.RenderStreaming sample, the value of the candidate property of iceCandidate passed by OnIceCandidate is
"candidate: xxxxxxxxxxxxxxxxxxx typ host xxxxxxxxxxxxxxxxx"
It is passed in a well-known format.

I still don't know why this difference occurs.

 

Thank you so much, is there something that will convert it to a string so I can send it over the signaling server or do I have to write my own ToString method?

 

Thanks to you guys for the quick fix!
I have updated the WebRTC package to v2.3.1 in my project at hand and confirmed that the problem is fixed.