Official Unity Render Streaming Introduction & FAQ

Hi,

I have a problem with determining the correct -force-device-index parameter when starting streaming instances in -batchmode.

Whenever I disconnect my remote desktop session from our render server, the indexes get mixed up. While one of the NVidia cards has index 0 during the remote desktop connection, 0 points to the weak on-board graphics adapter after disconnecting.

My scenario:
I have integrated WebRTC in our app to run several instances in parallel on a Windows render server with multiple NVIDIA graphics cards. When a user connects via browser, an instance is launched exclusively and starts streaming its content to the user. After the browser session is finished, the instance will be terminated. The life cycle of instances is managed by a watchdog component connected to an ASP .NET based web portal with signaling server functionality.
This watchdog needs to assure some load balancing to use all of the GPUs which is done by starting with different -force-device-index parameters.

Details:
I tried to use a query to ManagementObjectSearcher("select * from Win32_VideoController") to get all adapters and filter usable ones. I see them all (except for the "Microsoft Basic Render Driver") and their order seems to be stable:
Scenario 1 with RDP: 3 GPUs reported by system:
- Ignoring GPU: Microsoft Remote Display Adapter, memory: 0 MB
+[0] Found suitable GPU: NVIDIA GeForce GTX 1650, memory: 4095 MB
-[1] Ignoring GPU: Intel(R) HD Graphics P530, memory: 1024 MB
Scenario 2 without RDP: 2 GPUs reported by system:
+[0] Found suitable GPU: NVIDIA GeForce GTX 1650, memory: 4095 MB
-[1] Ignoring GPU: Intel(R) HD Graphics P530, memory: 1024 MB

According to this launching with index [0] should always force using the NVIDIA card. But in scenario 2 SystemInfo.graphicsDeviceName reports the Intel card and NVIDIA shows up at index 1.

I did some tests with a C++ based tool enumerating with IDXGIFactory1. It appears to be the way what the Unity player performs. But having a C++ DLL tightly coupled to Windows and DirectX as graphics API does not sound promising in terms of platform independency.

What can I do to correctly determine the index in advance as the Unity player expects it?

Thanks

 

We are on 2020.3.14 f1

We tested on Windows 10 Pro 20H2 on 1 machine and 2 PCs running 21H1. In the meanwhile it turned out that on the third machine having 1 NVIDIA Quadro RTX 5000 even the correlation between the output of DirectX adapter list and the one reported by our app did not fit together.

 

Yes of course you are totally right and even a Unity API would not help as users need to know this before launching.

As posted in my update #754 even the C++ DirectX way is not suitable. I created a small test project and ran it on 3 different machines combined with a preceding output of DXTest (the ones with "===") I got:

Spoiler: PC 1 (OK)

=== (0) NVIDIA Quadro RTX 5000, 16170 MB ===
=== (1) NVIDIA Quadro RTX 5000, 16170 MB ===
=== (2) Microsoft Basic Render Driver, 0 MB ===
------- 2021-08-18 12:38:34Z
(0) NVIDIA Quadro RTX 5000), 16170 MB
Args: -force-device-index 0 -batchmode
------- 2021-08-18 12:38:36Z
(1) NVIDIA Quadro RTX 5000), 16170 MB
Args: -force-device-index 1 -batchmode
------- 2021-08-18 12:38:38Z
(2) Microsoft Basic Render Driver), 32738 MB
Args: -force-device-index 2 -batchmode

Spoiler: PC 2 (Not matching)

=== (0) Microsoft Basic Render Driver, 0 MB ===
=== (1) NVIDIA Quadro RTX 5000, 16170 MB ===
=== (2) Microsoft Basic Render Driver, 0 MB ===
------- 2021-08-18 12:20:32Z
(0) NVIDIA Quadro RTX 5000), 16170 MB
Args: -force-device-index 0 -batchmode
------- 2021-08-18 12:20:33Z
(1) Microsoft Basic Render Driver), 16213 MB
Args: -force-device-index 1 -batchmode
------- 2021-08-18 12:20:35Z
(2) Microsoft Basic Render Driver), 16213 MB
Args: -force-device-index 2 -batchmode

Spoiler: PC 3 (OK)

=== (0) Intel(R) HD Graphics P530, 128 MB ===
=== (1) NVIDIA GeForce GTX 1650, 3962 MB ===
=== (2) Microsoft Basic Render Driver, 0 MB ===
------- 2021-08-18 12:12:57Z
(0) Intel(R) HD Graphics P530), 16329 MB
Args: -force-device-index 0 -batchmode
------- 2021-08-18 12:12:58Z
(1) NVIDIA GeForce GTX 1650), 3962 MB
Args: -force-device-index 1 -batchmode
------- 2021-08-18 12:13:01Z
(2) Microsoft Basic Render Driver), 16329 MB
Args: -force-device-index 2 -batchmode
------- 2021-08-18 12:13:03Z

Spoiler: PC 3 Remote Desktop (OK)

=== (0) NVIDIA GeForce GTX 1650, 3962 MB ===
=== (1) Intel(R) HD Graphics P530, 128 MB ===
=== (2) NVIDIA GeForce GTX 1650, 3962 MB ===
=== (3) Microsoft Basic Render Driver, 0 MB ===
------- 2021-08-18 12:12:20Z
(0) NVIDIA GeForce GTX 1650), 3962 MB
Args: -force-device-index 0 -batchmode
------- 2021-08-18 12:12:20Z
(1) Intel(R) HD Graphics P530), 16329 MB
Args: -force-device-index 1 -batchmode
------- 2021-08-18 12:12:21Z
(2) NVIDIA GeForce GTX 1650), 3962 MB
Args: -force-device-index 2 -batchmode
------- 2021-08-18 12:12:22Z
(3) Microsoft Basic Render Driver), 16329 MB
Args: -force-device-index 3 -batchmode

In most cases the results are equal. But PC 2 indicates that Unity might do some kind of sorting in special situations.
Even performing a test loop is not totally reliable as the situation changes whenever a remote desktop session starts or terminates and the first adapter is removed.
Thus it will be tricky when comes to load balancing on render streaming servers with multiple adapters.

It would be helpful to know how Unity interprets and maps the force-device-index parameter. Is there any guideline or specification?

 

It looks like Unity runs IDXGIFactory6::EnumAdapterByGpuPreference for determining the indices.

Can you confirm this? If so it would be easy to provide the correct index.

 

Thanks for confirming this. I was still confused about this at the beginning because this is the case where my PC 2 did not fit.

But adding that NvOptimusEnablement mentioned in Let's talk about the mysterious -gpu command line argument seems to make it. Now my DX-Test tool reports the adapter list as Unity does.

 

@kazuki_unity729 can we expect support for WebRTC 2.4.0-exp.4 soon?

 

I do have a log from Player.Log file and attaching it to this reply.

Thank you very much for your help!

 

Sorry, I should have been more clear, I meant so that it works with RenderStreaming, as currently, this results in compile errors.

 

Hi, everyone!

I added the Unity Render Streaming package to the existing Unity project and deployed an AWS EC2 web server and a TURN server using coturn. If you open a web browser on a PC with the Unity client, screen streaming is possible normally. However, streaming is not possible from a remote PC that exists elsewhere. Are there any problems with the code?

The TURN server created on the server was tested normally.

P.s. Fortunately, once you've been lucky enough to connect again, the connection won't happen again.

 

Hi, kazuki.

Sorry for the late reply. I couldn't find a clear way, so I solved it via Socket.io.

Thank you.

 

I haven't found a way, does anyone know how to dynamically resize a videostreaming in runtime state? Thanks for your reply.

 

Thanks your reply.

I tried changing 'X' 'Y'(StreamingSize) at runtime , and triggered the 'OnDisable' and 'OnEnable' events in the 'CameraStreamer' script, but I had to refresh the page to display the screen.

Code (CSharp):

1.  
2.         void OnEnable()
3.         {
4.             // todo(kazuki): remove bitrate parameter because it is not supported
5.             m_track = m_camera.CaptureStreamTrack(x, y, 1000000);
6.             RenderStreaming.Instance?.AddVideoStreamTrack(m_track);
7.         }
8.  
9.         void OnDisable()
10.         {
11.             RenderStreaming.Instance?.RemoveVideoStreamTrack(m_track);
12.         }

How can I change the resolution of a video without refreshing the page?Or any other way to do that now

 

I had a question while applying the package.

How does live streaming work if I build an application that includes that package and use it in multiple places?

If all clients are matched for each application, can you prevent it by adding a code that limits the number of internally connected people? (e.g., only allow up to 2 people to access one application)

Best regards.

 

Hello, I would like to know if streaming a RenderTexture is supported ? If so, how do we send the texture from the renderTexture continously ?
I've been trying to replace the default camera track with

Code (CSharp):

1. new VideoStreamTrack("Internal", renderTextureTest);

I pretty much replicated the code from the following issue https://github.com/Unity-Technologies/UnityRenderStreaming/issues/360 but without success.

While my renderTexture in itself renders what I need, the received texture on the client is completely black (it still receives a texture, so it's not a failed track transfer). Does anybody ever worked on something similar ? There seems to be no such resources online about transferring renderTexture.

Thanks.

 

@unity_vC13nyYgPniOjQ

I finally was able to use this code for using screen instead of single camera

Code (CSharp):

1. using System;
2. using System.Collections;
3. using Unity.WebRTC;
4. using UnityEngine;
5. using UnityEngine.Experimental.Rendering;
6.  
7. namespace Unity.RenderStreaming
8. {
9.     public class ScreenStreamer : VideoStreamBase
10.     {
11.         public override Texture SendTexture => rt;
12.  
13.         [SerializeField]
14.         private RenderTexture rt;
15.         private Resolution resolution;
16.  
17.         private RenderTexture sc;
18.  
19.         void Start()
20.         {
21.             StartCoroutine(Render());
22.         }
23.  
24.         private IEnumerator Render()
25.         {
26.             while (true)
27.             {
28.                 yield return new WaitForEndOfFrame();
29.                 if (rt != null)
30.                 {
31.                     ScreenCapture.CaptureScreenshotIntoRenderTexture(rt);
32.                 }
33.             }
34.         }
35.  
36.         protected override MediaStreamTrack CreateTrack()
37.         {
38.             resolution = GetResolution();
39.             RenderTextureFormat format = WebRTC.WebRTC.GetSupportedRenderTextureFormat(SystemInfo.graphicsDeviceType);
40.             GraphicsFormat graphicsFormat = GraphicsFormatUtility.GetGraphicsFormat(format, RenderTextureReadWrite.Default);
41.             GraphicsFormat compatibleFormat = SystemInfo.GetCompatibleFormat(graphicsFormat, FormatUsage.Render);
42.             GraphicsFormat format1 = graphicsFormat == compatibleFormat ? graphicsFormat : compatibleFormat;
43.             {
44.                 rt = new RenderTexture(resolution.width, resolution.height, 0, format1);
45.                 rt.Create();
46.             }
47.             return new VideoStreamTrack(rt.GetNativeTexturePtr(), resolution.width, resolution.height, format1);
48.         }
49.  
50.         Resolution GetResolution()
51.         {
52.             Resolution r = new Resolution();
53. #if !UNITY_EDITOR
54.             System.Type T = System.Type.GetType("UnityEditor.GameView,UnityEditor");
55.             System.Reflection.MethodInfo GetSizeOfMainGameView =
56.             T.GetMethod("GetSizeOfMainGameView", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Static);
57.             Vector2 Res = (Vector2)GetSizeOfMainGameView.Invoke(null, null);
58.             r.width = (int)Res.x;
59.             r.height = (int)Res.y;
60. #else
61.             r.width = Screen.width;
62.             r.height = Screen.height;
63. #endif
64.             return r;
65.         }
66.     }
67. }
68.  

 

Hello. I have been following this tutorial but I am still not able to interact wwith UI Unity elements fro mthe browser.
There is sommething else needed to configure it?
In my Unity Editor side, I can see that the input debugger is receiving the correct mouse position from the browser and also the left and right click events, but those events can't push buttons or anything else.

Thanks

 

Thank you so much, it's working as I needed now

 

I understand what you mean.

Sorry, but I have one more question.

What exactly is the difference between "SingleConnection" and "Broadcast" components?

According to the manual description, does the "SingleConnection" component perform a stream to only one peer?

 

@kazuki_unity729
Thanks for the latest update! We've been looking forward to it.

A question from our team:
How would we go about using the receive audio viewer with the broadcast and simple sample receiver or similar classes? Thanks again for your hard work.

Edit (more details):
https://imgur.com/a/evEGehU

The first image is my changes to the script based off of broadcast.cs

The 2nd image shows the host crashing calling NativeMethods.ProcessAudio after a client connects

 

Hello everyone,

I have come a long way with my application using RenderStreaming and WebRTC.

However, I still have occasional crashes every now and then that are coming from the webrtc.dll

My build is for Windows and Android and I have found that the error is this:

Faulting application name: Arcade Framework (local).exe, version: 2020.3.11.51119, time stamp: 0x60acdd78
Faulting module name: webrtc.dll, version: 0.0.0.0, time stamp: 0x60c08f6a
Exception code: 0xc0000005
Fault offset: 0x000000000013a5b4
Faulting process id: 0x16ac
Faulting application start time: 0x01d7ae07d978b901
Faulting application path: C:\Users\Isen\Desktop\Arcade Demo 67 - Central Clear Streams - Copy\Arcade Framework (local).exe
Faulting module path: C:\Users\Isen\Desktop\Arcade Demo 67 - Central Clear Streams - Copy\ Arcade Framework (local)_Data\Plugins\x86_64\webrtc.dll
Report Id: d3d52651-1aea-42c8-b9bb-9d90147a7d03
Faulting package full name:
Faulting package-relative application ID:



It happens mostly on the Windows side that creates more streams compared to the Android side that
on every session (game round) creates only one Broadcasting Media Stream.


It actually happens sometimes when I am trying to stop the render streaming connection
so I would do something in the RenderStreaming.cs like RenderStreaming.Stop()
then wait for a couple of seconds and then delete the game object if it still exists.
But the crash is happening by the time I am asking the renderStreaming to stop and not after
2 seconds when I am trying to delete the gameobject...

Is there a way I could manage to deal with this problem like catch the exception on the webrtc.dll.....??

Or maybe I could check any condition that would make things easier to control like for instance
RenderStreaming.isRunning or SingleConnection.IsActive....?



Thanks for any advice!!!

 

Related to my previous message I have another question,

So when stopping a WebRTC Broadcast - Receiver Stream
are we supposed to call the SingleConnection.DeleteConnection(connectionId)
before anything else??

Or actually, how are we supposed to stop/kill/clear an established stream?

Thanks !!

 

Thank you for the quick response,

So I am using RenderStreaming 3.1.0 - exp1
that uses WebRTC 2.4.0-exp.3

However, I have made them both local and put them in my Assets folder. then I had to modify them a bit...
Mainly the RenderStreaming.cs()
I haven't touched the WebRTC package other than using some Debug.Log()...

I was experimenting yesterday and now I am calling
Singleconnection.DeleteConnetion(connectionID) for the receiver
and Broadcast.DeleteConnection(connectionID) for the broadcaster

AND I am NOT calling RenderStreaming.Stop();

then after a couple of seconds I delete the objects these components are attached on

I didnt have a crash since then and I tried like around 20 sessions (starting- deleting streams) in a row (normally at 12-15 sessions I would get a crash)

However, if I try to close my app window on Windows, or If I call Application.Quit() ,
the app will freeze trying to close ( not responding state).
I think this is related to the RenderStreaming module cause if I never create a stream
and I just play for a few minutes then I can kill the app normally...

Thats the state now...

Cheers

 

Hi, I have a problem to get the render streaming to work on a single computer without an internet connection.

I didn't get any error message in Unity (2019.4.30f1) or in the Browser (Chrome / Safari), but the screen stays gray after I hit the play button in the browser.

I have used the setup described in the tutorial section of the documentation (3.1).
But I modified the following parts of the example:
- I replaced the external links to javascript libraries in the public/videoplayer/index.html with local ones
- I removed the Google Stun Server from the Ice Server List in in the Unity App and the WebApp (public/js/config.js)
- I repackaged the Webserver

Did I missed something? Or is an Ice Server mandatory for it to work?

Thanks in advance!

 

Hi,

I would like to change video resolution when refreshing the video player page. I have tried changing the resolution of a render texture but when I refresh the site, video becomes streched (for example if I change the aspect from 1:1 to 9:16). Is there a way to correctly reset render streaming just like going out of the play mode and enter back in? I have tried to call Stop function in RenderStreaming.cs but when I call Run and refresh the video player page I get the following error: InvalidOperationException: This instance has been disposed. Unity.WebRTC.MediaStreamTrack ... and there is no video on the screen.

Thank you for your help!

 

So I am also getting the same error as epicoro in the previous post....
That is :
error: InvalidOperationException: This instance has been disposed. Unity.WebRTC.MediaStreamTrack


I think the main question here is how would we dispose of any resources used when we want to stop
and clear a stream ..?

Is the DeleteConnection(connectionID) and the RenderStreaming.Stop() needed and in what order?
Is there anything else needed.....?

Would be really handy to know that.

In my case, I am not calling renderStreaming.Stop() but my app can not be killed/terminated (cause the webrtc.dll is not cleared yet....?)

If I do call renderStreaming.stop(), I might get an app crash....

Thanks for any help

 

Hello @kazuki_unity729

Do you have a planned date to release 3.1.0 as the stable branch?

Thanks

Tarik

 

If I connect from the browser while "Hardware Encoder Support" is enabled, the Unity Editor always crashes.

• Unity 2020.3.21f1
  
• HDRP 10.7.0
• Render Streaming 3.0.1
• Windows 10 Professional
• Firefox 93
• Nvidia GeForce RTX 2070 SUPER
• Nvidia driver version 496.49

There are no error messages in the Editor log. The stack trace is:

Code (log):

1.  
2. 0x00007FFE78A64F99 (KERNELBASE) RaiseException
3. 0x00007FFDEF82B3D5 (webrtc) UnityPluginUnload
4. 0x00007FFDEEF79328 (webrtc) UnityPluginUnload
5. 0x00007FFDEEF89714 (webrtc) UnityPluginUnload
6. 0x00007FFDEEF85D64 (webrtc) UnityPluginUnload
7. 0x00007FFDEEF78AAA (webrtc) VideoTrackRemoveSink
8. Signaling: Receiving message: {"from":"922064e6-44f1-4bc5-9b09-59ff47515d16","to":"","type":"candidate","data":{"candidate":"candidate:1 1 UDP 1685987327 76.174.77.172 54513 typ srflx raddr 0.0.0.0 rport 0","sdpMLineIndex":"2","sdpMid":2,"datetime":1635360475838}}
9. UnityEngine.StackTraceUtility:ExtractStackTrace ()
10. UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
11. UnityEngine.Logger:Log (UnityEngine.LogType,object)
12. UnityEngine.Debug:Log (object)
13. Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs:159)
14. WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
15. WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
16. WebSocketSharp.WebSocket:message ()
17. WebSocketSharp.WebSocket/<>c__DisplayClass174_0:<startReceiving>b__1 (WebSocketSharp.WebSocketFrame)
18. WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:<ReadFrameAsync>b__3 (WebSocketSharp.WebSocketFrame)
19. WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:<readPayloadDataAsync>b__0 (byte[])
20. WebSocketSharp.Ext/<>c__DisplayClass60_0:<ReadBytesAsync>b__0 (System.IAsyncResult)
21. System.Net.Sockets.SocketAsyncResult/<>c:<Complete>b__27_0 (object)
22. System.Threading.QueueUserWorkItemCallback:System.Threading.IThreadPoolWorkItem.ExecuteWorkItem ()
23. System.Threading.ThreadPoolWorkQueue:Dispatch ()
24. System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()
25.  
26. (Filename: Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs Line: 159)
27.  
28. Signaling: Receiving message: {"from":"922064e6-44f1-4bc5-9b09-59ff47515d16","to":"","type":"candidate","data":{"candidate":"candidate:1 2 UDP 1685987326 76.174.77.172 54515 typ srflx raddr 0.0.0.0 rport 0","sdpMLineIndex":"2","sdpMid":2,"datetime":1635360475842}}
29. UnityEngine.StackTraceUtility:ExtractStackTrace ()
30. UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
31. UnityEngine.Logger:Log (UnityEngine.LogType,object)
32. UnityEngine.Debug:Log (object)
33. Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs:159)
34. WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
35. WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
36. WebSocketSharp.WebSocket:message ()
37. WebSocketSharp.WebSocket/<>c__DisplayClass174_0:<startReceiving>b__1 (WebSocketSharp.WebSocketFrame)
38. WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:<ReadFrameAsync>b__3 (WebSocketSharp.WebSocketFrame)
39. WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:<readPayloadDataAsync>b__0 (byte[])
40. WebSocketSharp.Ext/<>c__DisplayClass60_0:<ReadBytesAsync>b__0 (System.IAsyncResult)
41. System.Net.Sockets.SocketAsyncResult/<>c:<Complete>b__27_0 (object)
42. System.Threading.QueueUserWorkItemCallback:System.Threading.IThreadPoolWorkItem.ExecuteWorkItem ()
43. System.Threading.ThreadPoolWorkQueue:Dispatch ()
44. System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()
45.  
46. (Filename: Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs Line: 159)
47.  
48. Signaling: Receiving message: {"from":"922064e6-44f1-4bc5-9b09-59ff47515d16","to":"","type":"candidate","data":{"candidate":"","sdpMLineIndex":"3","sdpMid":3,"datetime":1635360475870}}
49. UnityEngine.StackTraceUtility:ExtractStackTrace ()
50. UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
51. UnityEngine.Logger:Log (UnityEngine.LogType,object)
52. UnityEngine.Debug:Log (object)
53. Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs:159)
54. WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
55. WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
56. WebSocketSharp.WebSocket:message ()
57. WebSocketSharp.WebSocket/<>c__DisplayClass174_0:<startReceiving>b__1 (WebSocketSharp.WebSocketFrame)
58. WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:<ReadFrameAsync>b__3 (WebSocketSharp.WebSocketFrame)
59. WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:<readPayloadDataAsync>b__0 (byte[])
60. WebSocketSharp.Ext/<>c__DisplayClass60_0:<ReadBytesAsync>b__0 (System.IAsyncResult)
61. System.Net.Sockets.SocketAsyncResult/<>c:<Complete>b__27_0 (object)
62. System.Threading.QueueUserWorkItemCallback:System.Threading.IThreadPoolWorkItem.ExecuteWorkItem ()
63. System.Threading.ThreadPoolWorkQueue:Dispatch ()
64. System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()
65.  
66. (Filename: Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs Line: 159)
67.  
68. Signaling: Receiving message: {"from":"922064e6-44f1-4bc5-9b09-59ff47515d16","to":"","type":"candidate","data":{"candidate":"","sdpMLineIndex":"0","sdpMid":0,"datetime":1635360475900}}
69. UnityEngine.StackTraceUtility:ExtractStackTrace ()
70. UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
71. UnityEngine.Logger:Log (UnityEngine.LogType,object)
72. UnityEngine.Debug:Log (object)
73. Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs:159)
74. WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
75. WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
76. WebSocketSharp.WebSocket:message ()
77. WebSocketSharp.WebSocket/<>c__DisplayClass174_0:<startReceiving>b__1 (WebSocketSharp.WebSocketFrame)
78. WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:<ReadFrameAsync>b__3 (WebSocketSharp.WebSocketFrame)
79. WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:<readPayloadDataAsync>b__0 (byte[])
80. WebSocketSharp.Ext/<>c__DisplayClass60_0:<ReadBytesAsync>b__0 (System.IAsyncResult)
81. System.Net.Sockets.SocketAsyncResult/<>c:<Complete>b__27_0 (object)
82. System.Threading.QueueUserWorkItemCallback:System.Threading.IThreadPoolWorkItem.ExecuteWorkItem ()
83. System.Threading.ThreadPoolWorkQueue:Dispatch ()
84. System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()
85.  
86. (Filename: Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs Line: 159)
87.  
88. Signaling: Receiving message: {"from":"922064e6-44f1-4bc5-9b09-59ff47515d16","to":"","type":"candidate","data":{"candidate":"","sdpMLineIndex":"2","sdpMid":2,"datetime":1635360475944}}
89. UnityEngine.StackTraceUtility:ExtractStackTrace ()
90. UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
91. UnityEngine.Logger:Log (UnityEngine.LogType,object)
92. UnityEngine.Debug:Log (object)
93. Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs:159)
94. WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
95. WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
96. WebSocketSharp.WebSocket:message ()
97. WebSocketSharp.WebSocket/<>c__DisplayClass174_0:<startReceiving>b__1 (WebSocketSharp.WebSocketFrame)
98. WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:<ReadFrameAsync>b__3 (WebSocketSharp.WebSocketFrame)
99. WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:<readPayloadDataAsync>b__0 (byte[])
100. WebSocketSharp.Ext/<>c__DisplayClass60_0:<ReadBytesAsync>b__0 (System.IAsyncResult)
101. System.Net.Sockets.SocketAsyncResult/<>c:<Complete>b__27_0 (object)
102. System.Threading.QueueUserWorkItemCallback:System.Threading.IThreadPoolWorkItem.ExecuteWorkItem ()
103. System.Threading.ThreadPoolWorkQueue:Dispatch ()
104. System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()
105.  
106. (Filename: Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs Line: 159)
107.  
108. 0x00007FF77A279CAB (Unity) GfxDeviceWorker::RunCommand
109. 0x00007FF77A28256C (Unity) GfxDeviceWorker::RunExt
110. 0x00007FF77A282AB8 (Unity) GfxDeviceWorker::RunGfxDeviceWorker
111. 0x00007FF777C1C1C5 (Unity) Thread::RunThreadWrapper
112. 0x00007FFE7A367034 (KERNEL32) BaseThreadInitThunk
113. 0x00007FFE7AEC2651 (ntdll) RtlUserThreadStart
114.  

----

If I disable "Hardware Encoder Support", I get an error each time a client connects:

Code (csharp):

1.  
2. ArgumentException: create candidate is failed. error type:InvalidParameter, sdpMid:
3. sdpMid:0
4. sdpMLineIndex:0
5.  
6. Unity.WebRTC.RTCIceCandidate..ctor (Unity.WebRTC.RTCIceCandidateInit candidateInfo) (at Library/PackageCache/com.unity.webrtc@2.3.3-preview/Runtime/Scripts/RTCIceCandidate.cs:261)
7. Unity.RenderStreaming.RenderStreamingInternal.OnIceCandidate (Unity.RenderStreaming.Signaling.ISignaling signaling, Unity.RenderStreaming.CandidateData e) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/RenderStreamingInternal.cs:415)
8. Unity.RenderStreaming.Signaling.WebSocketSignaling+<>c__DisplayClass36_3.<WSProcessMessage>b__4 (System.Object d) (at Library/PackageCache/com.unity.renderstreaming@3.0.1-preview/Runtime/Scripts/Signaling/WebSocketSignaling.cs:212)
9. UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at <07c89f7520694139991332d3cf930d48>:0)
10. UnityEngine.UnitySynchronizationContext:ExecuteTasks()
11.  

Otherwise it works, but the quality and performance are terrible.

 

Hi Unity Team, Kazuki,

I'm currently planning a new game, and I'd need to render short videos of characters in Unity based on a given configuration, and that on demand. In other words, use Unity as a render server to generate videos given some data.

Can Unity Render Streaming helps ?

 

This package is designed for live-streaming video/remote play, not for recording. Maybe you want the Unity Recorder?

 

After more testing, we have found that Unity will crash if hardware encoding is enabled and the video resolution is less than 1280x720. It would be helpful if the documentation stated which resolutions are supported, and if the package displayed a warning when you enter an unsupported resolution.

 

Hi, i want ask about rotating texture, is there any option to rotate texture from WebCamTexture and send it rotated via WebRTC or i should rotate this texture on second side i.e web application? I have problem on android device, texture is rotated in portriat mode, i can rotate preview but sent texture is still rotated.

 

Hello, I have a problem. I use render streaming 2.2.1, unity2019.4.22, the set resolution is 1920 * 1080 / 2880 * 1080, and the graphics card is NVIDIA RTX 3070. In the process of using render streaming, I often encounter sudden disconnection of communication, unable to connect, and the frequency is very high. How can I solve this problem.I really need your help。

Connection method：

Normally connected output：

1.a.Not connected: the service needs to be restarted

b.Abnormally disconnected, can be reconnected without restarting the service:

2.When connected, you suddenly disconnect and enter disconnect. I don't know why you enter this function

 

@kazuki_unity729 Hi, Unity receives two video streams at the same time. How do I tell them apart,I compare the track ID with the Web side track ID, but the ID is different.

 

Hi Kazuki, Unity team,
Is there an official date and/or decision on making Unity Render Streaming an official product, and not only a preview/experimental feature? I am wondering if it is worth investigating this solution for a new project, I would need some certainty with the long term support.
Thanks!