Official Unity Render Streaming Introduction & FAQ

I want to use only one camera. what should I do?
When either camera is removed, the screen goes black.
I don't need a screen with a red border.

 

@kazuki_unity729, Is it possible to share the Unity Scene screen in the bidirectional Sample? I am trying to send "Screen Stream Sender" in the Bidirectional Sample instead of WebCam Streamer Sender so that the Sender can share the unity Scene Screen (MainCamera ) with the receiver and in return see the receivers Web came image.
thank you

 

I am following the latest sample tutorial with 3.1.0 the web app and unity editor running fine. No error on both consoles. But i am getting black screen on received html page. Remember I have empty scene where I have setup Render streaming, Broadcast, camera Stream Sender compoenet but i am not getting output or error on web. I have make sure that Firewall is turn off and my browser is with latest version. Do I am missing anything?

 

I am just testing the new render streaming package in unity 2020.3.22 but sadly the receiver (html page) is not receiving my camera stream. While both consoles are clear there is not error. Currently i have Main camera which have camera Stream Sender is attached and its is showing two info:

While on another empty object I have attached

1. Render Streaming
2. Broadcast
3. Broadcast Sample

One strange thing I found is that the sample scene (Receiver) is working fine. But the same setup which I have made is not working. Probably the reason is given in above screenshot. why it is happening?

While the original documentation has also been missed that you have to attached the broadcast sample script.

 

I have a issue about I try to copy example from broadcast and receiver.
It sent signaling only that there is a DataChannelBase component like input sender and Boolean isLocal checked.
Even if I did that, it would be send offer signaling several times cause linking failed. (like #502)
How do I fix it?

 

Hi, I use WebBrowserInput demo on LAN, WebSocket, everything else by default. I opened it in the browser and could see the Unity camera's real-time content and interactively control the lighting. When I mapped the machine to a WAN address, I could open a web page on another machine using a browser on the WAN and see buttons and a gray background. But clicking the button does not work, and you cannot see the video. How should I modify it? Thank you very much!

 

thank you for reply.
To be precise, the scene has two cameras, but on the web I want to see only one camera screen.
When using broadcast, only the camera whose target display is display1 will be shown on the web screen.
But I want to show the screen of the camera whose target display is display2 on the web screen.

What I want is to show the screen of the camera whose target display is display2 on the web screen and operate the camera in Unity from the browser.

(I use a translator so the sentences may be weird. Sorry)

 

Currently, we are developing an app that remotely sends video of scenes including volumetric video.
If you execute the AudioStreamTrack constructor with no arguments, the audio will be sent from the AudioListener, so a stream is generated with the following code and set in the pc.

Code (CSharp):

1. var audioTrack = new AudioStreamTrack();
2. var videoTrack = new VideoStreamTrack(rt);
3. var mediaStream = new MediaStream();
4. peer.AddTrack(audioTrack, mediaStream);
5. peer.AddTrack(videoTrack, mediaStream);
6.  
7. StartCoroutine(CreateDescription(RTCSdpType.Offer));
8. // ....

When I run it, the video is sent and displayed on the remote side, but there is no audio.
For the time being, the OnTrack event on AudioTrack has occurred on the remote side.

We have prepared a test project here.
The file size is about 1GB because it contains volumetric video.
It consists of three projects: sender, receiver, and signaling server.

• 4dv --- sender project
• WebRTCReceiver --- Recipient project
  
• ConsoleSignalingServer --- Simple signaling server for console

In terms of execution order, first start ConsoleSignalingServer, then execute the sending 4dv project, and then execute WebRTCReceiver.
https://drive.google.com/drive/folders/161WdQ9JtKc4Uu-b8PtXgb6OkDWzUqha6

 

[QUOTE = "kazuki_unity729、post：8005601、member：1918555"] AudioListenerでAudioStreamTrackを使用するために、この指示に従いましたか？
https://docs.unity3d.com/Packages/com.unity.webrtc@2.4/manual/audiostreaming.html [/ QUOTE]
Sorry, I was able to receive audio by implementing it according to the document.

 

The version of the Unity Render Streaming package is 3.1.0-exp.3.
The "Broadcast" sample has one camera, but I want to use two.
I want to display one camera on the unity game tab and the other camera on the web. (like RSPic.PNG)

And i want to control unity camera on the web
I tried all the samples but couldn't control the Unity camera from the web.

 

@kazuki_unity729

Just came across this package in development, very cool. How does performance compare to existing assets like FMETP Stream? Also, is the Legacy Input System supported? If not, are we able to send our own custom packets to the Server?

Looking through the source code of your package it seems batchmode won't work at all because you're using ScreenCapture.CaptureScreenshotIntoRenderTexture. Is batchmode something that'll be supported in the future? I'd be curious as to how you'll get Screen Space Overlay UI working in batchmode.

 

Surely Unity would want to support all its existing features though? Is this something you might plan to support in the future? What about batchmode in general with Screen Space - Camera? That is a lot easier to support.

 

It's "com.unity.webrtc" instead of "Render Streaming"

* OS: iOS 15
* Unity version: 2021.2.13
* 2.4.0-exp.6

When building with XCode, the following error occurs.

Code (CSharp):

1. Undefined symbol: _webrtc_audio_processor_destroy
2. Undefined symbol: _webrtc_audio_processor_process_reverse
3. Undefined symbol: _webrtc_audio_processor_process
4. Undefined symbol: _webrtc_audio_processor_create
5. Undefined symbol: _webrtc_audio_processor_init
6. Undefined symbol: _webrtc_audio_processor_set_param

It seems that you can build if you set some settings, but where should you set?

 

I have 3 questions:

1. Is there a sample scene for connecting two independent peers? I tried running the sample scenes on two devices at the same time hoping one of the samples would work for local networked peers or some testing signaling servers, but the device only connects to itself. Is that by design, or I am using the samples incorrectly?

2. Is there a way to get direct access to incoming audio data, instead of using AudioSource.SetTrack extension? I need to do something else with the incoming data, but AudioStreamTrack doesn't allow access to the audio data.

3. If there is no way to get access to audio data, I would need more control over the AudioSource spatial properties. Currently, there is no way to position/pan the "receiving" AudioSource.

 

is there a way to get notified when the system has connected to the Signaling server? I see that RenderStreamingInternal gets access to ISignaling and can respond to ISignaling.OnStart event.
But from the outside, I can't get either notified nor the internal state. I'm currently using HTTP signaling with a server deployed in GCloud, and sometimes it's taking more than 5 seconds to connect and create a session. If I attempt to create a connection before that, of course it falls apart, so I need to know when the system is ready for exchanging offers/answers.
Any ideas?

 

Http requests take too long and they're blocking the main thread
really need these requests to run in a background thread and then run callbacks in the main thread. It's doing that for some requests, but we need all of them to run that way if we want to use this solution

 

well we switched to WebSockets now! It looks like it's supported by Google App Engine in the flex environment. We initially thought that HTTP was our only option

 

hello! how can i use js button to transfer a string as a parameter in unity function and call it. the demo is based on "WebBrowserInput".

 

@kazuki_unity729 is it possible to implement a conference call or meeting call where multiple clients join for a video chat just like in Zoom? what would be the go-ahead to do so? I was thinking to have more than one Single Connection Handler but not sure about the overhead and the connection part.

 

I tried your product and it seems great. However I have an issue. I'm trying to open a new webpage from streaming.
Using Application.OpenUrl open a webpage inside the furioos webplayer. I want to open this webpage outside the furioos webplayer. Is it possible to do that ? Is there a sample project available ?

 

Is it possible to capture and stream a 360 (stereoscopic) camera view?

 

Hi, I'm trying to use the multiplay sample with more than 5 users with cameras. They all have controls and everything is running smoothly except Unity's framerate is dropping drastically on each connection.

The initial host's framerate is around 110 and it drops to 60 on the first connection. Then on the second connection, it drops down to 30, and so on.

Meanwhile, the CPU & GPU usage of the unity standalone windows PC build is around <10%.

I need to figure out a solution to run this sample with at least 15 users.

The hardware I'm using to test:
Intel i7-7700k
RTX 3080
128Gb ddr4 ram

 

@kazuki_unity729 any response to my earlier questions? Thanks!

 

Batchmode is a Unity Command Line Argument you can read more about it here: https://docs.unity3d.com/Manual/PlayerCommandLineArguments.html

We have found a new solution for Screen Space - Overlay using FMETP Stream. Thanks for your help though.

 

@kazuki_unity729 is there any sample code to send a simple string between Client and Server instead of InputEvents?
I am trying to see the InputRemoting.cs script but as the script is part of the package now I can't edit/Debug it.

 

Thank you for your reply! We will be waiting for the update. Ideally, if one PC can run up to 50 users this can be very reliable alternative to WebGL games. Which is something that most of our clients ask for their blockchain games. Fast loading times and high-end graphics.

 

Hi Kannan,

Thanks for the answer, I will try to explore the dataChannel documentation. I think I need to install SDK to use it right?
So it's not possible to implement all the system unity side only.
Is there a sample project where information is sent to player and player react?

Cheers

 

We use websockets for our siginaling. With the open websocket you can push messages to the client. Let's say you want to open a url only on a specific client's machine when they click a Unity button. Detect in the app which user clicked a given button and then from there you can push that url down to the client. Your client side webserver code will need to catch that url and open it up from the html/javascript side.

There is some scaffolding you will need to setup to manage which user input triggered the button click.
If you are communicating with the webserver from the client over HTTP you will be using some type of polling logic and maybe a custom endpoint defined in the webserver.

It is doable.

 

Hi guys,

is there an approximate timeline when version 3.1 will no longer be considered experimetal?

Thx.

 

Hi ,Unity Forma is the render Streaming used in the current official open source version?