Official Unity Render Streaming Introduction & FAQ

Yeah, except this does not work when the server webpages are loaded on iOS. The iPad Pro is not able to get past these invalid certificates, and no visuals are displayed. Either in Chrome or Safari.

 

Cool, that works!. It would be nice to have that position = pos in there in the example code so others can use it as well
So now I am able to precisely click from the browser in the Unity world using this

Code (CSharp):

1.  
2. var ray = MainCamera.ScreenPointToRay(new Vector2( (x/1280) * Screen.width, (y/720) * Screen.height ));
3. if (Physics.Raycast(ray, out var hit, 1000))
4. {
5.    //do something with  hit.point or hit.collider
6. }
7.  

Thanks!

 

I like to integrate the UnityRenderStremaing and WebRTC packages into an existing project to provide a prototye for the Windows version. Now the nightly build crashes for all non-Windows platforms like WebGL, Android, ...

How can I deal with this kind of problem?
In another project I ended up with manipulating the package manifest in an editor callback method. This worked basically but was a pretty unstable approach.

 

I can see Unity.webrtc release 2.0 is already in git, but is it actually official released yet? I am asking because I need the multi-camera support

 

I post the workaround I found for my own problem. I made a copy of the WebRTC package content from /Library/PackageCache/com.unity.webrtc@1.0.1-preview to /Packages/com.unity.webrtc (without version! ) and make the following 2 changes:

• Unity.WebRTC.Runtime.asmdef:
  Exlude non-Widnows-Platforms
• Add some #if UNITY_STANDALONE_WIN || UNITY_EDITOR_WIN to Assets/Scripts/RenderStreaming.cs so that it is considered only for Windows (or to whatever classes you have on your own that uses namespace Unity.WebRTC)

It would be great if the asmdef changes will be included in the next version of WebRTC or an alternative approach like providing dummy DLLs, ... Thanks

 

Why web server does not using web socket as signaling mechanism??

 

I'm trying the plugin and got the following error for RenderStreaming in the Inspector window:
The associated script can not be loaded. Please fix any compile errors and assign a valid script.

I'm following the tutorial. Any idea how to resolve the error?

 

How can I use it in Android WebView ?

 

Yes, I've been running this on AWS EC2 for a couple of months now, as a HMI/SCADA interface with HDRP, and I'm really impressed by the low latency.. I chose a g4dn.xlarge NVIDIA Gaming Server with Windows Server. It will cost you around $0.7 per hour, so make sure you shut it down when not using it...

Edit: I should add that the AWS SDK for .NET works well from the EC2, giving direct Lambda and DynamoDb (and a few other services) access. This, in combination with a Amazon Kinesis Video Stream to a texture in the WebRTC stream is kind of kick-ass, especially if your other AWS stuff is in the same region...

 

When is version 2.1 likely to be released? I am currently trying to use the render streaming features in a project however we use dependencies which require the OpenGLCore Graphics API for Windows. Is there an estimated time when OpenGL will be supported?

 

Hi, sorry if this has been asked or is easy to solve. everything is working locally, but I want to deploy this to a website but I am confused as to what should be deployed to the client side.

I've been looking here and have everything done for the webserver with the certificates and keys. Do I have to add a url to the hosts file? How do I point the client side to the server?

 

Thanks for your reply.

I guess what I'm asking is if I had a domain and I wanted someone to be able to do this through a url like www.examplesite.com/examplestream what do I need to setup on the www.examplesite.com/examplestream side to get it to function

 

Hi, what other GPUs can we expect to be supported and when? I have to decide now whether to replace my 2019 macbook pro or not...

Also, on guthub it says "This solution is optimised for NVIDIA graphics cards."


This implies that the package also works on other GPUs, albeit with lower performance. Is that correct?

Although on the github FAQ page it says:
"Make sure you're using a graphics card that supports Nvcodec"

this is very conflicting information!

 

Hi guys, as we excited for the future update regarding the Linux support ... can tell us if all of this will work in Headless mode?

 

You are right, just tried some sample project on Docker (ubuntu 18 with X) it didn't get any pixel data on headless.
but it did in -batchmode! (and that good enough!!

 

Is it possible to stream the Windows desktop or Unity Editor? If I wanted to make some runtime customization when working on a 3D model concept? If so can you provide me with some guidance?

 

Kazuki, could you please post an example for output from Unity to JavaScript in browser?

I'm guessing it's as easy as e.g. channel.Send("hello"); in RenderStreaming.cs -- and then
create a ondatachannel with channel.OnMessage in videoplayer.js?

 

Good Afternoon @kazuki_unity729
I have write a demo for webrtc on unity, and it works well on PC chrome browser.
Now I want to make a unity application which contains chrome base browser, and use the browser to see the streaming output. After I click the play button, the render streaming application paused and there is a error log "Network Error: Unity.WebRTC.RTCError"

 

Hi @kazuki_unity729

The render streaming seems to be working for PC and Mac platforms over a local network as well as via the URL signalling for WiFi networks. However, when I try to test it on a smartphone (Android and iOS) then no rendering occurs. The basic framework shows up on the screen, however nothing after that, whereas the Mac and Windows versions have everything from rendering to input.

Any idea what the problem might be?

Also, any timeframe for when microphone input will be accepted from the browser side?

Thanks

 

I got the mobile version working on Android (Chrome) but not on iOS. Tried using Chrome, Firefox and Safari in iOS but it does not seems to work. Are iOS devices supported or is only Android currently supported?

Also, any update on when microphone input will be added to this component?

Thanks again for your help and support

 

I found a similar problem on iOS: It works (of course under https only) but only if the ocnnection goes over Wifi. Whenever I turn off Wifi and switch to a pretty good LTE connection, the OnDataChannel callback is never called. The last Unity console output shows signaling.PostCandidate().
The same behaviour occurs when I pause Wifi during an established connection. The data channel gets lost reproducably afterwards.

 

Lastly, when is the microphone input being added or planned for this component?

Thanks for your help and support

 

Yes specifically accepting microphone input from the browser side to send back to the running project. That would be a very useful feature to include in this already amazing API

Will be patiently waiting for further development on this component! Thanks again for the help!

 

When is Unity going to add in all of the mouse input functions so that we don't have to code around this? This WebRTC capability is amazing, but it's really lacking in the input department - the other glaring issue being that I can't click on UI buttons and wasPressedThisFrame does not get picked up (I realize the browser click may be delayed getting to Unity, but this should be handled). This package has absolutely phenomenal potential, but these issues are HUGE hindrances. I'd just love to know what the roadmap is for addressing them. Thanks for all the hard work though!

 

Yes. I just tested it: Wifi works, LTE not. There are 3 calls to Signaling.PostCandidate() and the WebRequests get back without any errors after 1 second. But I observed a different bahavior of RTCIceConnectionState changes in the RTCPeerConnection.OnIceConnectionChange delegate:

• Wifi to ethernet in the same company network directly jumps to Connected after about 1 second.
  
• LTE shows Checking after 6 secs and Failed after another 15 secs.

The Safari web developer tools on my Mac show the same output from video-player.js.

I see. The control center suspends the browser and thus the connection is dropped.

 

It looks like some firewall problem. The LTE based IP seems to be blocked according to the firewall logs. A similar behavior is found when a Windows host is connected via VPN. On the other hand a Mac with Tunnelblick VPN has no problem. I guess it depends on either the VPN implementation or the VPN connection was preferred.

I am no networking expert but as far as I understand it, a STUN server might not be sufficient in some cases and a TURN server is needed. Is this correct?

 

Is there any way I can get RenderStreaming to work whilst in -batchmode on Windows 10? I tried manually grabbing the camera and calling camera.Render() on an update/coroutine. I still get a black screen in my browser (audio and input works though).

This would be necessary for my setup of running Unity on a server machine.

Thanks Kazuki, Sindharta