Question Using WebRTC package as a client for Unity Render Streaming

Hello,

Apologies if this is a stupid question, but is the WebRTC package appropriate to be used as a client for remotely viewing another Unity app, using Unity Render Streaming? We would like to use it for Edge Rendering, i.e. render a scene on a PC then view it on an Android/iOS device running our client app.

We can't just use a browser because we want to add extra details on the client, and make the whole process invisible to the user.

If so, what is the sample in the WebRTC package closest to being able to do that (I don't see any that obviously set up a remote URL)?

Thanks,
David

 

Hi Kazuki, thanks very much for your reply.

The Android support coming is great news - we are looking for a cross-platform solution.

I was more asking if this is an appropriate use of WebRTC - is it already compatible with Unity Render Streaming out of the box? Which of the WebRTC sample scenes would be a good starting point for that?

 

Ok, that's good to know, thank you.

I'm looking for a bit of guidance on how to do exactly that (implement the signaling) - half of it will have been done already by Unity Render Streaming's webserver, right? So I would need to know what to get and push to start getting the video stream from there?

 

Hello, I see there aren't any official builds with android support, but there is an experimental branch. Can I somehow use it?

EDIT: I managed to use the experimental plugin, but on my android device I get an exception "Unable to load dll 'webRTC': the specified module could not be found". Is there anything I can do about it?

 

@krystianjablonski3
* Set Player-> Android-> Other Settings-> Scripting Backend in Project Settings to IL2CPP
* Set Player-> Android-> Other Settings-> Target Architectures in Project Settings to ARM64
* Make sure you have the libwebrtc.aar file in Plugins / Android
Unity Render Streaming currently only supports ARM64.

 

That worked! Thank you so much!

 

Will there also be webGl support? Soon or later?

 

I would like to lodge a request for WebGL support..

 

Hi,
I want to followup on "dtaddis" request here.
I have more or less the same request and I hope that someone can guide me.
My goal is to create an app which has AR on client-A and streams this to client-B. In the same time client-B should be able to stream his cam to client-A and bidrectional on both clients with Audio. client-A should receive events from client-B. Client-B should be Browser or WebGL, client-A IOS/Android.
Basically it seems that Unity-WebRTC Package does match the requirements, but I cannot read if it is compatible with AR and signaling process is missing. If I screen Unity-Render-Streaming, I'm confused and not 100% sure if bidrectional Video/Audio Browser<>iOS/Android is working.
Does anyone knows more about these two packages and may guide me to the right solution?

 

Hi ,
Could you please let me know, if AR screen sharing is possible using webrtc in unity for Android and if it's available.
I am stuck with the signalling server part of the webrtc could someone provide me an implementation example which I could use as a starting guide

 

Is it possible to create an android game and play it on an android phone while streaming on another phone ?

 

Thanks! I'll test it. WebGL support would be great!

 

Can I stream a camera from one Unity project and see the rendered image in another Unity project? I can't figure out how to make render streaming work and keep seeing "Signaling: WS connection closed, code: 1006" appear repeatedly.

I've searched for hours and can't find any explanation or documentation that shows how to get any of the examples working.

 

I want to do almost the same thing as @simu3105 but haven't found any tutorial/sample on the web. I want to create a cross platform video chat app between IOS/Android and Browser using WebRTC. I have already implemented the browser's implementation and the Unity implementation using video streaming with WebRTC but I am confused about how to connect them both using a signaling server / Unity render streaming.

@simu3105 did you solve your problem? If yes, then can you provide any solution on how you did this? Thanks.

 

I am wondering if you have any information about how to get started with the networking aspects of the signaling process, that you mention we have to implement ourselves. Going through the WebRTC tutorials and samples, they always have two local instances of RTCPeerConnection but I have seen no indication of how this would look in a networked environment of any kind. Is there any sample available anywhere of a Unity WebRTC app running on two different computers? In our case we want to have in-game voice chat that includes an arbitrary data channel.

 

Does this mean that I am able to build the unity app with render streaming on android?

 

yes

 

That is great. So I can share with you what I am seeking to achieve. I need to build a unity application with ARFoundation and stream that app to the browser. so is this achievable?

 

I think you can use the ScreenStreamSender component.

 

I have added a render streaming, broadcast, and a ScreenStreamSender components to the ar camera. It does not work and I am getting a black screen on the browser.

 

Now it works only if I make the ar camera to render in a target texture, so I need the ar foundation camera to render to both a target texture and the screen. Any ideas?

 

I am using the following script in order to blit the camera frames to a render texture. the only problem here is that it does not show the rendered image it only shows the camera with no rendering. to explain this more if I am having a 3d cube in the ARFoundation scene, this script will show the camera without the cube on the render texture, although the scene is rendered on the screen normally. what I need to do is to render on both the camera and the rendered texture.

Code (CSharp):

1. public class CameraTextureMixer : MonoBehaviour
2. {
3.  
4.     [SerializeField]
5.     private RenderTexture ARTexture;
6.  
7.     private void Awake()
8.     {
9.         ARCameraManager cameraManager = GetComponent<ARCameraManager>();
10.         ARCameraBackground ARBackground = GetComponent<ARCameraBackground>();
11.  
12.         cameraManager.frameReceived += (e) =>
13.         {
14.             foreach (var t in e.textures)
15.             {
16.                 Graphics.Blit(t, ARTexture, ARBackground.material);
17.             }
18.         };
19.  
20.     }
21.  

 

I am using WebRTC 2.4.0-exp.7 rather than using Render Streaming. It works on Android app on WIFI but it does not work when one of client is on cellular. I am using google stun server as below.
{'iceServers': [{'urls': 'stun:stun.l.google.com:19302'}]}

Do I have to use a TURN server to make it done?

 

Can we have a sample how to join a room between 2 devices in Unity please ? I'm not expert on networking ... I tried WebRTC on Browser in another place before: Step 1. Type the name of Room ; Step 2. Click Join : Which device joined in this Room first will be the Host ( Local ) , other devices join after will be Clients (Remote) . I don't understand how Clients join in the Host in Webrtc Unity Sample... I'm trying com.unity.webrtc@2.4.0-exp.8

 

Not yet. I will try it tonight. Thank you very much for the link !

 

I tried it. It worked on PC and Unity Editor. But it failed on Publishing Web HTML5 and iOS ... I'm not good at Networking. I will wait for next versions which can be used for everyone . Thank you for your hard works.

 

Can we please have a very simple example demonstrating how to connect to a remote WebRTC stream to be able to access a video stream?

I am trying to figure out how I can stream WebRTC video over network into Unity to be applied as a video texture. It would be great if there was a simple example detailing how to connect to an external IP or URL to access a stream/track.

 

Did you get a feedback on this ?

I'm in the same need. The documentation is very poor and the samples only focus on same machine/network example.
There is not signaling examples in them either.

 

Did anyone succeeded in creating project with multiple video streaming or audio streaming or messaging application?

If yes, then please share sample project so I can have an idea about what should be implemented. I do know about Photon and Agora, and they seem to be very easy to implement so similar way I thought, sample projects from WebRTC and Renderer steaming will be easy, but I am not getting anything from there. Sample projects works, but I am not able to figure out about where to start taking reference to create my solution.

And If I build for android directly the sample scenes, will it work? or do I ned to do something else also (coding wise)

Help Needed.

 

How do I connect my windows application (EXE) with android app (APK)?

 

I have followed the Unity render streaming's Broadcast example and it is working fine

Now I want to see he same thing in my Android device, which is in the same network, In android browser it works fine but I want that same feed inside an android app.

As you can see in photon networking and Agora framework, there is structure like "Lobby", "Room".
So WebRTC works like that only or not.
Should I enter ip that mentioned in "webserer.exe" inside android app?

How can I achieve this?
I checked the document of tutorial, but I am not clear about how can I make full script, Should I create offer first or answer, and where should I write code that will connect to the ip that I give as an input?