Official Unity Render Streaming Introduction & FAQ

Has anyone tried with Gear VR

Or
https://github.com/stewdio/THREE.VRController

 

Has anyone tried running this in the cloud? AWS or otherwise?

 

using the maxed out new macbook 2019 bootcamp Win 64...
does it not work using AMD? up to date on radeon 17.12
[WebRTC] The hardware codec driver not installed

System.Exception: Cannot load. Incorrect path: Packages/com.unity.render-pipelines.high-definition/Runtime/RenderPipelineResources/ShaderGraph/AutodeskInteractiveTransparent.ShaderGraph Null returned.
at UnityEngine.Experimental.Rendering.HDPipeline.ResourceReloader.Load (System.String path, System.Type type, System.Boolean builtin) [0x00036] in

 

Any idea when you will add multiple cameras?

 

Okay this is exactly what I wanted local-co-op now works online without having to do anything this looks like a gamechanger
https://support.steampowered.com/kb_article.php?ref=3629-RIAV-1617#whatisit
https://forum.unity.com/threads/steam-remote-play-split-screen-multiplayer-out-of-the-box.758171/

 

Kazuki, would you expect that uGUI buttons are able to be clicked through the remote stream? In my simple test, it doesn't get clicked. It does get clicked when I try it locally.
My app has only one camera, for GUI (2D app), and a secondary camera only to display the visuals locally using HDRP Render Texture Blitter.

Secondly, it appears that the WebRTC stream steals the audio output, such that I can hear the audio in the remote stream but not locally. Is there a "HDRP Render Texture Blitter" equivalent for audio output?

Thanks

 

I do indeed have the InputSystemUIInputModule on the EventSystem object (I replaced the legacy component with it, via the button that appears on the Event System component when you have the legacy component on it).
It does not give me a hover nor the click events when interacting through the Remote video stream. It does work through the local window.

I have a zip of my project if this is something you can take a look at. It's 40MB.

Thanks

 

Is this compatible with Oculus Quest?

 

I am working on streaming video from one Unity client to another. I want to stream the local render texture to a remote render texture without using the browser. How would I set up my render texture on the remote Unity client to listen to the video stream? Thanks in advance!

 

I have investigated more information about the GUI event limitations.

Unity is set to not process events while it does not have focus. Therefore while it will work if the game is in the foreground on the "server" machine and you are playing remotely on a different machine, it won't work if the game is not in the foreground, or at all if you are trying to test on your own workstation through a browser (both the browser and Unity cannot have focus at once).

This seems like an unintended consequence regarding operating through RenderStreaming, where the user expects the game to be in "focus" if the web browser is on-screen. I could see how this could complicate the InputSystem code. Likely you'd want the concept of focus to be per-peer and the InputSystemUIInputModule would process events for those peers who are active and not for others (or locally).

As a hack, I've made two modifications to the input code. First, I've commented out the condition in InputSystemUIInputModule:oProcess() that forgoes processing events if the app doesn't have focus. This fixes it for standalone. Secondly, I commented out the early-out check for InputUpdateType.Editor in InputActionState::IInputStateChangeMonitor.NotifyControlStateChanged() which fixes it for running on the same machine while in the editor. This allows me to develop on my workstation the most efficiently. I do not know the extent of side effects these changes will have.

Is there any chance this issue of focus would be resolved in a future update?

Thanks

 

I would like to use the Render Streaming functionality but with Unity's default render pipeline. How would I remove the HDRP from this project? Thanks!

 

I am experiencing video freezing and grey screen when my (Chrome) browser is on a screen that is 2650x1600 and larger. Just after this happens the input key presses and button events, e.g. the light turning on and off, still come through to the Unity client from the browser. I am also not able to get any browser on any device in my local network to get the video stream. On any device I try I am seeing a grey screen after clicking play in the browser. However, I have been able to get video up and streaming again by power cycling my server machine. Just restarting the node server doesn't fix the issue. I might also add that when I refresh the browser and get the grey screen I am seeing a valid connection state in the js console (see image).
My goal is to stream a video at 2650x1600 at ~30fps or more. Is this achievable? Or am I hitting some bits-per-second capacity from the Unity client upload stream? Thank you!

 

kazuki_unity729, sindharta_at_unity and anyone who is anywhere related to the creation of this thing.
Guys this is amazing! well done! just set it up and it f***** works!! you are gods to me!
I do have a lot of questions and information I wish to gather on this amazing thing.
First of all - where is the best place to ask them?
...top stuff that and the very important to me and my colleagues to know:
When could be expected Linux support?
Will it work headless? (server build)
any chance to make an LWRP template (it will be far more useful for most use cases)

Many Thanks!

 

Hi, can you elaborate on how to make it on regular or LWRP? (Mabey a small tutorial or something)
Thanks!

 

いちいち、ref つけなければいけないし、RTCSdpTypeが頭文字大文字なもんだから素直にjsonに変換できないし、イベントは += で書けないし、実装するのにイライラが募る。

 

What about Unity acquiring obvioos:
https://blogs.unity3d.com/2019/11/01/unity-acquires-obvioos-creators-of-furioos/

I have tried it and it already works ok, but I get strong blocky artefacts in the streamed video.

Or is this offtopic?

 

Thanks Bro!
we still running tests with the HDRP and it still looks great!!!.
sorry for asking kind of the same question but it's going to be important to us very soon and it seems the answer is out there...
Is it possible to use the default rendering pipeline (not the HDRP or the LWRP)?
and do you have an implimintatiopn of the LWRP / (default render pipeline - if the answer for the above is positive)
version of HDRPRenderTextureBlitter.

Again, many thanks, and you are doing something incredible here!

 

Hi @Kazuki and @sindharta, thanks for the fast response, it is very encouraging and motivating for rapid tests.
I can confirm from last night tests:

• We were successful to stream realtime webcam video input on world scale canvas over 3 different OS: Windows, Linux, and Android (all with chrome browser
• Server build option (the checkbox) from inside unity build settings - failed (a decent amount of errors on the command prompt)
• Silent launching via command prompt (-batchMode) - failed, but different, the app itself runs but there is no rendering... meaning we getting black screen in the browser, we also confirmed it with capturing snapshots and build png images using render texture attached to the camera.

Hopefully, this information helped you, and please continue this awesome work!

BTW - Apparently there is no need for the new Input system.. all the tests above ran without the package

 

Love the package @kazuki_unity729 !

I had one question:

I think I have a pretty good understanding of how the RemoteInput is captured and sent, but wanted to verify:

Do you think its possible for me to edit RemoteInput to also capture things like gyroscope or the 'position' the phone reports from AR Kit?

Gyroscope seems a bit more doable, but unsure about AR Kit.....would love some input if you have time. Don't want to waste time trying to extend RemoteInput if it isn't really meant to be extended to further inputs

 

Running into an issue where the outputed renderTexture seems to be the wrong color space? I'm using LWRP if that makes a difference...

Notice how the 2nd image is slightly lighter in color (which makes me think its a gamma vs linear color space issue). It's subtle but its enough to be annoying that the colors don't match what's actually being rendered.

Any thoughts @sindharta_at_unity or @kazuki_unity729 ?

(Render Texture version)


(Unity Version)

 

After looking further into it, the renderTexture that is being sent has the same color as the game view, which is different than the remote texture on the web page.

Perhaps it has something to do with how the webpage is interpreting the texture? It's a subtle but important difference between the colors....

 

Unity 2019.2.9f1
LWRP 6.9.1

I would upgrade to URP, but it does not seem like a trivial namespace change like its being advertised (just from my brief looking into it, they at the least swapped in a whole new post-processing implementation in this "simple name change"). Also, we've noticed that upgrading the render pipelines, while fixing some issues, creates just as many new ones with each new version.

But curious if this is specific to this LWRP implementation

 

Hi, I'm making a MOBA that requires thousands of particles to be simulated and synchronized across all clients, has many physics calculations that must be synchronized every iteration, and has projectile motion with non-deterministic behavior (wind, electrostatic forces, or another player can all affect the motion).

Would implementing this game using render streaming, such that all physics and effects are on the server and cameras from the simulated "world" server have their sound and video streams relayed to clients, be a viable approach to networking my game?

 

Ran into another issue....

If I use the RenderStreaming component, it seems to be muting the audio on the computer and sends it to the remote device.....which is cool.

But I don't want it to mute the computer audio. If I don't send the audio stream over, the computer plays audio as expected. Is there a way to have both the computer and the remote stream both playing back audio?

 

Is there any projected date on 2.0 release?

Also, server GPU power will not be a problem.

In terms of latency, how many ms can be expected?

 

Some other notes:

- Video stream will randomly cut out (although RemoteInput still works). Unclear what causes this.
- The video stream was incredibly choppy when played back in Chrome on Windows. iOS Safari works great though.

I can file these on GitHub if that's easier

 

This is really promising. I have two questions.

How can I get the UI canvas and button interaction to work in the browser. I tried setting the Canvas render mode to Screen Space - Camera and set the camera to the streaming camera. I do get the UI in the UI but UI is not visible on the host.

I have to set the target texture Depth buffer of the streaming camera to "At least 16 bits depth in order to see textures and object, can I set this somewhere else?

 

Hi, I have gone through the github tutorial steps 3 times, each time getting "NET::ERR_CERT_INVALID" in Chrome and Safari.

Can you please give further advice?

 

Also I would like to add, that without this SSL certificate, I am unable to view anything on an iPad Pro. The screen remains solid color, however the html buttons and microphone do control the unity scene.

 

How can I get the remote mouse position? I do get the delta values using RemoteInput.RemoteMouse.delta.ReadValue()
but we need to be able to click on objects so I was trying RemoteInput.RemoteMouse.position.ReadValue() and RemoteInput.RemoteMouse.position.x.ReadValue() but those return 0,0

 

how can we do this?, I need the mouse position for clicking objects