Official Unity Render Streaming Package Release Announcement

kazuki_unity729 · Feb 28, 2020

Unity Render Streaming Update v1.2.0

We announce about releasing the version 1.2.0 of Unity Render Streaming.
At the same time, version 1.1.0 of the WebRTC package has been released.

As a note, Please use it with Unity 2019.3.
In this time the main changes are below.

WebRTC package (doc)

IL2CPP support

Added Linux OpenGL hardware encoder

Added MacOS Metal software encoder

Added DirectX11 software encoder

Changed to be able to specify any sound source for the audio streaming

Unity Render Streaming package (doc)

Added AudioStreamer class

Added a flag to select encoder

Added a sample with Universal Render Pipeline

Fixed to display the mouse pointer in the correct position

Fixed to download webserver from a known version if the current version doesn’t exist

The plan for the next version release is in April.
Please take a look at these Github repositories.

Unity Render Streaming

WebRTC for Unity

Written in Japanese below.

Unity Render Streaming アップデート v1.2.0

Unity Render Streaming バージョン 1.2.0 の公開についてお知らせします。
同時に WebRTC パッケージについてもバージョン 1.1.0 を公開します。

なお、Unity バージョンは 2019.3 をご利用ください。
今回の主な修正は以下の通りになります。

WebRTC パッケージ (ドキュメント)

IL2CPP サポート

Linux OpenGL ハードウェアエンコーダー追加

MacOS Metal ソフトウェアエンコーダー追加

Windows DirectX11 ソフトウェアエンコーダー追加

オーディオストリーミングに任意の音源を指定できるように変更

Unity Render Streaming パッケージ (ドキュメント)

AudioStreamer クラスの追加

エンコーダーの選択フラグの追加

Universal Render Pipeline のサンプルの追加

マウスポインタの表示位置の修正

エディタから Web アプリケーションをダウンロードをできない問題を修正

次回のバージョン 2.0 は4月リリースを予定しています。
開発状況は以下の Github リポジトリをご覧ください。

Unity Render Streaming

WebRTC for Unity

kazuki_unity729 · Feb 28, 2020

Unity Render Streaming Update v1.2.1
This is a hotfix version for the issue reported on GitHub Issue.

WebRTC package (doc)

Fixed: DLL import error

Unity Render Streaming package (doc)

Changed: Upgrading WebRTC to 1.1.1-preview

Written in Japanese below.

Unity Render Streaming アップデート v1.2.1
このアップデートは GitHub Issue で報告された問題に対するホットフィックス版です。

WebRTC package (doc)

DLL 読み込みエラーの修正

Unity Render Streaming package (doc)

WebRTC パッケージを 1.1.1-preview にアップデート

kazuki_unity729 · Apr 1, 2020

Released a development version of webrtc package
For developing the new version of Unity RenderStreaming, we released version 2.0 of com.unity.webrtc tentatively.
So the version 2.0 is in development, please do not use it. instead, you can use the version 1.1.2.

WebRTC パッケージの開発版をリリースしました
Unity RenderStreaming の新バージョン開発のため、現在、 com.unity.webrtc パッケージのバージョン 2.0 を仮でリリースしています。
バージョン 2.0 は現在開発中のため、利用を控えてください。代わりに 1.1.2 をご利用ください。

kazuki_unity729 · Apr 30, 2020

Unity Render Streaming Update v2.0.0
Hello developers, We are announcing the new version v2.0.0.
If you find bugs, please let me know them on GitHub Issue page.
And would happy for me that you discuss more actively on Unity Forum.

WebRTC package (doc)

Multi camera support

DirectX 12 API support

Published VideoStreamTrack API

Published AudioStreamTrack API

Unity Render Streaming package (doc)

Update sample to use multi camera

Gamepad input controller support

Changed: Upgrading WebRTC to 2.0.0-preview

Changed: Upgrading New Input System to 1.0.0-preview.6

kazuki_unity729 · Jul 30, 2020

Released WebRTC package hotfix version v2.0.5
We announce the release of WebRTC package to fix security issue which caused by libwebrtc.

The detail of issue is here.
https://bugs.chromium.org/p/project-zero/issues/detail?id=2034

Next major version 2.1 is in testing phase now. Please wait for the release.

bisc67 · Aug 2, 2020

Please fix the current documentation and examples. The documentation is not very clear, and the samples don't work. The API's in the examples documentation do not match the current API.

kazuki_unity729 · Aug 25, 2020

Released QA version v2.1.0-preview.1

This version is for QA.
If you cooperate our project, please try it and report your issues.

kazuki_unity729 · Oct 27, 2020

Released WebRTC package version v2.2.0

We pleasure to announce the new package version release.

Change log

Added

Software decoder support

Hardware encoder (VideoToolbox) support on macOS

Vulkan API support on Linux and Windows

Linux IL2CPP support

Add WebRTC samples (MultiplePeerConnections, MultiVideoReceive, MungeSDP, VideoReceive)

Changed

Upgrade libwebrtc m85

Upgrade NVIDIA Codec SDK 9.1

Changed RTCPeerConnection behaviour to throw exceptions when pass invalid arguments to SetLocalDescription, SetRemoteDescription

Please give me your feedback regarding our package. If you have some issues, you can post the message to GitHub page.

kazuki_unity729 · Dec 25, 2020

Released WebRTC package version v2.3.0

we pleasure to share an announcement that the new version of the WebRTC package at the end of 2020.

Change log

Added

iOS platform support

H.264 HW decoder (VideoToolbox) support on macOS

Added `GetCapabilities` method to the `RTCRtpSender` class and the `RTCRtpReceiver` class

Added `SetCodecPreferences` method to the `RTCRtpTransceiver` class.

Added two samples (`ChangeCodecs`, `TrickleIce`)

Added properties to the `RTCIceCandidate` class

Added properties to the `RTCDataChannelInit` class

Changed

Changed `RTCIceCandidate` type from `struct` to `class`

Changed `RTCIceCandidateInit` type from `struct` to `class`

Changed `RTCDataChannelInit` type from `struct` to `class`

Changed argumments of the `RTCPeerConnection.AddIceCandidate` method

Finally, iOS platform support has started! Android support is being prepared as the next version.
Enjoy the new year holidays!

kazuki_unity729 · Mar 3, 2021

Released Unity Render Streaming package version v3.0.0

There is a big change!
We made many samples to explain how to develop the application using the streaming technology. And we are preparing the package documentation to describe how to use them. Moreover, this is the first release of the mobile platform support!

Change log

Added

Supported iOS platform.

Added new samples.

Added a documentation for samples.

Added components (InputSystemChannelSender, InputSystemChannelReceiver, WebBrowserInputChannelReceiver, SingleConnection and Broadcast).

Changed

Moved scripts from the sample folder to Package Manager.

Upgrading WebRTC package to 2.3.3-preview.

Known issue

The compile error occurred when importing the package sample. This issue will be fixed soon. The pull request fixing it is here

kazuki_unity729 · Apr 26, 2021

Released WebRTC package version 2.4.0-exp.1

We announced the new version of the WebRTC package.
The changelog is below.

Added

Android ARM64 platform support

Added a sample scene "Menu" which developers can go back and forth between sample scenes

Added a sample scene "PerfectNegotiation"

Added the software encoder on Linux support when using OpenGL Core graphics API

Added the RestartIce method to the RTCPeerConnection class

Added the Streams property to the RTCRtpReceiver class

Changed

Unity 2020.3 support

Upgrade libwebrtc m89

Changed the argument type of the RTCPeerConnection.CreateOffer method and the RTCPeerConnection.CreateAnswer method

Fixed

Fixed crash for accessing properties of RTCDataChannel instance

Fixed crash when using the invalid graphics format to stream video on macOS

WayneVenter · Apr 29, 2021

@kazuki_unity729 Hope you can help me, this is WebRTC related, I'm trying to implement a WS or Socket.IO server with the example DataChannelSample.cs file, I would like to use as much of a vanilla WebRTC implementation as possible, using the complete signaling server from here https://www.tutorialspoint.com/webrtc/webrtc_signaling.htm this is the most basic / reference solution people use. I can't figure out where in the Sample to implement the signaling server.
IEnumerator OnCreateOfferSuccess(RTCSessionDescription desc)
{
Debug.Log($"Offer from pc1\n{desc.sdp}");
Debug.Log("pc1 setLocalDescription start");
var op = pc1.SetLocalDescription(ref desc);
yield return op;

if (!op.IsError)
{
OnSetLocalSuccess(pc1);
}
else
{
var error = op.Error;
OnSetSessionDescriptionError(ref error);
}

Debug.Log("pc2 setRemoteDescription start");
var op2 = pc2.SetRemoteDescription(ref desc);
yield return op2;
if (!op2.IsError)
{
OnSetRemoteSuccess(pc2);
}
else
{
var error = op2.Error;
OnSetSessionDescriptionError(ref error);
}
Debug.Log("pc2 createAnswer start");
// Since the 'remote' side has no media stream we need
// to pass in the right constraints in order for it to
// accept the incoming offer of audio and video.

var op3 = pc2.CreateAnswer();
yield return op3;
if (!op3.IsError)
{
yield return OnCreateAnswerSuccess(op3.Desc);
}
else
{
OnCreateSessionDescriptionError(op3.Error);
}
}
I suspect it should happen after the:
var op = pc1.SetLocalDescription(ref desc);
yield return op;
But its not clear how and when the SDP is processed by PC2 and the answer is generated and sent back.
Would it be possible to provide some guidance as to what functions can be separated to be closer to real world implementation with a signaling server and not just pc1/pc2 in the same scene.

Tanks in advance.

kazuki_unity729 · May 1, 2021

WayneVenter said: ↑

@kazuki_unity729 Hope you can help me, this is WebRTC related, I'm trying to implement a WS or Socket.IO server with the example DataChannelSample.cs file, I would like to use as much of a vanilla WebRTC implementation as possible, using the complete signaling server from here https://www.tutorialspoint.com/webrtc/webrtc_signaling.htm this is the most basic / reference solution people use. I can't figure out where in the Sample to implement the signaling server.
IEnumerator OnCreateOfferSuccess(RTCSessionDescription desc)
{
Debug.Log($"Offer from pc1\n{desc.sdp}");
Debug.Log("pc1 setLocalDescription start");
var op = pc1.SetLocalDescription(ref desc);
yield return op;

if (!op.IsError)
{
OnSetLocalSuccess(pc1);
}
else
{
var error = op.Error;
OnSetSessionDescriptionError(ref error);
}

Debug.Log("pc2 setRemoteDescription start");
var op2 = pc2.SetRemoteDescription(ref desc);
yield return op2;
if (!op2.IsError)
{
OnSetRemoteSuccess(pc2);
}
else
{
var error = op2.Error;
OnSetSessionDescriptionError(ref error);
}
Debug.Log("pc2 createAnswer start");
// Since the 'remote' side has no media stream we need
// to pass in the right constraints in order for it to
// accept the incoming offer of audio and video.

var op3 = pc2.CreateAnswer();
yield return op3;
if (!op3.IsError)
{
yield return OnCreateAnswerSuccess(op3.Desc);
}
else
{
OnCreateSessionDescriptionError(op3.Error);
}
}
I suspect it should happen after the:
var op = pc1.SetLocalDescription(ref desc);
yield return op;
But its not clear how and when the SDP is processed by PC2 and the answer is generated and sent back.
Would it be possible to provide some guidance as to what functions can be separated to be closer to real world implementation with a signaling server and not just pc1/pc2 in the same scene.

Tanks in advance.
Click to expand...

Have you tried Unity Render Stremaing?
https://docs.unity3d.com/Packages/com.unity.renderstreaming@3.0/manual/index.html

P.S.
Could you post these topics to other threads?
I would like to use this thread for announcing new releases.

kazuki_unity729 · Jun 15, 2021

Released Unity Render Streaming package version v3.1.0-exp.1

We released a new version of Unity Render Streaming package. Please refer to the changelog below.

Added
- Android platform support
- Added the `Menu` scene into samples

Changed
- Added drop-down list UI for selecting the video capture device in the `Bidirectional` scene
- Changed the signaling process for exchanging SDPs

Fixed
- Fixed errors when disposing the scene while streaming video
- Fixed the long processing time when disposing the signaling instance

kazuki_unity729 · Aug 19, 2021

Released WebRTC package version 2.4.0-exp.4

We announced the new version of the WebRTC package.
The changelog is below.

Added

mac M1 architecture native support

Audio stream rendering support

Add two scenes (Audio and MultiAudioReceive) into the package sample

Add RTCAudioSourceStats and VideoSourceStats class

Changed

Add the audio waveform graph to MultiplePeerConnections scene in the sample

Fixed

Fix the crash on Windows with Vulkan API on VideoReceieSample

Fix the crash when calling WebRTC.Initialize twice

Fix the error in the build process on Unity2021.2

wechat_os_Qy0wqbCaPPoRQtwTT86V0uglo · Oct 11, 2021

After using webrtc to push the stream, it cannot be used at the Android end when playing in unity, and the socket is not connected

kazuki_unity729 · Oct 12, 2021

wechat_os_Qy0wqbCaPPoRQtwTT86V0uglo said: ↑

After using webrtc to push the stream, it cannot be used at the Android end when playing in unity, and the socket is not connected
Click to expand...

Hi, Could I get more detail for your issue?
Are you trying the WebRTC sample on your Android device? How about editor?
Thanks.

In addition, this thread is for announcing the new release version.
Please make the new thread for your issue.

kazuki_unity729 · Dec 2, 2021

Released Unity Render Streaming package version v3.1.0-exp.2

We released a new version of Unity Render Streaming package.
Please refer to the changelog below. Feel free to feedback for our package. Thanks.

Added

Added audio streaming receiver.

Added the Multiplay scene into samples.

Added components for streaming.

MicrophoneStreamSender

AudioStreamReceiver

ScreenStreamSender

Added components for controlling user input.

InputSender

InputReceiver

Changed

Upgraded WebRTC package to 2.4.0-exp.4.

Upgraded Input System package to 1.0.2.

Renamed components.

CameraStreamer -> CameraStreamSender

ReceiveVideoViewer -> VideoStreamReceiver

WebCamStreamer -> WebCamStreamSender

AudioStreamer -> AudioStreamSender

InputSystemChannnelSender -> InputSender

InputSystemChannnelReceiver -> InputReceiver

Improved controlling user input from browsers.

Updated manual documents.

Fixed

Fixed resource leak of sender stream tracks.

kazuki_unity729 · Feb 4, 2022

Released WebRTC package version 2.4.0-exp.5

We announced the new version of the WebRTC package.
The changelog is below.

Added

Added the ability to execute pending native tasks manually from the main thread.

Added support for sending basic native collections (NativeArray<T>, NativeSlice<T> and NativeArray<T>.ReadOnly) to RTCDataChannel.

Added a property RTCRtpTransceiver.Mid.

Added an argument enableNativeLog for WebRTC.Initialize method.

Added "E2ELatency" scene in the sample.

Added a constuctor without arguments in AudioStreamTrack class.

Changed

Upgraded libwebrtc m92.

Changed compiler for native plugin on Windows (MSVC to Clang).

Changed to use OnAudioFilterRead method in MonoBehaviour for audio rendering.

Changed resizing texture dynamically on receiver side when using a property scaleResolutionDownBy in RTCRtpEncodingParameters class.

Changed the exception type ObjectDisposedException when accessing instance after call Dispose method.

Fixed

Fixed a crash bug where initializing video streaming on Unity Editor on Apple Silicon.

Fixed a crash bug where configuring OpenGL Core as a graphics API on Unity Editor on windows.

Fixed a bug that sending stereo audio produces mono clip on some peers.

Fixed a crach bug where using Full HD resolution with HWA enabled on macOS(Intel).

Fixed a crash bug where using Full HD resolution with HWA enabled on Windows.

Removed

Removed iOS Simulator (x86_64) support for make simple building process for iOS

kazuki_unity729 · Mar 1, 2022

Released Unity Render Streaming package version v3.1.0-exp.3

Hi, we are announcing the new version of the Unity Render Streaming package. Please let you try.

Changed

Upgraded the version of Input System package 1.3.0.

Upgraded the version of WebRTC package 2.4.0-exp.6.

Added microphone device option for streaming audio in the "Bidirectional" sample.

Integrate AR Foundation Sample (only use AR Foundation version 4.1.9, deprecated version 2.1)

Fixed

Fixed the worker thread is not closed when stopping the signaling client.

Fixed keyboard input sending only sends one key at a time from browser.

Fixed video flipped vertically when uging GLES/GLCore graphics API.

Fixed the crach when encoding the high resolution on macOS using HWA.

Fixed the gap of the input position between sender and receiver.

Fixed the unexpected behaviour of input after using Gyro sample.

kazuki_unity729 · May 18, 2022

Released WebRTC package version 2.4.0-exp.7

We announced the new version of the WebRTC package.
The changelog is below.

Added

Supported Unity 2021.3 LTS.

Added clang-format file.

Added CMakePresets.json file.

Supported NVIDIA decoder for H264 codec.

Added the codec selection option and video resolution option in the menu scene.

Changed

Moved CPU loads of the color conversion from the rendering thread to the worker thread.

Upgraded NVIDIA Codec SDK 11.0.

Fixed

Removed unnecessary dependencies of the native plugin on Linux.

Removed

Removed an argument EncoderType for WebRTC.Initialize method.

Removed WebRTC.HardwareEncoderSupport method.

kazuki_unity729 · Jul 12, 2022

Released WebRTC package version 2.4.0-exp.8

We announced the new version of the WebRTC package.
The changelog is below.

Added

Supported video encoding framerate control.

Added the items in Profiler Window to show the CPU loads of video encoding/decoding thread.

Add a new sample "ReplaceTrack" to demonstrate RTCRtpSender.ReplaceTrack method.

Changed

Changed arguments of RTCPeerConnection.AddTransceiver method to pass initial information of RTCRtpTransceiver when instanting it.

Fixed

Fixed the crash because of the old version of NVIDIA graphics driver.

Fixed the issue when garbage collected but the finalizer hasn't been called.

Improved performance of video streaming when using Vulkan graphics API.

This gif image is the new sample "ReplaceTrack" to demonstrating the feature as it is possible to replace the video track dynamically. Please try that and feedback.

And we have a next plan that we improve usability and flexability of Unity Render Streaming for developers.

kazuki_unity729 · Aug 8, 2022

Released WebRTC package hot fix version

We released the hotfix version 2.4.0-exp.9 and 2.4.0-exp.10.
We fixed critical issues belows, and so, please update the latest version 2.4.0-exp.10 if you are using old one.

2.4.0-exp.9
Fixed

Fix the crash bug on Android devices like pixel4a when using Vulkan.

2.4.0-exp.10
Fixed

Fix crash when streaming video between different platforms.

kazuki_unity729 · Oct 11, 2022

Released WebRTC package hot fix version

2.4.0-exp.11

Changed

Add ValidationExceptions.json to suppress warnings about the package validation.

Stop to use flip shader so that removing Resources folder.

Change parameter of VideoStraamTrack.CaptureStreamTrack method.

Fixed

Fix a bug when calling GetStats method multiple times at the same time.

kazuki_unity729 · Oct 11, 2022

Released Unity Render Streaming package version v3.1.0-exp.4

Hi, Thank you for waiting!
We are announcing the new version of the Unity Render Streaming package. This version contains big changes from the previous version. Especially, component design are changed a lot. Additionaly, many issues are fixed because of updating the WebRTC package.

You can select codec on the VideoStreamSender component.

VideoStreamSender component provides the options of Video Source Type.

Available to downscale video resolution dynamically on Unity Editor.

Stats component in sample can get more information of streaming.

Please try new features yourself! Thank you!

CHANGELOG

Added

Added APIs to select codec for audio/video streaming.

Added APIs to configure settings like bitrate for audio/video streaming.

Changed

Upgraded the version of Input System package 1.4.1.

Upgraded the version of WebRTC package 2.4.0-exp.11.

Merge features of three classes to VideoStreamSender.

MicrophoneStreamSender

AudioStreamReceiver

ScreenStreamSender

Improve user interface on the inspector window of components below.

VideoStreamSender

VideoStreamReceiver

AudioStreamSender

AudioStreamReceiver

Fixed

Reduce CPU loads of the rendering thread by updating WebRTC package.

Fixed the crash when video resolutions higher than 1080p with hardware encoding.

Removed

No longer support Unity 2019.4.

kazuki_unity729 · Nov 15, 2022

Released WebRTC package hot fix version

We released the first pre-release version of the WebRTC package to prepare the official release. The major version is changed because of following the rules of semantic versioning. The details of this version is below.
We will continue to improve for a while as pre-release version.

3.0.0-pre.1

Changed

Add AudioStreamTrack constructor to set AudioListener.

Change WebRTC.Initialize method to not throw exception when using OpenGL Core on Windows or macOS.

Remove dependency libc++.so.1 in Linux native plugin

Fixed

Fix SetRemoteDescription and SetLocalDescription method of RTCPeerConnection class to work correct when calling multiple at the same time.

kazuki_unity729 · Dec 12, 2022

Released new version of WebRTC package

3.0.0-pre.2

Changed

Upgrade libwebrtc M107.

Change that invoking initialization process automatically just after launching Unity Editor.

Obsolete WebRTC.Initialize and WebRTC.Initialize.

Fixed

Fix Profile results are displayed as different items on macOS.

Fix stopping at first received frame for seconds when using H264 codec.

Fix the crash when streaming video which resolution is small on macOS Apple Silicon.

Fix the freeze when using over WQHD (2560×1440) resolution on Ubuntu 20.04.

kazuki_unity729 · Feb 24, 2023

Released new version of Unity Render Streaming package

3.0.0-exp.6

This version contains an essential feature "Automatic Streaming" which makes possible to set up the Unity project for streaming video/audio automatically. Please try this feature with your Unity project!
Furthermore, there are some new features in this version. Please see below.

Changelog

Added

Support Automatic Streaming.

Support Render Streaming Wizard.

Support Command line arguments.

Add Render Streaming Settings in the Project Settings window.

Changed

Rename classes.

RenderStreaming > SignalingManager

Websocket is in default for signaling protocol instead of HTTP polling.

Changed a unit of the HTTP polling interval, second to millisecond.

Wizard Window

Project Settings Window

Command line arguments

kazuki_unity729 · Aug 8, 2023

Released new version of Unity Render Streaming package

3.0.0-exp.7

Long time no see! We released a new version of the package this week. This version contains some fixes for critical bugs, so I recommend you to update your package to the latest version.

Changelog

Added

Added configurable logger to enable users to customize logging for their environment.

Changed

Upgrade the version of WebRTC package 3.0.0-pre.6.

Add AudioStreamSender.loopback property.

Fixed

Fixed error on HTTP signaling when using short polling interval.

Fixed SignalingManager so that ICE server configurations aren't effected.

Added a workaround to fix an issue where InputField wasn't worked when entering characters from browsers or a Receiver scene of package sample.

kazuki_unity729 · Nov 6, 2023

Released new version of WebRTC package

3.0.0-pre.7

Added

Add Android x86_64 support experimentally.

Add an property .CanTrickleIceCandidates in RTCPeerConnection class.

Add an configurable logger in WebRTC class.

Support scale resolution for NVIDIA H.264 codec.

Support simulcast for NVIDIA H.264 codec.

Add an property SyncApplicationFramerate in RTCRtpSender class to improve latency of video streaming.

Changed

Upgrade libwebrtc M116.

Change Android API minimum level to 23 because the API level 22 is obsoleted on Unity2020.3 LTS.

Add RTCDataChannel.onError event.

Change a constructor of VideoStreamTrack to make simple to flip video texture.

Fixed

Fix an issue when setting a scale factor of video encoding to a fractional scaling ratio.

Fix an occasional crash issue when using hardware encoder with D3D11 graphic device.

Fix an crash when nvEncodeAPI64.dll is not found on Windows.

Optimize audio sampling when using mute.

Search Unity

Unity ID

Useful Searches

Official Unity Render Streaming Package Release Announcement

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies