Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official Unity Render Streaming Status Update (Q4 2022)

Discussion in 'Unity Render Streaming' started by kazuki_unity729, Dec 20, 2022.

  1. kazuki_unity729

    kazuki_unity729

    Unity Technologies

    Joined:
    Aug 2, 2018
    Posts:
    803
    Contributors in 2022
    First, We posted a look back at last year 2021 and our plans for 2022 in December 2021 here. In this year as well as last year, we have received various feedback from many developers. We would like to thank all the developers in the below list who have made pull requests to our products in 2022. Thank you developers!
    A look back at 2022
    Looking back at 2022, the notable work of our team includes as follows.
    • Hardware encoder performance improvements
    • Add NvCodec hardware decoder
    • UX improvements for streaming components
    • Integration of WebRTC with the Unity profiler
    Many other modifications have been made to the two packages WebRTC and Unity Render Streaming. Most notably, the Unity Render Streaming 3.1.0-exp.4 version has been upgraded to allow real-time control of streaming in the Unity Editor component. Please see this post for more information about this version change.

    A case study
    At Unity Sync which is an online event hosted by Unity Japan in October 2022, a case study of Unity Render Streaming was presented by the third party developer. The video is in Japanese, but once you watch it you will get many ideas from it.

    https://learning.unity3d.jp/8908/

    Plan for 2023
    For the WebRTC package, we are beginning preparations to release it as an official package. We have already updated the major version of the package and released it as 3.0.0-pre.1. During the pre-release period, we will focus primarily on improving API deficiencies, bug fixes, and performance improvements. These issues can be viewed on GitHub Issues, and new ones can be added.

    We also plan to address the following issues that could not be started in 2022.

    Hardware Acceleration Support
    We will continue to provide support for hardware codecs. First, there is room for performance improvements in existing video codec implementations. In particular, the CPU load on video decoders for mobile platforms can be significantly improved.
    In addition, support for codes of NvCodec (AV1, VP8, VP9), as well as Intel OneVPL, AMD Advanced Media Framework.
    Metadata Support
    The Encoded Transforms API can be used to add metadata to streaming video (example). This can be used to synchronize video and input. This feature is increasingly requested by developers.
    Platform Support
    We have received reports from users that Unity Render Streaming does not work on some Linux distributions, possibly because QA has not been performed on Red Hat-Based Linux Distributions. We are planning to support Rocky Linux since Cent OS is already out of support.
    Next, I would like to discuss our plans for the Unity Render Streaming package. Currently, this package has some missing features and stability issues, so it will continue to be developed as an experimental package in 2023.

    We plan to support the following issues.

    Configuration Window and Wizard
    The current version of Unity Render Streaming is available by setting up several dedicated components in a scene created by developers, but this process is a bit complicated and inconvenient. Therefore, the next version will provide the ability to configure scenes automatically. Developers can make streaming applications with minimal configuration in the settings window.
    Additionally, we will provide a wizard window which will be provided to suggest the best project settings for using Unity Render Streaming.

    Command Line Configuration Support
    It is inconvenient for users to have to rebuild the application or prepare a UI for changing settings. Therefore, we provide the ability to set configuration items such as signaling server URLs from the command line at application runtime.

    Integration with SFU
    SFU is a mechanism to deliver streaming to more users, reducing the processing load on the sender side due to the number of viewers (reference).
    Providing a way for Unity Render Streaming and SFU to work together to address more use cases.

    Sample for Docker Container
    A user asks about an efficient and scalable way to use the Unity runtime for streaming applications.

    We support the creation of a Docker container to address this issue.

    Extensibility of signaling logic
    Provide APIs to facilitate user customization of signaling.

    Create package for web client modules
    We plan to make a package as reusable modules for creating web clients.

    Frequently Asked Questions
    Many users have asked the following questions.
    • Release date of the officially supported version
    • VR/AR support
    Release date of the officially supported version
    The WebRTC package is now stable and is being prepared for official release by the end of 2023. The timing for official packaging of Unity Render Streaming has not yet been determined.
    VR/AR Support
    We have received many bugs with specific VR/AR headsets.
    Issues on certain Android SoCs may take longer to be addressed.

    In Closing
    If you have any questions about our products, please feel free to reply to this thread or create a new thread in the Forum. There are already many threads discussing various topics, so if you search in the forum, you may possibly find what you are looking for.

    So I look forward to seeing you all here again next year. Have a great holiday!
     
    M_MG_S and gtk2k like this.