Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct here to familiarize yourself with the rules and how to post constructively.

  2. Unity 2022.1 is now available as the latest Tech release.
    Dismiss Notice
  3. Improve your project's performance with our new guide on profiling in Unity.
    Dismiss Notice
NEW FORUM USER NOTICE Unity 2022.1 is now available New guide on profiling in Unity

Unity Unity Render Streaming Status Update (Q4 2021)

Discussion in 'Unity Render Streaming' started by kazuki_unity729, Dec 23, 2021.

  1. kazuki_unity729

    kazuki_unity729

    Unity Technologies

    Joined:
    Aug 2, 2018
    Posts:
    418
    Hello everyone. This thread is for sharing with you about the development status of Unity Render Streaming, and the plans for 2022. If you have any questions about the 2022 project initiatives, please send them to this thread.

    Many users have asked us about our product roadmap. I will try to answer in this thread as much as possible. If you have individual questions, please post another thread on the Unity Forum. If you have a bug report, please send it to GitHub Issues.

    Looking back on 2021
    In 2021, we mainly dealt with the following contents.
    • Mobile platform support (iOS, Android)
    • Integration with Input System
    • Audio stream support
    We have released 2.4.0-exp.4 for the WebRTC package, and 3.1.0-exp.2 for the Unity Render Streaming package. For details on the update, see the changelog of each package.
    Contributors
    We have received various feedback from many developers. We would like to thank all the developers who have a proven track record of pull requests being brought upstream in 2021.
    WebGL platform support
    At this moment, WebRTC package does not officially support the WebGL platform, but developers of the community have been developed.
    We pay tribute to all the developers who have helped with WebGL support.
    Many other people have sent us detailed bug reports and checked for fixes. Thank you again.

    Plan for 2022
    In 2022, we plan to focus on improving stability and performance. In addition, there are two additional functions: hardware acceleration support for video codecs and metadata support.

    If you have any other requests, please reply to this thread.

    Stability
    We have received many bug reports from developers and are working on them. Regarding the behavior of Unity2021.2 on Apple Silicon, we have confirmed that it is currently unstable. We will deal with it as a priority.
    Also, using hardware acceleration encoder issues haven't been addressed since they were first reported in 2020. We will raise the priority and respond. The related tickets are listed below.
    Performance measurement and improvement
    We have received reports of delays in video streams compared to Google Chrome. We will continue to investigate and respond to problems.
    We continue to improve audio delays with user feedback.
    Hardware Acceleration Support
    Many users have requested hardware acceleration video codecs.
    We have also received requests for hardware decoder support.
    In addition, we have received requests for APIs for users to create their own codecs.
    We discussed about the topic. I hope we can share our plan in the near future.

    Metadata Support
    Encoded Transforms API is to add metadata to Video Frame. The draft is published by W3C.
    Content related to metadata is also discussed in the Unity Forum. It is used to send information that needs to be synchronized with the video frame. For example, the timestamp on the stream sender and the location information of the camera.
    Other Features
    We plan to add the following in Unity Render Streaming.

    Zero Configuration
    To stream a Unity scene which the developer created, component settings are required. Simply add a package and you'll be able to stream instantly without any configuration.

    Command line options
    After creating the executable file, it was necessary to rebuild to change the settings such as signaling server URL and port. Providing command line options simplifies configuration when deploying the server.

    Controlling the quality and resolution of the video stream
    The video streaming couldn't freely choose the codec and bandwidth. Prepare an API that allows fine control.

    Frequently Asked Question
    We have received the following questions from users.
    • Hololens2 support
    • WebGL support
    • Validated package
    • Various hardware acceleration of the codec
    • VR / AR platform support
    Hololens2 support
    Inquiry about the support of the Hololens2 has been asked from many users. It is discussed in this Issue.
    Mentioned in this issue, we are currently lowering our priority due to the high cost of supporting Windows ARM64.

    WebGL support
    Community-based development. You can check the operation by checking out the following pull request source branch.
    Landing on the upstream is still difficult at the moment, but we will continue to work towards future support.

    Verified Package
    We have received questions about when to.
    Unity package is promoted to a validated package, it guarantees a 3-year long term support (LTS). Please see the following materials for the package life cycle.
    The two packages are based on the package development cycle and are currently in the experimental stage. Development team aims to move to a validated package, but needs to clear challenges for long-term support and is currently working on it.

    Support for various hardware codecs
    The following hardware codecs are candidates, but the support time is not decided.
    • VAAPI (Support for AMD GPU) Support
    • Windows Media Foundation Codecs
    • Intel Media SDK
    VR / AR platform
    VR/AR platform is not supported yet, but developers have already reported it. Latency issues have been reported with Oculus Quest 2.
    WebRTC package includes an AR Foundation sample, so please give it a try and give us feedback.
    In Closing
    We hope this allows you to get a better understanding of how we are approaching development. Also, We wait for your feedback in this thread.

    Last, best wishes for the holidays and New Year 2022!
     
    kazuki_unity729, Dec 23, 2021
    #1
    Baggers_ likes this.
unityunity