Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

WebRTC receiver jitter buffer?

Discussion in 'Unity Render Streaming' started by olavrv, Sep 27, 2022.

  1. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    We have some problems with latency; The latency is low when we connect to the WebRTC sender via web browser, but when the Unity application is loading a scene, it seems like the buffer of video stream on receiver builds up - and latency is increased from 60ms to 500ms

    We have monitored the jitter buffer on the web receiver, and it seems that this builds up as a result of the Unity application loading scenes.

    Is there a way to "reset" / clear the jitter buffer (if this is the cause of the latency)?

    Appreciate any help with this issue, thanks!
     
  2. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
  3. kazuki_unity729

    kazuki_unity729

    Unity Technologies

    Joined:
    Aug 2, 2018
    Posts:
    803
    Which version of the WebRTC package are you using?
     
  4. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    We are using 2.4.0-exp.10
     
  5. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    @kazuki_unity729 I would really appreciate it if you can help with this - because other than this issue everything works beautiful! :cool:
     
  6. kazuki_unity729

    kazuki_unity729

    Unity Technologies

    Joined:
    Aug 2, 2018
    Posts:
    803
    > We have monitored the jitter buffer on the web receiver, and it seems that this builds up as a result of the Unity application loading scenes.

    I assume that loading scenes would make a CPU spike and it makes delay of packing audio buffer.
    AudioStreamTrack makes the component internally and fetches audio buffer using OnAudioFilterRead method here. I have not been sure the detail of issue yet, but I guess there are isues around these code.
    https://github.com/Unity-Technologi.../Runtime/Scripts/AudioCustomFilter.cs#L46-L57
     
  7. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    We are not streaming audio and have not added this component. Will it help to add this component?
     
  8. kazuki_unity729

    kazuki_unity729

    Unity Technologies

    Joined:
    Aug 2, 2018
    Posts:
    803
    Sorry I misunderstood,
    I would like to know the easiest way to replicate your issue?
    Is your issue reproduced by repeating loading scene?
     
  9. olavrv

    olavrv

    Joined:
    May 26, 2015
    Posts:
    515
    When the Unity application that is sending video stream is loading a scene, the WebRTC receiver builds up a latency in video stream. The latency is higher the longer the loading time is on the sender. It seems like it is the "freezing" of the sender application that is causing this latency, since we tried right clicking in unity editor on sender app, and this had same result.
     
  10. kazuki_unity729

    kazuki_unity729

    Unity Technologies

    Joined:
    Aug 2, 2018
    Posts:
    803
    Thank you for the detail, I will try to replicate the issue.
     
    SHO4X likes this.