Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

How to send to RTMP endpoint

Discussion in 'Unity Render Streaming' started by ECHO-425, Jan 25, 2023.

  1. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    We're trying to send the captured audio/video from Unity Render Streamer through to an rtmp endpoint (YouTube, Twitch, etc.). However, it's been tough finding anything that doesn't require additional resources that can't be compiled with our Unity build.

    Our previous solution was to use 3 different plugins to get a/v our through NDI and stream with OBS, but the plugins all have different support levels and now have versions out of sync. We've also tried several Asset Store packages, but they either have too much processor overhead or have the audio & video out of sync.

    So, we're hoping to find a solution via Unity Render Streamer. Anyone know of any plugins or solutions out there, or have a lead on how we might be able to say, attach the Unity Renderer Streamer streams to the ffmpeg dll to send to an RTMP endpoint...?

    Thanks in advance for any time + thoughts!
     
    miracletechteam1 likes this.
  2. winlin_srs

    winlin_srs

    Joined:
    Jan 10, 2023
    Posts:
    4
  3. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    I appreciate the share @winlin_srs and I saw that, but it looks like you need to have the srs webapp still running + in docker as opposed to having the solution built-in to a unity build? I was also having a hard time finding the rtmp conversion info/example to test, though I saw it briefly mentioned in the original post...

    If you can give me any info + point me in the right direction to address those two issues, I'd love to pursue this further!
     
  4. winlin_srs

    winlin_srs

    Joined:
    Jan 10, 2023
    Posts:
    4
  5. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    Unfortunately, the SRS solution won't work with us since it requires an external resource/server.

    If anyone has any further insights into how we can solve this, please do let me know. We're looking to hire a freelancer and would be grateful for any insights the help us minimize the time we have that freelancer working on a solution from scratch ~
     
  6. julienkay

    julienkay

    Joined:
    Nov 12, 2013
    Posts:
    167
    I can give some insight here, not sure if it's going to be helpful to you though.

    You mentioned platforms like YouTube and Twitch. These two platforms both have ingestion endpoints for WebRTC streams nowadays. In both cases these are not officially documented though. But both platforms use WebRTC as an ingestion protocol to allow users to stream directly from a browser. (E.g. https://youtube.com/webcam)

    I've made some experiments to stream to Twitch directly using WebRTC and posted the results here. My verdict was, that it's not yet robust enough. I got it to work on Android pretty reliably with a specific version of the WebRTC package. But on Windows I ran into the issue that Twitch requires a specific Profile Level of the h264 codec to be used and Unity's WebRTC package doesn't give you this fine control on which Profile Level is being used (at least as far as I can tell.) I've created an issue for this here on GitHub:

    But I do think for streaming directly from an app to the ingestion platform without any server in between, WebRTC might be the best approach as opposed to RTMP.
     
  7. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    This is really great info, thanks @julienkay! I'll try this out as well, but also take your word for it that it might not be ready.
     
  8. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    Ahh I see, so this solution still requires spinning up a web application to do the conversion @julienkay?
     
  9. julienkay

    julienkay

    Joined:
    Nov 12, 2013
    Posts:
    167
    No web app involved. Goes straight from app to streaming platform. The youtube link was just to show they have their WebRTC backend working in production.
     
  10. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    So you just put in their (new) webrtc endpoints into the existing Unity WebRTC plugin using Unity Render Streaming and it worked? From the video I thought you made a custom solution or that it required a webapp/server :)
     
  11. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    I guess I'm not clear where you get those endpoints since the article you linked to talks about running an AWS webserver
     
  12. julienkay

    julienkay

    Joined:
    Nov 12, 2013
    Posts:
    167
    For Twitch, I used https://ingest.twitch.tv/ingests which returns the available endpoints in JSON format (This link if you want see the endpoints for your region in the browser). Then do the typical WebRTC Offer/Answer handshake with that endpoint.

    As I said, the API endpoints are not really officially documented. And since signaling is not part of WebRTC, the platforms both implement their own HTTP based signaling process. I had to reverse engineer that part using the available browser apps https://youtube.com/webcam for YouTube and https://stream.ivs.rocks/ for Twitch (which uses the same backend as the AWS IVS solution). YouTubes signaling process seemed a bit more complex last time I've looked at it, so I didn't pursue it further at the time, but for Twitch it was relatively straightforward.
     
    Last edited: Jun 25, 2023
    miracletechteam1 likes this.
  13. julienkay

    julienkay

    Joined:
    Nov 12, 2013
    Posts:
    167
    Sorry, I missed that part of your question. I actually don't use the Render Streaming package at all, I use the WebRTC package directly and implement the signaling handshake with the streaming platform as outlined above.
     
  14. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    Interesting... but then how are you capturing the audio/video being sent out?
     
  15. ECHO-425

    ECHO-425

    Joined:
    Feb 2, 2018
    Posts:
    19
    Sorry for all the questions, but I'm unclear as to how you can send to one of the rtmp endpoints listed for twitch using webrtc? Will Unity's WebRTC actually connect/handshake with an rtmp endpoint?
     
  16. julienkay

    julienkay

    Joined:
    Nov 12, 2013
    Posts:
    167
    That's part of the WebRTC package more than Render Streaming. The WebRTC package lets you capture any camera, which is what the Render Streaming package is also using as it's built on top of the WebRTC package.

    No worries. Yeah, that's also nowhere documented, but every one of those RTMP endpoints has a matching endpoint for the WebRTC signaling. I.e.
    rtmp://sfo.contribute.live-video.net/app/{stream_key}
    has a matching endpoint
    https://sfo.webrtc.live-video.net/4443/offer/
     
    miracletechteam1 likes this.