Utilities Streaming from Mobile/XR Apps directly to Twitch

I've been experimenting with streaming directly from within a Unity app to a Twitch channel. WebRTC is the underlying technology to make this work. Up until this year, most streaming software would use the RTMP protocol to send a video feed to Twitch's ingestion servers. Amazon/Twitch just recently added the capability to stream to their streaming infrastructure using WebRTC. Since Unity is working on a neat WebRTC package for their own Render Streaming services, I experimented with using that package to establish a connection from a Unity camera to a Twitch Stream and I did get it to work.

On desktop platforms, there might not be much of a benefit to streaming directly from within a Unity application. People use broadcasting software like OBS anyway and usually want to add all sorts of overlays on top. But for mobile or even VR and upcoming AR devices, that capability might be more useful. Anyway, here's a short video of me interacting with a demo scene on an Oculus Quest and streaming directly to Twitch:

Twitter Link

I made this post mainly to show the initial prototype. There currently are too many stability issues with it. But if that ever improves in the future, I might consider releasing it as a utility on the Asset Store. So I'd be curious if there's any interest in a tool like that at all.

 

Interest here for exactly this!

 

Hey
thanks for the comment! So you're having VR in mind as well? Any particular platform you're targeting?

I probably should have been a bit clearer in the original post about what I mean by "stability issues". I've been periodically checking the state of this and things haven't improved much sadly. This is what I'm getting on different platforms:
- Windows: Does not work, no connection can be established due to the issue of not being able to specify an appropriate H.264 Profile Level (documented here)
- Android/Oculus Quest: Successfully able to stream using a specific version of the WebRTC package, as shown in the video above, but usually crashes after some time (also mentioned in the GitHub issue above). Not usable yet.
- Android/Galaxy S8: Works pretty reliably. Didn't encounter any issues so far.

 

Sorry for the slow reply, thank you for all the details!! Yes, looking at VR on Quest 2. Do you have more details on what "crashes after some time" means?

 

Meaning it properly hard-crashes the application, sometimes after 10 seconds, sometimes after 2 minutes. As mentioned, on Quest it's not usable at this time, but otherwise I'd be happy to offer a beta test.
I'm not sure to which degree this is an Oculus issue or an issue with the WebRTC package. I'd be curious if any of the other headsets (Vive, Pico, the upcoming Lynx HMD, ...) handle this better though.

 

I'm looking to do this on Quest 2 and Quest Pro specifically. Anything I can do to assist or move this forward?

 

I've just rechecked this and the good news is, things seem to have improved on Quest stability-wise. I'm not getting any crashes anymore (tested with a Quest Pro).

There's still some way to go to make this good from a usability standpoint. But if anyone wanted to beta-test at this point, I'd be happy to give them access.

 

Our biggest problem is accessing the android hardware encoder (MediaCodec) from Unity. Tried FFMpeg but its limited to software encoding and is too slow on Quest. It would appear that WebRTC has access to the hardware encoder which is why I'm looking for any pointers.

 

Also, let me know if you're up for any collaboration / testing. I have every HMD / config under the sun at my studio.

 

Yeah still keen @julienkay

 

@the_greenlig DM'd you