Bug WebRTC stream sending continuous black image

I have a game running on Meta Quest 2/Pro and a companion app running on Android smartphone for video chat. Signaling works fine and streaming is established between devices. The stream from Android can be seen in VR, as well as the capture video taken on the Quest. But on the Android side the received stream appears as a black image.

I use RTCPeerConnection.GetStats to check the streams for sent/received bytes and packets. The amount of data is very low for the stream appearing as a black image. When I run the game on Windows in the Unity Editor, the stream properly shows as the captured video seen inside VR and the data amount is 2 times higher for bytes and 3 times higher for packets.

My guess is that there is some problem before or during video encoding on the Quest. The cause for this might be linked to lack of resources (memory, available threads, I dont know). When I use the same code from my game in a small sample scene, streaming works (but since nothing is happening in the scene the same image repeats and the sent data amount is also quite low).

I'm not getting any error messages. What would be helpful if the WebRTC encoder could communicate some log info and errors, if there are any occuring of course.

I previously asked for help regarding this problem in the WebRTC FAQ thread as well.

 

I forgot to mention this, but it seemingly occurs with all codecs. I tried various filtering, also forcing MediaCodec implementations over internal (which strangely didn't work), but it occurs regardless of codec. Examples are VP8, VP9, AV1, H264.

It works when using my code in a small sample scene.

Yes, that works. I tried that and reported back in this answer. Note that I said that "the last step fails" which was a conclusion I drew from no video showing but I wasn't looking at the logs at that moment. I later edited to report in the same post that the steps indeed complete; but that just means there must be a different reason for the stream appearing as a black image.

Note that in the next post I report that Quest 2 to Quest Pro also doesn't work, in this case both show black image remote streams. So it has nothing to do with if its server/host, offer or answer.

The condition for this bug occuring is: (a) within the context of my game and (b) running on Quest. When run in Windows, or using the same WebRTC integration in a small sample scene, the bug does not occur.

 

I made a sample scene for gtk2k, but the bug didn't occur in it. He also confirmed to me that it ran fine on his end. He also made an example of his own that works and I have talked to another user who has it working on his Quest.

So I cannot replicate the issue outside my own game. Not sure what you can find with your own test scenes but thanks for checking anyway.