Improving render streaming latency from PC > Quest 2

I am very new to WebRTC and am finding it very cool. I have a project idea in mind that I need super low latency video stream from one Unity project on PC into another Unity project running on a Quest 2. I've managed to build the examples and I can stream a render texture from the PC game into the Quest and see the video in VR !

The only issue I am having is the latency is ~ 1 second:

It appears to be working okay but I really need less than 0.25 second latency for my project idea to work.

Is the render streaming codec lossless? Is there a lossy option? Could the latency be minimized if the Quest somehow dropped old frames and only rendered the most recent frame?

If I use PC for both projects then the video appears perfectly synced in real time, and even over a local network in web browsers the video appears instant.

It's only in the Quest 2 I notice the latency becomes very high and sometimes video playback is a little choppy.

Is there anything i can do to improve render streaming performance or use faster decoding? I tried reducing the stream resolution to 800x600 but the frame rates in the Quest weren't affected and remained kinda slow.

I have the Quest connected to gigabit WIFI and the PC directly connected to the router, I don't think the network is causing the latency.

I know the Quest 2 can stream realtime HD full screen VR over WIFI, I sometime play VR games from my PC over WIFI directly on the Quest and it works phenomenally well.

But Unity's render streaming is really struggling to keep up.

Is there a faster decoder or more efficient way to stream video on a Quest/Android?

 

I just tested by explicitly enabling hardware encoding by setting:

WebRTC.WebRTC.Initialize(EncoderType.Hardware);

.

Then when I tried to connect the Receiver the Broadcast log showed a error:
[impolite-Unity.RenderStreaming.PeerConnection] Failed to set remote offer sdp: Failed to set remote video description send parameters for m-section with mid='0'.

Does this mean the hardware codec is not available? Could it be the reason why the Quest video stream looks so delayed? Can I be sure if h264 is being used instead of v8?

When using the WebBrowserInput demo it works with and without Hardware Encoding enabled, And I do believe I can see a slight difference in frame rate/quality when switching between them. My hunch is that the codecs are changing and h264 is working for WebBrowser demo, but I am unsure how to enable h264 codec for the Broadcast and Receiver demo - it always produces the above error.

I am using a 2080TI GPU, I believe it supports HW h264.