Question "Stream" Cameraview of Player 1 to RawImage that Player 2 sees

I'm working on a Project where one Player is in VR while the other Player has a UI infront of him with Buttons that change things for the VR Player.
What I'm trying to archive is that the "UI Player" is able to see what the VR Player sees. I did this by adding a child camera to the VR Camera and sending the renderTexture from that child to a RawImage on my Canvas.
I managed to get that set up but it only works localy.
Whenever I try with a friend in Multiplayer (I'm using Pun) he either can't even see the RawImage or it is just a picture and not moving.
I added a photon view to the VR Player and a Photon transform view to the Camera.

 

This is about the only PUN-specific thing I could find. Not sure how helpful it is though.
https://forum.photonengine.com/discussion/14586/streaming-video-frames-with-rpc