Which eye is the worldToCameraMatrix set to in VR/AR?

In a nutshell the question is which eye sets the cameras worldToCameraMatrix, used by many functions presumably like ViewportToWorld() for example. And as a follow up, how do functions like ViewportToWorld() operate in stereo rendering applications? The intuitive way for this to work, would be to update the worldToCameraMatrix when the rendering for each eye is performed, but I just want to verify that this is the case.

I'm creating an application where I need explicit control over the view matrices and set them using SetStereoViewMatrix().

 

I'm not sure I can answer all your questions, but if you use Single Pass Multiview, Unity internally passes an array of two matrices (left and right eye) to the shaders, and uses an eye index to select the matrix to use when rendering the scene. I'm not sure how this works if you don't use single pass rendering, I have never tried that.