Stereo frustum corners seem incorrect

Hi,

I'm trying to pass frustum corners to a post-render shader (Graphics.Blit) so I can reconstruct the world position of each fragment using the depth buffer. I'm using the new Camera.CalculateFrustumCorners function. This works fine when rendering normally, however when doing stereo rendering (single-pass) the result is not as expected. I'm able to "sort of" reconstruct the world position, but it is obviously not entirely correct, and goes way off when the camera is rolled to the side.
What I'm wondering is: Are there any working examples of reconstructing world position in VR/stereo? Everything I've been able to find online is either mono rendered or people having the same issue with no solution.

Here's my current (non-working) setup:
1. Calculate frustum corners using Camera.CalculateFrustumCorners, for both eyes
2. Transform to world space using camera.transform.TransformVector + normalizing
3. Pass as vector array shader uniform.
4. Calculate interpolated view ray in the vertex shader using vertex coordinates to calculate the index and unity_StereoEyeIndex to determine which array to sample (I've confirmed this is working as intended)
6. Construct world position by sampling depth buffer and multiplying with the interpolated ray + _WorldSpaceCameraPos

Here's an image showing the resulting world position. As you can see from the green lines (y-position), they are way off in each eye:


Any help is appreciated, thanks.

 

// world space frustum, for camera view space, remove the view to world multiplications
// cameraFrustumCorners is 8 Vector4's, using either mono eye for the first four vectors, or for stereo the first four vectors are the left eye, the second four are the right eye. Can index into array in shader using stereo eye index * 4 + index.

Code (CSharp):

1. private void CalculateFrustumCorners(Camera camera)
2. {
3.     float farClipPlane = Mathf.Min(100000.0f, camera.farClipPlane);
4.     camera.CalculateFrustumCorners(camera.rect, farClipPlane, (camera.stereoEnabled ? Camera.MonoOrStereoscopicEye.Left : Camera.MonoOrStereoscopicEye.Mono), cameraFrustumCornersTemp);
5.     cameraFrustumCornersTemp[0].z = -cameraFrustumCornersTemp[0].z;
6.     cameraFrustumCornersTemp[1].z = -cameraFrustumCornersTemp[1].z;
7.     cameraFrustumCornersTemp[2].z = -cameraFrustumCornersTemp[2].z;
8.     cameraFrustumCornersTemp[3].z = -cameraFrustumCornersTemp[3].z;
9.     Matrix4x4 leftViewToWorld = (camera.stereoEnabled ? camera.GetStereoViewMatrix(Camera.StereoscopicEye.Left).inverse : camera.cameraToWorldMatrix);
10.     cameraFrustumCorners[0] = leftViewToWorld * cameraFrustumCornersTemp[0]; // bottom left
11.     cameraFrustumCorners[1] = leftViewToWorld * cameraFrustumCornersTemp[1]; // top left
12.     cameraFrustumCorners[2] = leftViewToWorld * cameraFrustumCornersTemp[2]; // top right
13.     cameraFrustumCorners[3] = leftViewToWorld * cameraFrustumCornersTemp[3]; // bottom right
14.     camera.CalculateFrustumCorners(camera.rect, farClipPlane, Camera.MonoOrStereoscopicEye.Right, cameraFrustumCornersTemp);
15.     cameraFrustumCornersTemp[0].z = -cameraFrustumCornersTemp[0].z;
16.     cameraFrustumCornersTemp[1].z = -cameraFrustumCornersTemp[1].z;
17.     cameraFrustumCornersTemp[2].z = -cameraFrustumCornersTemp[2].z;
18.     cameraFrustumCornersTemp[3].z = -cameraFrustumCornersTemp[3].z;
19.     Matrix4x4 rightViewToWorld = camera.GetStereoViewMatrix(Camera.StereoscopicEye.Right).inverse;
20.     cameraFrustumCorners[4] = rightViewToWorld * cameraFrustumCornersTemp[0];
21.     cameraFrustumCorners[5] = rightViewToWorld * cameraFrustumCornersTemp[1];
22.     cameraFrustumCorners[6] = rightViewToWorld * cameraFrustumCornersTemp[2];
23.     cameraFrustumCorners[7] = rightViewToWorld * cameraFrustumCornersTemp[3];
24. }