Get left/right camera's textures offsets

Hi guys!

As we all know, VR engines makes some "shifts" of eye's cameras to achive stereoscopic effect. In very high level of abstractions it's means that the same image looks slightly shifted to the right in the left camera and to the left in the right camera.
In my case, I would like to know the exact value of that shift in texture's pixels units.
I work on HTC Vive with display size of 1512x1680 per eye, as well as rendered textures by default.
Roughly measured by direct comparing of 2 saved RenderTexuture, that shift is about 40-60px.
How to get that value programmatically?

There are several open discussion about left/right camera positions which uses XR.InputTracking.GetLocalPosition(Rotation) and cameraToWolrdMatrix. But I don't really understand is it right way to get desired value of shift in pixels or should I use some different approach?

Providing the visual example:

The black boxes on this images are located on the same positions (in pixel coordinates of image 1512x1680), but they covers different areas of the same picture because VR engine shifted that pictures to left and right to get stereoscopic effect.

 

A shift is perhaps the wrong way to think about it. It's two camera views that are separated by ~65 cm (depending on your IPD settings). Additionally most VR devices have skewed projections such that "straight forward" isn't at the center of the rendered image.

The result of this is the offset between the eyes isn't some constant value. The difference in position changes depending on how far away it is ... that's how stereoscopic version works. You would need to pick a position in 3d first and determine the "shift" from that by calculating the on screen positions.

The XR position and rotation could get you part of the way, but what you're really going to need is the view matrix and projection matrix for both eyes.

These two functions get you the necessary matrices.
https://docs.unity3d.com/ScriptReference/Camera.GetStereoViewMatrix.html
https://docs.unity3d.com/ScriptReference/Camera.GetStereoNonJitteredProjectionMatrix.html
https://docs.unity3d.com/ScriptReference/GL.GetGPUProjectionMatrix.html

And you can use the functions from XRSettings to get the resolution in script if you aren't already.
https://docs.unity3d.com/ScriptReference/XR.XRSettings.html

Then you need to take a world space position and multiply it by the view matrix, then the projection matrix, then apply the perspective divide to get the normalized screen position, and multiply that by the resolution.

Psuedo code version:
Vector4 worldPos = trans.position;
worldPos.w = 1.0f;
Vector4 viewPos = viewMatrix * worldPos;
Vector4 clipPos = projMatrix * viewPos;
Vector2 screenPos = Vector2(clipPos.x / clipPos.w, clipPos.y / clipPos.w);
Vector2 pixelPos = Vector2(screenPos.x * eyeResWidth, screenPos.y, eyeResHeight);

 

Thanks bgolus! It's sounds little bit complicated from my point of view, I expected more simple way on other level of abstraction. But anyway, thanks again, I'll try to sort it out somehow )

 

Hi @bgolus. Thanks for your recommendation. But I have some difficulties with this task. Could you help me?
I did what you said.

Code (CSharp):

1. struct appdata
2.             {
3.                 float4 vertex : POSITION;
4.                 float2 uv : TEXCOORD0;
5.             };
6.  
7.             struct v2f
8.             {
9.                 float4 vertex : SV_POSITION;
10.                 float4 screenPos: TEXCOORD1;
11.             };
12.  
13.             //simple vertex shader
14.             v2f vert (appdata v)
15.             {
16.                 v2f o;
17.  
18.                 o.vertex = UnityObjectToClipPos(v.vertex);
19.                 o.screenPos = ComputeScreenPos(o.vertex);
20.                 return o;
21.             }
22.  
23.             //world space fragment shader
24.             fixed4 frag (v2f i) : SV_Target
25.             {
26.                 i.screenPos /= i.screenPos.w;
27.  
28.                 //read none linear depth texture, accounting for
29.                 float d = SAMPLE_DEPTH_TEXTURE(_CameraDepthTexture,                          UnityStereoTransformScreenSpaceTex(i.screenPos)); // non-linear Z
30.  
31.                 //pick one of the passed in projection/view matrices based on stereo eye selection
32.                 float4x4 proj, eyeToWorld, proji, eyeToWorldi;
33.  
34.                 if (unity_StereoEyeIndex == 0)
35.                 {
36.                     proj = _LeftViewFromScreen;
37.                     eyeToWorld = _LeftWorldFromView;
38.                     proji =_RightScreenFromView;
39.                     eyeToWorldi = _RightViewFromWorld;
40.                 }
41.  
42.                 else
43.                 {
44.                     proj = _RightViewFromScreen;
45.                     eyeToWorld = _RightWorldFromView;
46.                     proji = _LeftScreenFromView;
47.                     eyeToWorldi = _LeftViewFromWorld;
48.                 }
49.                  
50.                 float2 uvClip = i.screenPos.xy ;
51.                 float4 clipPos = float4(uvClip, d, 1.0);
52.  
53.                 float4 viewPos = mul(proj, clipPos); // inverse projection by clip position
54.                 viewPos /= viewPos.w; // perspective division
55.                 float4 worldPos = mul(eyeToWorld, viewPos);
56.                 float4 worldPoss = float4(worldPos.xyz,1);
57.          
58.                 float4 viewPosi = mul(eyeToWorldi, worldPoss);
59.                 float4 clipPosi = mul(proji,viewPosi);
60.  
61.                 float res = abs(-clipPos.x+ clipPosi.x/clipPosi.w);
62.  
63.                 return float4(res,0,0,0);

But I can't get the correct shift. What am I doing wrong?

 

Your question is a little different than what this post is about, but this might help you:
https://github.com/chriscummings100...ldSpacePostEffect/WorldSpacePostEffect.shader