[SOLVED] VR Portals - Wrong X axis parallax with stereoscopic renderTexture on Oculus Go

Hello community.

I'm trying to adapt the brackey portal tutorial to VR, and I'm cloth to it but I still got deformations when I turn my head left or right in front of the portal.
I also got deformations when I look forward standing on the left or right of the portal.
It seems to be a parallax problem that I can't grasp.


When I'm turning my head, you can see the green pillars of the portal structure appearing behind the grey one.

Here are some explanations of the set up of my scene.

• For rendering a correct image inside the portal in VR, I cloned the LeftEyeAnchor & RighEyeAnchor in an other gameobject who hold them.
• I'm capturing the original LeftEyeAnchor & RightEyeAnchor rotational datas for applying them to my clones
• I render each of this clones view on one plane who have the same position in space as the other, creating a stereoscopic renderTexture.

Here is the script I attached to my clones holder (the game object who holds my two cloned cameras)

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.UI;
5.  
6. public class PortalCamera : MonoBehaviour {
7.  
8.     public Transform player;
9.     public GameObject headtracker_LEFT;
10.     public GameObject headtracker_RIGHT;
11.     public GameObject CameraB_RIGHT;
12.     public GameObject CameraB_LEFT;
13.     public Vector3 CameraB_Pos;
14.  
15.     // LateUpdate is called once after each frame
16.     void LateUpdate()
17.     {
18.         //Parenting B Cameras to player movements
19.         transform.position = player.transform.TransformPoint(CameraB_Pos.x, PlayerScript.globalCameraHeight, CameraB_Pos.z);
20.  
21.         //Parenting B Cameras to OVR left and right cameras roation
22.         //Left
23.         CameraB_LEFT.transform.eulerAngles = new Vector3(headtracker_LEFT.transform.eulerAngles.x, headtracker_LEFT.transform.eulerAngles.y, headtracker_LEFT.transform.eulerAngles.z);
24.  
25.         //Right
26.         CameraB_RIGHT.transform.eulerAngles = new Vector3(headtracker_RIGHT.transform.eulerAngles.x, headtracker_RIGHT.transform.eulerAngles.y, headtracker_RIGHT.transform.eulerAngles.z);
27.     }
28.  
29. }
30.  

I also share my unity project here (enable the Fly Control script attached to the player for moving in the scene in unity, and don't forget disabled it befor building the app for viewing it in the GO )
fromsmash.com/govrportal


Does anyone have any idea?
Thanks!

 

Hey @Untoldecay ,
Tried downloading your project and made sure all the settings were the same as what you told me over on my thread and this is what i saw when I had ran it in editor using the Oculus Rift Headest. Any idea why this isnt working for me?

==

 

I have no clues, I'm pretty new to unity so I don't have a large experience but I'll try to help.
What are the white areas on each side of the left eye? A bug I suppose but I'm not sure.
Could you take an other screenshot of the scene but at some distance of the portal?

In few days when I'll have some time and when I'll solve all my portal related problems, I'll post my process and all the ressources I found helpful, but here are some leads to begin with.

• All the position / rotation tracking who are replicated from the controller(s) or the headset should be placed in the Application.onBeforeRender() function for preventing any lags or delays. I found a german article illustrating it here
• The portal image distorsion have something to do with the cameras projection matrix, on which I understand the concept, but not really their integration. I found some code exemples here
• Here is a great article about portals, but more in depth than the brackey one ( keep in mind that for VR and stereo you have to use sets of 2 cameras instead of 1 )

I hope this will help for now.

 

My problem is solved

I think the parallax problems was due to 2 things

• Tracking the headset or the remote have to be fast enough to prevent lags, and LateUpdate is not fast enough. I should have used Application.onBeforeRender()
• Parallax problems due to stereoscopic view and cameras projection matrix. I needed to clip the renderTexture plane from the concerned camera views ( It was clearly over my head, and it still is ).

There are other points who differs from the Brackey tutorial or the on ( more in depth ) made by Tom Hulton

• It could be obvious for the majority, but It came to me after I began the tutorials : VR needs stereoscopic render for create depth, so I had to 2 cameras instead of one for each place where a camera was needed. I just copied the LeftEyeAnchor and RightEyeAnchor cameras for keeping the same settings and used the layer system + culling mask for displaying different content to each eyes.
• I you need to cameras, you need 2 overlapping renderTextures for this to work
• ! You can move the cameras parent position using the player position but you have to rotate each cameras apart. For getting my player cameras rotation &/or position, I placed a dummy camera ( not rendering anything ) as a tracker inside my LeftEyeAnchor & RightEyeAnchor.
• By default, disable the MeshRenderer of the destination portal if not needed, and enable it once the teleportation happened.
  
• And overall, I'm not sure but you should use MultiPass Stereo Rendering Method ( in the Player settings of your project ) for this to word.

In the previous version of my code I was calculating the position of player's camera clones directly, using only transform.TransformPoint(), it seemed shorter and better, but now I came back to calculate it relative to the portal ( as Brackey or Tom Hulton are doing ).

So here is the code on my cameras holders ( the gameobject containing my cloned cameras )

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using UnityEngine.UI;
5.  
6. public class RelativePortalCamera : MonoBehaviour {
7.  
8.     //Referring to the dummy cameras placed inside LeftEyeAnchor & RightEyeAnchor used as position / rotation trackers
9.     public GameObject Player;
10.     public Camera MainCamera_L;
11.     public Camera MainCamera_R;
12.  
13.     //Pairs of cameras used for the renderTexture
14.     public GameObject PortalCamera_Holder;
15.     public Camera PortalCamera_L;
16.     public Camera PortalCamera_R;
17.  
18.     //Portals
19.     public Transform Source;
20.     public Transform Destination;
21.  
22.     //Debug hud
23.     public static Text debugText_PosCompare;
24.     public static Text debugText_EulerRightCompare;
25.     public static Text debugText_EulerLeftCompare;
26.     public static Text debugText_Headset;
27.  
28.  
29.  
30.  
31.     void Start()
32.     {
33.         Application.onBeforeRender += OnBeforeRender;
34.  
35.         //Debug
36.         debugText_PosCompare = GameObject.Find("debugText_PosCompare").GetComponent<Text>();
37.         debugText_EulerRightCompare = GameObject.Find("debugText_EulerRightCompare").GetComponent<Text>();
38.         debugText_EulerLeftCompare = GameObject.Find("debugText_EulerLeftCompare").GetComponent<Text>();
39.  
40.     }
41.  
42.  
43.  
44.  
45.  
46.     // OneBeforeRender is used for better pos / rotation tracking.
47.     // Putting all the tracking stuff inside should kill all lags.
48.     // https://bit.ly/2OTVi65
49.     void OnBeforeRender() {
50.  
51.  
52.         /*
53.         ----------
54.         TRACKING & CLIP
55.         ----------    
56.         */
57.  
58.         // Holder position
59.         Vector3 cameraPositionInSourceSpace_L = Source.InverseTransformPoint(MainCamera_L.transform.position);
60.         PortalCamera_Holder.transform.position = Destination.TransformPoint(cameraPositionInSourceSpace_L);
61.  
62.         // Left rotation
63.         Quaternion cameraRotationInSourceSpace_L = Quaternion.Inverse(Source.rotation) * MainCamera_L.transform.rotation;
64.         PortalCamera_L.transform.rotation = Destination.rotation * cameraRotationInSourceSpace_L;
65.  
66.         // Right rotation
67.         // Vector3 cameraPositionInSourceSpace_R = Source.InverseTransformPoint(MainCamera_R.transform.position);
68.         // PortalCamera_R.transform.position = Destination.TransformPoint(cameraPositionInSourceSpace_R);
69.         Quaternion cameraRotationInSourceSpace_R = Quaternion.Inverse(Source.rotation) * MainCamera_R.transform.rotation;
70.         PortalCamera_R.transform.rotation = Destination.rotation * cameraRotationInSourceSpace_R;
71.  
72.         // Cliplane projection matrix
73.         // All the code relative to this part come from https://bit.ly/2Kc6VGN and here https://bit.ly/2WUpwZx
74.         Vector4 clipPlaneB_L = CameraSpacePlane(PortalCamera_L.GetComponent<Camera>(), Destination.transform.position, Destination.transform.forward * -1, 1.0f);
75.         Matrix4x4 projectionB_L = MainCamera_L.GetComponent<Camera>().GetStereoProjectionMatrix(Camera.StereoscopicEye.Left);
76.         CalculateObliqueMatrix(ref projectionB_L, clipPlaneB_L);
77.         PortalCamera_L.GetComponent<Camera>().projectionMatrix = MainCamera_L.GetComponent<Camera>().CalculateObliqueMatrix(clipPlaneB_L);
78.  
79.         Vector4 clipPlaneB_R = CameraSpacePlane(PortalCamera_R.GetComponent<Camera>(), Destination.transform.position, Destination.transform.forward * -1, 1.0f);
80.         Matrix4x4 projectionB_R = MainCamera_R.GetComponent<Camera>().GetStereoProjectionMatrix(Camera.StereoscopicEye.Right);
81.         CalculateObliqueMatrix(ref projectionB_R, clipPlaneB_R);
82.         PortalCamera_R.GetComponent<Camera>().projectionMatrix = MainCamera_R.GetComponent<Camera>().CalculateObliqueMatrix(clipPlaneB_R);
83.  
84.  
85.         /*
86.         ----------
87.         DEBUG
88.         ----------    
89.         */
90.         //hud
91.         debugText_PosCompare.text = "PlayerPos : " + Player.transform.position + " || HolderPos : " + PortalCamera_Holder.transform.position;
92.         debugText_EulerLeftCompare.text = "MainL : " + MainCamera_L.transform.eulerAngles + " ||  CloneL : " + PortalCamera_L.transform.eulerAngles;
93.         debugText_EulerRightCompare.text = "MainR : " + MainCamera_R.transform.eulerAngles + " ||  CloneR : " + PortalCamera_R.transform.eulerAngles;
94.     }
95.  
96.  
97.  
98.  
99.     /*
100.     ----------
101.     FUNCTIONS
102.     ----------
103.     */
104.  
105.     // Extended sign: returns -1, 0 or 1 based on sign of a
106.     private static float sgn(float a)
107.     {
108.         if (a > 0.0f) return 1.0f;
109.         if (a < 0.0f) return -1.0f;
110.         return 0.0f;
111.     }
112.  
113.     // Given position/normal of the plane, calculates plane in camera space.
114.     private Vector4 CameraSpacePlane(Camera cam, Vector3 pos, Vector3 normal, float sideSign)
115.     {
116.         Vector3 offsetPos = pos + normal;
117.         Matrix4x4 m = cam.worldToCameraMatrix;
118.         Vector3 cpos = m.MultiplyPoint(offsetPos);
119.         Vector3 cnormal = m.MultiplyVector(normal).normalized * sideSign;
120.         return new Vector4(cnormal.x, cnormal.y, cnormal.z, -Vector3.Dot(cpos, cnormal));
121.     }
122.  
123.     // Adjusts the given projection matrix so that near plane is the given clipPlane
124.     // clipPlane is given in camera space. See article in Game Programming Gems 5 and
125.     // http://aras-p.info/texts/obliqueortho.html
126.     private static void CalculateObliqueMatrix(ref Matrix4x4 projection, Vector4 clipPlane)
127.     {
128.         Vector4 q = projection.inverse * new Vector4(
129.             sgn(clipPlane.x),
130.             sgn(clipPlane.y),
131.             1.0f,
132.             1.0f
133.         );
134.         Vector4 c = clipPlane * (2.0F / (Vector4.Dot(clipPlane, q)));
135.         // third row = clip plane - fourth row
136.         projection[2] = c.x - projection[3];
137.         projection[6] = c.y - projection[7];
138.         projection[10] = c.z - projection[11];
139.         projection[14] = c.w - projection[15];
140.     }
141. }

Thanks to @CaptainScience & @JoeStrout & @EthanF4D the help they provided : )

 

Hi @Untoldecay !

Is there a reason why your RelativePortalCamera script only works on one portal at a time? I'm trying to have multiple portals in my scene, but the effect only happens to one of the portals - the rest of the other portals are static.

Thanks!

 

It looks really good!
Would you be able to share the unity project? If it is fine, I would like to use it on my (non-commercial) project =)

 

Hey there,

Sorry folks I totally stopped this project.
It was partly working, and the other side of the protal, once the player teleported, wasn't displaying the rend texture right.
I never continued, but I will post a link to my project files if I got the time ( I have to search / find them in a 2 years mess of files haha )

Does some of you succeed in this adventure ?

 

I've been working this problem for the last few months, so any code you can post would be super useful!

 

@Untoldecay Have you happened to find the files that you used? I am working on the same problem, so any help would be great! Thanks!

 

Please post the code! i have the same issue (