WorldToScreenPoint while using the Oculus

I implemented an image-plane selection technique that ignores depth. For this I am casting a ray from the screen, and whatever object is hit first is selected.

var screenPoint = Camera.main.WorldToScreenPoint(_pinchPos);
var ray = Camera.main.ScreenPointToRay(screenPoint);
RaycastHit hit;
if (Physics.Raycast(ray, out hit, 10, LayerMask))
{

}

Everything works well without the oculus connected, however I dont have an oculus to test if it works with it connected. My question is, does Camera.main.WorldToScreenPoint, and Camera.main.ScreenPointToRay behave the same way when the oculus is connected? I know that certain things dont work with the Oculus such as screenspace ui.

If it doesnt work, how would I achieve the same thing?

 

bump.

i've run into the same issue, camera.worldToScreenPoint is returning some unexpected values while using the oculus, making my direction indicator (which is on a screen space camera UI) act weird.

everything works fine without the oculus, meaing the indicator points where it should.

anyone have any ideas?

 

ahh, found a solution... if anyone arrives here from google (like me) here's what i did:

i was using Screen.width/height for calculations. sadly, while the HMD is active, these still contain the dimensions of the game window, while camera.worldtoscreenpoint returns coordinates in respect to the oculus view dimensions.

changing these to the indicator's parent canvas's width/height (form canvas.rectTransform.sizeDelta) fixed the weirdness for me.

(i would be nice to have a reference somewhere as to what is different in vr modes, and maybe there is, and i just don't know about it?)

 

Sound like the same problem here:

 

I made a solution here: https://gist.github.com/jackbrookes/5c4edc20a465a6d6043f319ef3538207

 

I've got a very similar issue, and was hoping someone could help.

Trying to show an indicator onscreen that is lined up with a target object if the player has it somewhere in view, using a canvas UI. If not, I will show an offscreen indicator with an arrow. This code 100% works in a 3D environment, but once I enter the XR realm and a world space canvas I get wacky values when I try to do a WorldToScreenPoint.

This is the onscreen code:

Code (CSharp):

1. private void LateUpdate()
2.     {
3.         Camera cam = null;
4.         cam = gm.XR_Level.mainCam; // Points to the center XR eye
5.  
6.         Vector3 viewPos = cam.WorldToViewportPoint(GetCurrentMissionPt()); // CurrentMissionPt is the destination transform
7.        
8.         if (viewPos.x > 0 && viewPos.x < 1 && viewPos.y > 0 && viewPos.y < 1 && viewPos.z > 0)
9.         {
10.             iWaypointOnscreen.gameObject.SetActive(true);
11.             iWaypointOffscreen.gameObject.SetActive(false);
12.             gOffscreen.gameObject.SetActive(false);
13.             Vector3 screenPos = cam.WorldToScreenPoint(GetCurrentMissionPt());
14.          
15.             iWaypointOnscreen.gameObject.transform.position = new Vector3(screenPos.x, screenPos.y, 0);
16.         }
17.         else
18.         {
19.             iWaypointOnscreen.gameObject.SetActive(false);
20.             iWaypointOffscreen.gameObject.SetActive(true);
21.             gOffscreen.gameObject.SetActive(true);
22.             var tmpVector = cam.gameObject.transform.InverseTransformPoint(GetCurrentMissionPt());
23.             var angleToTarget = Mathf.Atan2(tmpVector.y, tmpVector.x) * Mathf.Rad2Deg;
24.             gOffscreen.gameObject.transform.localEulerAngles = new Vector3(0, 0, -angleToTarget);
25.         }
26.     }

Thank you!

 

It is in Japanese, but I was able to solve the problem by creating my own ScrenPointtoRay() as shown in this article.

https://deziowata.hateblo.jp/entry/2023/11/23/185553