Search Unity

How to convert a 3D coordinate to screen coordinate(Monitor,LeftEye)

Discussion in 'AR/VR (XR) Discussion' started by Deleted User, Aug 6, 2020.

  1. Deleted User

    Deleted User

    Guest

    Hello everyone ~

    I use an HMD with eye tracking, through eye tracking api, I can get the object which was being looked at.

    Now I want to display a marker on the monitor to represent this object, so I need to convert the world position of the 3d object to screen position.

    Here is my code

    Code (CSharp):
    1. Transform m_Transform;
    2.  
    3. // Start is called before the first frame update
    4. void Start()
    5. {
    6.     m_Transform = this.transform;
    7.     UnityEngine.XR.XRSettings.gameViewRenderMode = UnityEngine.XR.GameViewRenderMode.LeftEye;
    8. }
    9. Vector2 ScreenPos = Vector2.zero;
    10. // Update is called once per frame
    11. void Update()
    12. {
    13.     CalculateScreenPosition();
    14. }
    15.  
    16. void CalculateScreenPosition()
    17. {
    18.     var projMatrix = Camera.main.GetStereoProjectionMatrix(Camera.StereoscopicEye.Left);
    19.     var viewMatrix = Camera.main.GetStereoViewMatrix(Camera.StereoscopicEye.Left);
    20.     Vector4 worldPos = m_Transform.position;
    21.     worldPos.w = 1;
    22.     Vector4 viewPos = viewMatrix * worldPos;
    23.     Vector4 clipPos = projMatrix * viewPos;
    24.     ScreenPos = new Vector2(clipPos.x / clipPos.w, clipPos.y / clipPos.w);
    25. }
    26. private void OnGUI()
    27. {
    28.     if (ScreenPos != Vector2.zero)
    29.     {
    30.         GUI.Box(new Rect((ScreenPos.x + 0.5f) * Screen.width, (1 - (ScreenPos.y + 0.5f)) * Screen.height, 100, 100), "test");
    31.     }
    32. }
    but something wrong, the screen position is incorrect, here is the result (gif url: https://i.loli.net/2020/08/06/mTqf1Hb7xY4VsP5.gif):


    Can anyone solve this problem? thank u.
     

    Attached Files:

    Last edited by a moderator: Aug 11, 2020