Search Unity

Strange Editor Raycast bug

Discussion in 'Editor & General Support' started by delinx32, Nov 22, 2018.

  1. delinx32

    delinx32

    Joined:
    Apr 20, 2012
    Posts:
    417
    I have a project that uses raycasting in the editor to determine the 3d world position of the mouse cursor. The code is pretty straight forward and has always worked in the past. Recently I noticed that my selected points are off by quite a bit. I spent several hours trying to figure out what's wrong with my code only to realize that the issue only occurs in upgraded unity editors.

    I pulled a commit from my git that was prior to my project upgrade to test my theory and found out that the code works perfectly fine in 2017.2.0f3. So I copied the project folder and did nothing except open it in 2018.2.16f1 and the problem occurs. It seems like the editor canvas window does not start at 0,0 in the bottom left corner. so all of the math to calc where the mouse position is is off by something like 120 pixels on both the x and z.

    I wish I could get a screenshot of the effect in both versions, but when I print screen, it doesn't capture the mouse so you can't really see how far off it is.

    The code below seems to be the standard way to do this. hit.point should be the 3d position after raycasting.

    Anybody notice anything similar, or have any ideas?

    Code (CSharp):
    1.             SceneView sv = SceneView.lastActiveSceneView;
    2.             Vector3 mousePos = new Vector3(Event.current.mousePosition.x, sv.camera.pixelHeight - Event.current.mousePosition.y);
    3.  
    4.             RaycastHit hit;
    5.  
    6.             var ray = sv.camera.ScreenPointToRay(mousePos);
    7.  
    8.             if (Physics.Raycast(ray, out hit))
    9.             {
     
  2. delinx32

    delinx32

    Joined:
    Apr 20, 2012
    Posts:
    417

    I created two scripts to demonstrate the issue.

    Reproduce:

    Start a brand new project in the 2017 version
    Add a plane with a mesh collider
    Add the following 2 scripts to the new project
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. public class Test : MonoBehaviour {
    6.  
    7.     // Use this for initialization
    8.     void Start () {
    9.        
    10.     }
    11.    
    12.     // Update is called once per frame
    13.     void Update () {
    14.        
    15.     }
    16. }
    17.  
    And to the editor folder:

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEditor;
    4. using UnityEngine;
    5.  
    6. [CustomEditor(typeof(Test))]
    7. public class TestEditor : Editor
    8. {
    9.  
    10.     // Use this for initialization
    11.     void Start()
    12.     {
    13.  
    14.     }
    15.  
    16.     // Update is called once per frame
    17.     void Update()
    18.     {
    19.  
    20.     }
    21.  
    22.     Vector3 pos;
    23.     private void OnSceneGUI()
    24.     {
    25.         SceneView sv = SceneView.lastActiveSceneView;
    26.         Vector3 mousePos = new Vector3(Event.current.mousePosition.x, sv.camera.pixelHeight - Event.current.mousePosition.y);
    27.  
    28.         RaycastHit hit;
    29.  
    30.         var ray = sv.camera.ScreenPointToRay(mousePos);
    31.  
    32.         if (Physics.Raycast(ray, out hit))
    33.         {
    34.             int controlId = GUIUtility.GetControlID(FocusType.Passive);
    35.             HandleUtility.AddDefaultControl(controlId);
    36.             Handles.color = Color.blue;
    37.             Handles.SphereCap(controlId, hit.point, Quaternion.identity, 1);
    38.  
    39.         }
    40.     }
    41. }
    42.  
    Observe it working properly in 2017 my moving the mouse over the plane. The ball should follow the mouse.

    Copy the project and open it in 2018. Observe the misalignment.
     
  3. delinx32

    delinx32

    Joined:
    Apr 20, 2012
    Posts:
    417
    Posting the solution in case anyone else encounters this error and this thread:

    var ray = HandleUtility.GUIPointToWorldRay(Event.current.mousePosition);
     
    Lurking-Ninja likes this.