Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Bug Physics.Raycast() layer mask is to SELECT, not IGNORE layers

Discussion in 'Documentation' started by jakesee, Sep 22, 2023.

  1. jakesee

    jakesee

    Joined:
    Jan 7, 2020
    Posts:
    26
    In the docs https://docs.unity3d.com/ScriptReference/Physics.Raycast.html, the statement "A Layer mask that is used to selectively ignore colliders when casting a ray." is incorrect.

    Also, "You may optionally provide a LayerMask, to filter out any Colliders you aren't interested in generating collisions with." is incorrect.

    When testing, Physics.Raycast() layer mask is to SELECT, not IGNORE layers. I spent 2 hours pulling my hair off...

    Example:

    Code (CSharp):
    1. public static bool IsMouseButton(this Camera camera, int button, out RaycastHit hit, LayerMask layerMask)
    2.         {
    3.             hit = new RaycastHit();
    4.  
    5.             if (Input.GetMouseButton(button))
    6.             {
    7.                 Debug.Log($"Mouse button {button} is down, layer mask {layerMask.value}");
    8.  
    9.                 var ray = camera.ScreenPointToRay(Input.mousePosition);
    10.  
    11.                 // I tested layerMask and ~layerMask
    12.                 // layerMask proves to be used for SELECTING and not IGNORING layers
    13.                 if (Physics.Raycast(ray, out hit, Mathf.Infinity, layerMask))
    14.                 {
    15.                     Debug.Log($"Mouse button {button} is down, hit {hit.collider.gameObject.name}");
    16.                     return true;
    17.                 }
    18.             }
    19.  
    20.             return false;
    21.         }
     
  2. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    577
    Hello,

    So sorry to hear you lost time to this. While this won't return back any of your two hours, I still thought to post and explain what was the original thought process of the author of that doc (for historical purposes and curiosity of whoever reaches to this forum thread in the coming years with the same question).

    The default value of the layer mask is to include objects from all possible layers, so in the practical sense one would only use a custom set layer mask to have some layers excluded (in other words -- ignored).

    We're going to change this wording for clarity in the future releases.

    Anthony
     
  3. jakesee

    jakesee

    Joined:
    Jan 7, 2020
    Posts:
    26
    @yant suggestion: It would help tremendously if the docs included a proper/comprehensive example of how the mask is intended to be used. As of writing, none of the examples show use of mask. Thanks.