Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Raycast "layermask" parameter

Discussion in 'Scripting' started by Voodin, Aug 3, 2020.

  1. Voodin

    Voodin

    Joined:
    Apr 19, 2018
    Posts:
    48
    I'm attempting to make a point and click game, and am working on the actual pointing and clicking script. Im using a raycast to find the point on the ground that the ray hits and moving toward it.

    Code (CSharp):
    1. public override void Movement()
    2.         {
    3.             mousePos = Input.mousePosition;
    4.             ray = cam.ScreenPointToRay(mousePos);
    5.             var mask = 9;
    6.  
    7.             RaycastHit hit;
    8.             if(Physics.Raycast(ray,out hit,Mathf.Infinity,mask))
    9.             {
    10.                 Debug.Log(hit.point);
    11.                 transform.position = Vector3.MoveTowards(transform.position, hit.point, .2f);
    12.             }
    13.         }
    in the inspector I created a layer named collision which the ground is assigned to, it is the 10th layer, however using 9 for the layermask, makes my player object rush toward the screen. the player is on the default layer. I tried using 10 for layer mask and nothing happens.
    upload_2020-8-2_19-53-17.png
    upload_2020-8-2_19-53-39.png
    I don't understand what's going on here, if "default" actually means it has the tag of all of the other layers or something, why doesn't the player move toward the mouse when the layermask parameter is set to 10? also why isn't the properly numbered layer mask working?
     
    aparajithsairam and hopetolive like this.
  2. Kurt-Dekker

    Kurt-Dekker

    Joined:
    Mar 16, 2013
    Posts:
    39,071
    The layer is a number from 0 to 31. It corresponds to each of those named layers.

    A layer MASK is a 32-bit bitfield that says which of these you want to ignore. This enables you to ignore (mask) multiple layers, each one represented by a single bit in the 32-bit integer.

    In your code above, 9 (Collision) is a layer, NOT a mask. Raycast takes a layermask.

    If you want to turn a layer name into a LayerMask, here's the utility you want:

    https://docs.unity3d.com/ScriptReference/LayerMask.GetMask.html

    You can also go cheap and cheerful and rotate the bits yourself:

    Code (csharp):
    1. int layer = 9;
    2.  
    3. int layerMask = 1 << layer;   // means take 1 and rotate it left by "layer" bit positions
    That would cause layer 9 to be masked, ie,. selected to be hit by the raycast.

    If you want to mask more than one layer, OR the bit-shifted results together:

    Code (csharp):
    1. int layerA = 9;
    2. int layerB = 12;
    3.  
    4. int layerMaskCombined = (1 << layerA) | (1 << layerB);
    To invert those masks (eg, take their opposite), use the tilde (~) (NOT A MINUS SIGN!) to flip every bit:

    Code (csharp):
    1. layerMask = ~layerMask;  // tilde ~ is bitwise flip
    That will return things marked with layer 9 or layer 12.

    I'm not going to try to add in graphics to show off bitfields... there's TONs of bitfield diagrams and tutorials all over the internet. It's the basic fundamentals of math on a digital computer.

    EDIT: corrected 1/5/2023 in response to @Bunny83 below... thanks man!
     
    Last edited: Jan 6, 2023
    pansoul, S1NATB, UltiPangol and 10 others like this.
  3. Voodin

    Voodin

    Joined:
    Apr 19, 2018
    Posts:
    48
    Boy do I feel dumb, I thought I needed the exact opposite of what I did lol. I implemented the bitflip but I'm still getting a bit of issue. I'm attempting to make a pre-rendered background game, and the current method I'm using has the background in front of an orthographic camera on its own layer. So the floor is on the collision layer, and the background is on the background layer, however if the mouse passes over the background, my character moves toward the background. upload_2020-8-2_20-16-8.png
    As you can see in the above screenshot, BG is clearly on the BG layer, any idea why this is happening?
     
  4. JBEDUnity

    JBEDUnity

    Joined:
    Jul 13, 2020
    Posts:
    23
    I was able to find an example that uses this same method. I'm new to C#, used to VBA. Correct me if I'm wrong but what's happening is we create a bit shift mask (still a little hazy on this) that takes the integer 1 and shifts the bit so the integer becomes 9. No clue why we have to do this an why we can't just use integer 9 unless there's no other way to have a "NOT" operator. Then we do the inverse so the raycast selects everything "not" integer 9.

    Is this the only way to do this? I'm asking cause I'm confused as to why to do this. It seems like I'm climbing up a fire escape when I could just use the elevator.
     
    Last edited: Aug 3, 2020
    Theleoboy likes this.
  5. PraetorBlue

    PraetorBlue

    Joined:
    Dec 13, 2012
    Posts:
    7,933
    You're not shifting until the integer "becomes 9", you're shifting a single "on" bit 9 places to the left. The value of the integer actually becomes 2^9. But the value doesn't really matter, Unity treats the mask more directly like an array of 32 bits that are on or off to determine which of the 32 layers are "on"

    Regardless, my preferred way of creating LayerMasks is to forgo the whole less-than-readable bit-shifting thing and just do this:
    Code (CSharp):
    1. [SerializeField]
    2. LayerMask mask;
    3.  
    4. void Update() {
    5.   SomeRaycastFunction(..., ..., ..., mask.value);
    6. }
    You can set the mask in the inspector this way the same way you pick culling masks for cameras etc...
     
  6. desertmen

    desertmen

    Joined:
    Mar 1, 2017
    Posts:
    1
    I am currently working with unity 2021.3 and for some reason my masks are flipped. For example if I want to ignore layer 3, my mask looks like this
    Code (CSharp):
    1. int mask = ~(1 << 3)
     
    tmarttao likes this.
  7. PraetorBlue

    PraetorBlue

    Joined:
    Dec 13, 2012
    Posts:
    7,933
    Yes this is completely expected. Nothing flipped about it.

    A layermask is a bitmask of layers you want the raycast to hit. It will only hit layers in the mask. If you want to hit all layers except layer 3, what you have written there makes sense.
     
    aparajithsairam and Bunny83 like this.
  8. Bunny83

    Bunny83

    Joined:
    Oct 18, 2010
    Posts:
    4,120
    Right, I guess Kurt had something mixed up when he wrote his answer back then. Because he explained it the other way round :) Yes, a mask is masking layers which you want / don't want to hit. Though a 1 in the mask means you can hit the layer, a 0 at the corresponding bit position means you can not hit that layer. So a mask of "0" would mean you can't hit anything while "-1" (equivalent to 0xFFFFFFFF) means all 32 layers can be hit, even the "ignore raycast layer" which is usually excluded in the Physics.DefaultRaycastLayers ^^.

    Just to make that clear again. We already have countless of other posts that explain this, but I think there's always someone that finds this thread before any other, so it doesn't hurt.

    A 32 bit integer, as the name suggests, has 32 bits. A bit can either be set (so it's 1) or clear (so the bit is 0). To visualize a bit mask, we can write the number as a binary number like that

    1111 1111 1111 1111 1111 1111 1111 1011


    The default layer masks are defined here. The binary number I just showed is the actual value of "DefaultRaycastLayers". So all 32 bits are set, except bit number 2 (the third layer since we start counting at 0) which represents the ignore raycast layer.

    When you would use the decimal number "9" as a layermask, you would actually use a mask like this:
    0000 0000 0000 0000 0000 0000 0000 1001
    which is binary for "9" is only includes layer 0 and layer 3. Layer 0 has the value 1 (== 1 << 0) while layer 3 has the value 8 (== 1 << 3). With the "bitshift left" operator you can easily get the actual mask value for a certain layer since the bit shift simply shifts the bits in the given number by x bits to the left. So by using the number 1 and shifting it x times to the left, you get the value for only layer x.

    While you could simply "add" several layer mask values up (1 + 8 == 9) it is not recommended as when you try to combine two layer masks that have "overlaps", you get a "carry" and it results in the wrong bits being set. To combine several bitmasks into one, you should use bitwise operators like or
    |
    , and
    &
    as well as not
    ~


    Just as an example, imagine the layermasks "33" (0010 0001, so layer 0 and layer 5) and the layermask 3 (0000 0011, so layer 0 and layer 1). When you just "add" them up (33 + 3 == 36) you get a mask that looks like this: 0010 0100. So we would get layer 5 and layer 2. Though our two initial masks contained layer 0 and 1. When using the proper bitwise or operator, each bit is "or"ed with the corresponding bits. So doing 33 | 3 == 35

    Code (CSharp):
    1. 0010 0001 (33)
    2. 0000 0011 ( 3)
    3. ---------
    4. 0010 0011 (35)
    5.  
    The bitwise "not" operator
    ~
    simply inverts all bits. So every bit that was originally 0 becomes a 1 and every 1 becomes a 0. So a mask like
    0000 0000 0000 0001 0000 0100 0010 0011
    ( == 1+2+32+1024+65536 == 66595), when you invert it it becomes
    1111 1111 1111 1110 1111 1011 1101 1100
    (== 4294900700 as an unsigned integer)

    For those not familiar with the binary system, if you're on a windows system you can start up the built-in calculator (calc.exe) and switch it to programmer mode. It allows you to view numbers as hexadecimal (base 16), decimal (our usual base 10 system), octal (base 8) or binary (base 2). All those number systems actually work exactly the same, though they have different number of symbols per digit. That changes the "value" of a certain digit position. While in the decimal system each digit place is the value of the previous place multiplied by 10 (so "ones" at the far right, then 10s, 100s, 1000s, ....), in binary each digit grows by a factor of 2. So we have ones then 2s, 4s, 8s, 16s, 32s, 64s, 128s, ...
    When you count in a number system you generally start at the ones place and go through the available digits. For decimal thats 0 through 9. When you add another one you get a carry into the 10s place and the ones go back to 0. In the binary system the only available digits are 0 and 1. So once we have a 1 in the ones place and we add one, we get a carry into the 2s place and the ones place goes back to 0. So we get "10"b.
    Code (CSharp):
    1.    BIN      DEC    binary composition     HEX   OCT
    2. ----------------------------------------------------
    3. 0000 0000  ( 0)  [                    ]  0x00   000
    4. 0000 0001  ( 1)  [                  1 ]  0x01   001
    5. 0000 0010  ( 2)  [              2     ]  0x02   002
    6. 0000 0011  ( 3)  [              2 + 1 ]  0x03   003
    7. 0000 0100  ( 4)  [          4         ]  0x04   004
    8. 0000 0101  ( 5)  [          4 +     1 ]  0x05   005
    9. 0000 0110  ( 6)  [          4 + 2     ]  0x06   006
    10. 0000 0111  ( 7)  [          4 + 2 + 1 ]  0x07   007
    11. 0000 1000  ( 8)  [      8             ]  0x08   010
    12. 0000 1001  ( 9)  [      8 +         1 ]  0x09   011
    13. 0000 1010  (10)  [      8 +     2     ]  0x0A   012
    14. 0000 1011  (11)  [      8 +     2 + 1 ]  0x0B   013
    15. 0000 1100  (12)  [      8 + 4         ]  0x0C   014
    16. 0000 1101  (13)  [      8 + 4 +     1 ]  0x0D   015
    17. 0000 1110  (14)  [      8 + 4 + 2     ]  0x0E   016
    18. 0000 1111  (15)  [      8 + 4 + 2 + 1 ]  0x0F   017
    19.  
    20. 0001 0000  (16)  [ 16                 ]  0x10   020
    21. 0001 0001  (17)  [ 16 +             1 ]  0x11   021
    22. 0001 0010  (18)  [ 16 +         2     ]  0x12   022
    23. 0001 0011  (19)  [ 16 +         2 + 1 ]  0x13   023
    24. 0001 0100  (20)  [ 16 +     4         ]  0x14   024
    25. .........
    Looking at that table you will see why we often use hexadecimal numbers instead of binary. They have a neat 4-to-1 relationship (every 4 binary digits make up a single hex digit). So it's easy to do a (partial) conversion between the two systems.
     
  9. Yeowza

    Yeowza

    Joined:
    May 9, 2014
    Posts:
    48
    can someone explain why we have to do this? Why can't unity just make it easy under the hood and let us specify which layer to use with an integer
     
  10. MelvMay

    MelvMay

    Unity Technologies

    Joined:
    May 24, 2013
    Posts:
    11,589
    Both the 2D (Box2D) and 3D (PhysX) use bitmasks to quickly determine what can hit what for collision shapes and queries. We're exposing that. Handling bit-masks is a basic thing most game-devs do so I'm not sure I see the issue.

    There's nothing difficult using "1 << myLayer" :)

    If you need to do it "visually" then add a LayerMask field as shown above and select it in the inspector.

    Adding overloads to all the queries which include a single layer would make even more of a mess of the number of overloads so that won't happen.
     
    Ryiah likes this.
  11. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    21,498
    Additionally adding more layers is as simple as:
    Code (csharp):
    1. var layers = 1 << someNumber | 1 << anotherNumber;
    Besides if you really wanted a simpler approach you can just code it yourself:
    Code (csharp):
    1. int GetLayerMask(int layerNumber) => 1 << layerNumber;
     
    PraetorBlue and MelvMay like this.
  12. kdgalla

    kdgalla

    Joined:
    Mar 15, 2013
    Posts:
    4,673
    Also, you can just declare a public LayerMask object and then you can simply select the layers you want with the inspector widget.
     
    Spy-Master, Ryiah and PraetorBlue like this.
  13. Yeowza

    Yeowza

    Joined:
    May 9, 2014
    Posts:
    48
    ya thats what I'm sayin, why not just add that in under the hood instead of making developers go look it up? it's such an easy thing to do.

    only reason I can think of is maybe the bitwise operation is cheaper than an int or something
     
  14. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    21,498
    A bitwise operation allows you to specify multiple layers which you can't do with a single number.
     
    Last edited: Mar 20, 2024
    Bunny83 and Spy-Master like this.
  15. PraetorBlue

    PraetorBlue

    Joined:
    Dec 13, 2012
    Posts:
    7,933
    The bitwise operation is using ints. There's nothing about it that isn't an int. It's so you can hold multiple layers in the mask.
     
    Bunny83 and Ryiah like this.