Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Which layer format to use where?

Discussion in 'Scripting' started by DiamondWolf2, Aug 14, 2019.

  1. DiamondWolf2

    DiamondWolf2

    Joined:
    Dec 9, 2018
    Posts:
    16
    I gave my project a few raycasts early on. These used a layermask of i.e. "1 << 16" to get layer 16. I then went to do the same thing with collisions - and it didn't work! After a few Debug.Log()s, it turns out I need regular 16 rather than bit 16. Is there a general rule for which number you need, or do you just have to know which function needs which number in advance? Why is this inconsistent in the first place?
     
  2. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    4,193
    It's pretty simple in theory, but in practice is causes some confusion. The important thing to notice is whether something is refering to the "layer" or whether it's referring to the "layer mask". For example, if you assign an object to a layer, or test whether it's on a certain layer, you would test the layer, which is just an integer from 0 (or 1?) to 32. However, for things like raycasting, those functions expect a layermask, which is the bit-shifted value you're referring to.

    So, if something says "layer", probably it's a integer less than 32. If it says LayerMask, it's the bit-shifted thing. That said, I know I've inverted these many times, resulting in confusing bugs.
     
  3. DiamondWolf2

    DiamondWolf2

    Joined:
    Dec 9, 2018
    Posts:
    16
    Layer is number, LayerMask is bit. Got it. I think.

    IMO they should pick one and stick with it for the whole engine. Would make life easier.
     
  4. kdgalla

    kdgalla

    Joined:
    Mar 15, 2013
    Posts:
    4,616
    They are not the same thing, because a LayerMask can represent multiple layers.

    Things like cameras, for example. don't have pick which layer you want to render. Rather, it gives a LayerMask, which allows you to choose any combination of layers. Even "all" or "none".

    Edit: Although, I don't know- is the terminology a little slippery here? Do they use LayerMask to describe both the bitmap for all 32 layers and also the bit mask for a single layer? In both cases, they are just the exact same type of bit mask, but I could understand how that might be confusing too.
     
    Last edited: Aug 14, 2019
    dgoyette likes this.
  5. DiamondWolf2

    DiamondWolf2

    Joined:
    Dec 9, 2018
    Posts:
    16
    If you want a single layer, then the layer mask only has one bit set. I don't see why they can't extend that and make all layers work on a "one layer, one bit" system.
     
  6. dgoyette

    dgoyette

    Joined:
    Jul 1, 2016
    Posts:
    4,193
    There are probably several reasons why that wouldn't be idea, but one that comes to mind is that bitmasks are usually pretty terrible to look at and work with. I'm much rather see that some gameobject is on layer 28 than to see that its layermask is 268435456. :) But another reason is that it prevents a single object from being associated with multiple layers. If we could assign an object's layermask, we could assign int.MaxValue and cause the object to be on all possible layer, which Unity clearly doesn't want.

    Come to think of it, that would be pretty nice...if layers were more like Enum flags. But I expect there's somewhere around 0% chance of that ever happening.

    Anyway, though I agree the layer vs layermask thing is confusing, you'll probably get used to it, and only make catastrophic mistakes with it once in a while. :)
     
  7. kdgalla

    kdgalla

    Joined:
    Mar 15, 2013
    Posts:
    4,616
    If it helps, I wrote some layer mask extension methods recently (for my own use):

    Code (CSharp):
    1.  
    2. public static void AddLayer(ref this LayerMask val, int layer )
    3.     {
    4.         val |= 1<<layer;
    5.     }
    6.  
    7.     public static void RemoveLayer(ref this LayerMask val, int layer)
    8.     {
    9.         val ^= 1 << layer;
    10.     }
    11.  
    12.     public static bool Contains(this LayerMask val, int layer)
    13.     {
    14.         return ((val & (1<<layer))>0);
    15.     }
    16.  
    These are not fully tested, but it adds these methods to Unity's LayerMask as alternatives to doing the bit-shift/bit-wise stuff yourself.
     
  8. hlw

    hlw

    Joined:
    Aug 12, 2017
    Posts:
    250
    It might cost you a small bit of performances and "Good practices dev fighter" to hate you, but you can create a struct that has two implicit conversions, to int, and to layerMask. If asked for an int, it'll implicitly convert to the biggest flag number set to true (Don't show that to any dev, changing values in an implicit conversion is considered as bad), if asked for a mask, it'll just be all the flags.