Resolved Unable to set layer mask

I'm building a procedural scene, in order to get the physics properly working, I need to set layer masks on certain objects. I have this:

Code (CSharp):

1. [SerializeField] LayerMask layer;
2.    .
3.    .
4.    .
5.       GameObject obj = builder.gameObject;
6.       obj.layer      = layer;

but the value of obj.layer becomes zero.

and this doesn't work either:

Code (CSharp):

1.  
2.       obj.layer = layer.value;

 

A LayerMask is not a layer. But, because both can boil down to an int, it can be confusing (and the compiler will let you do it). GameObjects don't have a LayerMask, they only have a layer, which is a plain old number from 0 to 31. While a LayerMask can be converted to an int, what it really is is a series of bits, one bit for each layer - so the bit at index 21 tells you whether layer 21 is enabled or disabled. But an integer with a 1 in bit 21 isn't the number 21, it's some number in the billions.

Basically, you want to make "layer" an int and put in the number of the layer there, not use a layermask.

 

Or specifically, it is 1 shifted left (doubled) 21 times, which my cacalator tells me is 2097152

The point of LayerMasks is that more than one bit can be turned on, such as giving a LayerMask to a Camera and saying "I want you to see Layer 4, Layer 9 and Layer 11, which would be:

(1 << 4) | (1 << 9) | (1 < 11)

Each bit individually shifted to position, then logical OR-ed together.

And for the OPPOSITE, like "I don't want you to see layer 4, 9 and 11," you take the final quantity and use the tilde (

~

) to bit-invert the mask.

 

I solved it like this:

Code (CSharp):

1.  
2.       obj.layer = (int) Mathf.Log(layer.value,2);

Obviously, this would fail if layer is set to two or more layers. This won't be a problem, If only there was a way to tell the editor to accept only one layer.

 

This is a clever use of log and integer cast, and it might work 100% of the time, however, I would stay away from it for this reason: it is a black box and it accepts floating point numbers. This implies that it is subject to floating point imprecision.

That means it might return X.999999 one time and (int) casting rounds down, so you would get X when the correct answer is (X+1)

Since powers of 2 are SUPER easy to handle with computers, this scenario would probably never happen, but I would prefer to guarantee it. I am not a floating point library implementor.

You can guarantee it with

Mathf.Round()

instead of the int cast.

 

Sit down, spend an hour to learn the basics how numbers look like in binary and then another half an hour to learn how to exploit it with bitwise operators and problem solved. It isn't that complicated (really) but it is incredibly powerful.

 

Nope, the source of the problem is this:

Code (CSharp):

1. [SerializeField] LayerMask layer;

Because, if the designer accidentally or intentionally sets 2 or more layers, there's no mathematical method that would tell you which one he/she actually wants to use. So the whole idea of using a LayerMask for this job is wrong. You have to use a string, that way if the designer types something that doesn't correspond to a valid layer, you can throw an exception OnAwake whenever LayerMask.NameToLayer returns -1.

 

You could make your own enum that corresponds 1-to-1 with your layers and just set up a public field of that.

Like this guy:

https://forum.unity.com/threads/global-enum-for-tags-and-layers.559030/

You could even write a little editor script to create that public enum based on your currently defined layers.

That way you would just have

Code (csharp):

1. public MyLayerEnum DesiredLayer;

And Unity will give you a nice drop down with the names in it.

That way you're not conflating the use of masks (multiple bits) with layers (bit positions).

 

I knew I had one of these somewhere from YEARS ago... I dug it up and blew the dust off it, cleaned it up and here it is in all its glory:

Editor script to auto-generate an enum for current project Layers:

https://pastebin.com/mi2Ct5XK

This lets you specify layers in the Unity inspector with:

Code (csharp):

1. public UnityLayer myLayer;

I'm surprised Unity doesn't support this yet; I predict they will someday.

Ninja edit: fixed the above link because I originally posted broken code.

 

Well, creating code from actual config data is possible, but generally not a good approach ^^. What Unity could provide is a simple struct type, just like LayerMask, and simply create a property drawer for that. So we actually storing the layer index as int value, but we get nice inspector support.

As you may guess, someone already did this years ago. Funnily the Unity editor does have a "LayerField" control to display a layer field dropdown. It's most likely the one that is used in the GameObject inspector at the top where you actually select the layer for the gameobject.