Question Unity2d: Using mouse to pick object with higher layer while two objects overlap.

I am working in unity 2d.

Objects A and B both have boxcollider2d component. They both have a script with the following code

private void OnMouseDown()
{
Debug.Log(this.name);
}
}

This will log the name of the object being clicked on. When they are overlapping, it seems to not matter which object is on a higher layer, meaning the object who's sprite is sitting on top of another object. What seems to matter is the last object that was moved, this is true even if it's the script or the scene editor itself during runtime. I removed all the code in the script that allowed me to move the object via mouse. So this must be something in unity itself. Let's say A is layer 1 and B is layer 2. So when they over lap, B would sit on top of 2. You would think that if I click the area where A and B overlap, the mouse will always detect B and log object B. However, if I use scene editor to move object A, it still shows A sits under B. However, if I click the overlap in game side, it actually logs A as if A is sitting on top. If I do the same to B, mouse now logs B. It doesn't matter what layer they're on.

Does anyone know what is happening and how can I make the mouse only detect the top layer object when there is an overlap?

edit:
Anyways, I just did further testing and it is highly related to z-position. If they are both same z position, initially 0, then what I said originally happens. However, if they are on different z-position. The one with the smaller number z-position will always get priority when overlapped. To be specific, if A is z=0, B is z=1, it doesn't matter who moved last, if A and B overlaps, A will get priority in the overlapped area. The layers matter for how the sprite is displayed but even if an object was on a higher layer, this z-position still applies. So even if sprite B displays on top of sprite A, sprite A still gets mouse click priority.

So I know now the solution is to modify z axis on each click however, I would rather not have to do that since this shouldn't even happen in the first place. Mouse should always detect top displayed object. Does anyone know why this happens in the first place?

 

Physics2D.Raycast is really not the correct query for this. Note that you're expecting some kind of 3D sorting (layer or z position) but you are only giving Raycast a position, so it really doesn't have enough information to deduce "closer" or "farther".

Since you want to cast a 3D ray (the ray from your camera to the game obejcts) against 2D colliders you should be using this:
https://docs.unity3d.com/ScriptReference/Physics2D.GetRayIntersection.html
with the ray from
https://docs.unity3d.com/ScriptReference/Camera.ScreenPointToRay.html

 

**edit: I am working in unity 2d.

It has nothing to do with it, here is the new script, still same results

private void OnMouseDown()
{
if (Input.GetMouseButtonDown(0))
{
Debug.Log(this.name);
}
}

Anyways, I just did further testing and it is highly related to z-position. If they are both same z position, initially 0, then what I said originally happens. However, if they are on different z-position. The one with the smaller number z-position will always get priority when overlapped. To be specific, if A is z=0, B is z=1, it doesn't matter who moved last, if A and B overlaps, A will get priority in the overlapped area. The layers matter for how the sprite is displayed but even if an object was on a higher layer, this z-position still applies. So even if sprite B displays on top of sprite A, sprite A still gets mouse click priority.

So I know now the solution is to modify z axis on each click however, I would rather not have to do that since this shouldn't even happen in the first place. Mouse should always detect top displayed object. Does anyone know why this happens in the first place?

 

I know you're working in 2D, that's why I linked a function that only works with 2D colliders. The ray is still 3D.

You need to think about this logically - the raycasts work with colliders. If the colliders are at the same Z coordinate, then it's a crapshoot which one will be hit by the raycast.

The stuff you are talking about - sorting layers - has nothing to do with colliders or raycasts. It's part of the SpriteRenderer and only affects rendering. Raycasts are completely unaware of sorting layers.