[Solved] NavMeshSurface Bug or feature? (SamplePosition)

Hey guys, stuck real hard

- Using navMeshSurface, the default example from https://github.com/Unity-Technologies/NavMeshComponents, scene 1_multiple_agent_sizes.unity


I've modified the movement script to the following:

Spoiler: script attached on the 2 agents in scene

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.AI;
3. using System.Collections.Generic;
4.  
5. // Use physics raycast hit from mouse click to set agent destination
6. [RequireComponent(typeof(NavMeshAgent))]
7. public class ClickToMove : MonoBehaviour
8. {
9.     NavMeshAgent m_Agent;
10.     RaycastHit m_HitInfo = new RaycastHit();
11.  
12.     void Start()
13.     {
14.         m_Agent = GetComponent<NavMeshAgent>();
15.  
16.  
17.     }
18.  
19.     void Update()
20.     {
21.         if (Input.GetMouseButtonDown(0) == false || Input.GetKey(KeyCode.LeftShift)) {
22.             return;
23.         }
24.  
25.         var ray = Camera.main.ScreenPointToRay(Input.mousePosition);
26.         if (Physics.Raycast(ray.origin, ray.direction, out m_HitInfo) == false) {
27.             return;
28.         }
29.  
30.         NavMeshHit nav_hit;
31.  
32.         NavMesh.SamplePosition(m_HitInfo.point, out nav_hit, 100, NavMesh.GetAreaFromName("Walkable"));
33.         m_Agent.destination = nav_hit.position; //always Vector3.zero (no collision)
34.     }
35.  
36.  
37. }

There is never a collision when I specify "Walkable" area, when using a NavMeshSurface.

However, if I use the unity's legacy nav system - the nav-raycast will work fine, and the ray will find interection with "Walkable" area.

However, things actually do work when I use "Humanoid" ...which is not an area type, but an agent type.
Putting breakpoint on NavMesh.GetAreaFromName("Humanoid"), and the returned value is '-1' which means raycast will search through all areas, and that's why it seems to sort-of work

Can someone please give a picture of how these areas & agentTypes are treated internally?
Currently, Walkable seems to not work with NavmeshSurface, although under 'advanced' Default Area is in fact "Walkable".

An interesting thing is even though raycast doesn't work with "Walkable", the agents still avoid the areas (the areas with high cost) when running around

 

Ok, I've done more research, and the correct way of working with NavMeshSurfaces and SamplePosition (as well as other similar functions) is as follows:

1) Use NavMeshQueryFilter when working with SamplePosition, it should go instead of the navMesh mask.

2) after creating an instance of NavMeshQueryFilter, before supplying it into SamplePosition, populate AreaMask and the agetType id fields of this NavMeshQueryFilter instance.

- to populate area mask, use the following

Code (CSharp):

1. filterInstance.areaMask = (1 << NavMesh.GetAreaFromName("myArea"));

Notice, the 1 << that's because GetAreaFromName returns a mere index: 1,2,3,4,5,6, but we need to have an int where a specific bit is 'on' and the rest of the bits are 'off'. For example, 0000 1000 is 8 in decimal and has a fourth bit 'on'.
0000 0001 is 1 in decimal and has the first bit 'on'
So if we were to use 1 << 4 , we would do 0000 0001 << 4 which shifts that '1' 4 places to the left.
Also, just as a reminder, if you have two masks, for example 0000 1000 and 000 0001, you can combine them together, like this

Code (CSharp):

1. int combined mask = mask1 | mask2;  //0000 1000 | 0000 00001    is 0000 1001

- to populate finter.agentTypeId, do it as follows:

<edit> it's actually better to use the NavMeshAgent.agentTypeID directly from your local component, I mention this in the end</edit>

Code (CSharp):

1. int get_agentTypeID_byName(string agentType_wantedName){
2. for(int i=0; i< NavMesh.GetSettingsCount(); ++i) {
3.             var settings =  NavMesh.GetSettingsByIndex(i);
4.  
5.             int agentTypeID = settings.agentTypeID;
6.  
7.             var settingsName = NavMesh.GetSettingsNameFromID( agentTypeID );
8.  
9.              if(settingsName == agentType_wantedName){
10.                   return settings.agentTypeId;
11.               }
12.  
13.         }//end for
14.  
15. }

All of this means that you will specify which navMeshSurfaces to search. Only those marked with 'agentTypeId' in the inspector - "Humanoid", "Ork" etc, and which areas of these navMeshSurfaces to cast against - "Lava", "Walkable" etc

----------------------
Bonus:

In my case I didn't realise that when doing SamplePoisition(), if it did manage to sample the "Ork" surface (just as I wanted), I was surprised why the "Humanoid" agent was still moving to the sampled position.
...Well, I've successfully sampled the position from an "Ork" surface, and told the human - 'hey, here is that Vector3 point located in the world, I don't care how you get there, just do it!'

so the human travelled to that point, but he travelled using the "Humanoid" surface, and probably never reached the point - it just came as close as its "Humaniod" surface allowed it to.

Therefore, it's actually smart to use the NavMeshAgent.agentTypeID directly from your local component, not the custom function get_agentTypeID_byName() which I've linked above.

In that case, SamplePosition() will return a point closest to the desired one, but that point will be located on the surface that's actually able to be traversed by your agent.

 

Thank you.. This really helped

 

It seems like your code at the top and what you are talking about at the bottom is very different? I know you mention a couple times where you figured out a better way to do something, but any chance you can elaborate more?

I have 4 agents, and 4 different surfaces, but cant seem to get them to work correctly when they all are active. When only 1 is its fine.

 

After spending in the likes of 3 hours trying everything I could, this was a lifesaver. I wondered how exactly navmesh.sampleposition knew which navAgent surface to use. Thanks for bringing this to light for me- it could have taken me some time before i discovered it on my own.

 

Thank you for sharing the knowledge. They could have put a warning, when trying to use that method overload, if the new package is installed. Or at least check if the legacy NavMesh was baked when using the method.

The next thing I hit, is when doing a raycast against the mesh, then checking for the SamplePosition with a distance of 0.001f does not work, while a distance of 0.01 works fine. I don't know why, but for now I'm in a rush and I will just accept fate.