http://docs.unity3d.com/ScriptReference/NavMesh.html You'll want to look at CalculatePath first and then FindClosestEdge
Use NavMesh.SamplePosition. This allows you to specify a maximum difference between the given point and the nearest point on the navmesh. If the nearest point is further away, the call returns false.