Spawn agent inside obstacle

Hello.
I'm building RTS game and I want to be able to "build" units inside buildings.
When unit is finished building, it walks from that building to some destination point (e.g. outer gates).
The problem is that building itself is an obstacle, and unit is agent, so when I try to do it in a straight way, unit just teleports to destination point (which already placed on navmesh surface).
When I disable NavMeshObstacle on building I'm able to make SetDestination (even through unit runs not in straight line for some reason, maybe because of BoxCollider) - but it is a bad soluton, cause other units may now step inside that building.
Only way I see - make custom animation with disabled NavMeshAgent component and move unit manually by changing it position.
Any thoughts?

 

I would recommend disabling the NavMeshAgent until they leave the building.

I am suffering from the exact opposite problem, where inaccessible areas, like inside rocks, mountains, buildings, etc. are deemed accessible, and thus the NavMeshAgent will bloody teleport inside of them. Then players can't find the agents, complete the mission, etc.