Question How to add undetectable obstacles to NavMesh Agents?

In my FPS game, zombies should not be able to steer around obstacles placed by the player during runtime and instead bump into them, meaning those will not possess any kinds of nav mesh script. Instead of steering around said obstacles, zombies will use a raycast in front of them to detect an obstacle and attempt to destroy it.

For some reason though, nav mesh agents seem to ignore collisions entirely and simply phase through any objects they can't detect. I attempted adding a second collider, checked the layers and attempted to use OnCollisionEnter or OnTriggerEnter to disable the agent on colliding, but none of those changed that. The only solution I was given was to use a BoxCast periodically to check for collisions, but that sounded like a very botched approach and I am sure there is a more straight forward way to make collisions enabled on a navmesh agent, right?

 

Does anyone have any ideas?