NavMeshAgent interferes with my rigidbody locomotion system. Stuck on ramps, stairs are broken, etc

Hi, I am working on a game where my NPCs need to be able to navigate up a series of platforms, climb stair-like steps, and jump over hazards. My character solution is generic and extensible, but my intention with most common enemies, from humanoids to cubes of slime, is to have them make use of my Rigidbody based locomotion system. Currently I use the navmesh agent to pilot my character like so:

Code (CSharp):

1. // Inside the AI chase player class.
2. public void FixedTick()
3. {
4.         _brain.Agent.SetDestination( _brain.Target.transform.position);
5.         _brain.Character.SetAxisInput("wishDir", _brain.Agent.desiredVelocity.normalized);
6.         _brain.Agent.nextPosition = _current.position;
7. }

It works nicely on flat ground, but struggles when climbing ramps, or stepping up stair-like steps. The guy uses the same locomotion and stair stepping logic as the slime does. I can even tell the player inputs to go into the slime and the slime will move just fine.

Edit: haha oops, I forgot to turn off my music before recording.

This is what my slime looks like in terms of colliders and whatnot:



Two box colliders, one for physical interaction with the world, and the outer is a trigger to hurt the player on contact so it can be ignored. The agent is tall as a temporary workaround to keep the slime chasing the player even when they are above the ground. The issue on the ramp appears to be that the agent has the box collider partially pushed into the ramp face, so that it gets caught on the flat face at the top of the ramp. The navmesh agent doesn't want to step up my little stepping block because I think the agent is forcing the object to stay in contact with the ground. Is there a way to allow my character to move vertically without the agent gluing its center of gravity to the ground?

While writing this, it occurred to me to maybe have a separate object that navigates and the physically simulated character attempts to follow, but this may be difficult to manage with the likelihood of the characters getting out of sync. I would really like to hear from someone with more experience. Thanks!