Bug Navmesh AI collision with player causes strange speed effect

So I'm a little lost as to whats causing this and hoping to get a little insight.

I have my player character that uses the CharacterController component for movement.
I have some enemies set up with a capsule collider and navmesh agent.

While the player collides with the AI while its following the player, it'll cause to the player to shoot past them at an abnormally fast speed. This is only when both player and AI are moving.

I'm using

_navAgent.destination

and

_characterController.Move()

for the AI and Player respectively.

I know about

OnControllerColliderHit

but I'm not sure if that would even work here or what I would check for to stop this random burst of speed.