Troops are overlapping

My game is 2D top-down shooter game. I use the free asset A* pathfinder project for pathfinding and patrol. I have patrol, chase, retreat, and attack setup. But my enemies are overlapping each other or walking through each other. If I enable RigidBody dynamic they get stuck or pushes each other instead of going around. Screenshots attached.

 

This is a hard problem to solve in the general case. Unity solves it internally by making NavMeshAgents try to wiggle around each other to some random extent.

For your non-NavMesh solution you need to come up with some other way, and it gets tricky.

For instance you could give every agent a priority, or even shifting priority given current tasks, and then make lower-priority agents give up moving if they get too close to high-priority ones. BUT... what if that low-priority agent notices and gives up while standing right in front of an oncoming agent? Same problem.

Another way is to make each agent feed their position back transiently into the cost graph of the level, at whatever granularity makes sense. This may complicate your pathing graph a lot, but it would let agents mark cells they are in as "difficult" so that others tend to avoid them. Again, still no 100% guarantee, since 3 agents might all be heading for the same doorway or gap.

Look up pathfinding deconflicting for more info.