Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Bug Agents breaking, getting stuck in corner

Discussion in 'ML-Agents' started by Hallahallan, Mar 28, 2023.

  1. Hallahallan


    Jan 16, 2023

    I am using the dodgeball env from ml-agents and I am trying to train a CTRNN through NEAT against the existing MA-POCA agent developed for the environment. Looking at the training results revealed that most of the NEAT agents that have a high fitness only have high fitness because the MA-POCA agent frequently gets stuck in corners, making a sure victory quite easy. This means that the agents that evolve and pass on their genomes in the CTRNN aren't very intelligent, only being able to beat opposing bots that are stuck in corners.

    My questions is to whether anyone has experience with this behavior in the ml-agents toolkit, especially in the dodgeball environment. It is fair to note that the environment has been modified to be a 1v1 scenario instead of 4v4 in elimination mode, and that this may be the cause or a factor of the MA-POCA agent breaking.

    Below are some images of the agent stuck in a corner and the fitness graph. The fitness shows high best fitness due to CTRNN agents winning against MA-POCA agents stuck in a corner, but doesn't show signs of improvement in average fitness, since MA-POCA agents that dont get stuck in corners beat the CTRNN agents.