Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Question How do you deal with decaying plasticity/overfitting?

Discussion in 'ML-Agents' started by LiterallyCoder, May 8, 2023.

  1. LiterallyCoder

    LiterallyCoder

    Joined:
    Jul 25, 2020
    Posts:
    87
    Hello everyone,

    Does anyone know if it's possible to mitigate the decaying plasticity that comes with static training environments if it's nearly impossible to randomize those environments? For example, if we were training an agent to tackle an obstacle course that has to be completed in a very specific order, it would be very difficult to procedurally generate such an obstacle course (for the sake of generalization) without completely breaking it. Is it even possible to make an agent more adaptable to new environments without constantly randomizing everything? This video outlines this issue and the types of environments I'm talking about in more detail, so please watch it and read the pinned comment:

    AI Warehouse's solution is to reset a random neuron every now and then so that the agent would forget specializations that it gained in certain rooms, allowing it to be more adaptable and not fall victim to overfitting. However, this isn't really possible with ML-Agents. I've also heard of something called L1 and L2 regularization, but I'm pretty those techniques aren't directly available with ML-Agents. Do you guys know of a way to train a highly adaptable agent in environments that cannot be changed that much? There's nothing more frustrating that an agent learning to get really good in one environment and then forgetting absolutely everything in a new environment and acting like a new-born baby.
     
    Major_Alfred likes this.