Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Which optimizer ML-Agents uses in general?

Discussion in 'ML-Agents' started by YodoLuca, Apr 1, 2023.

  1. YodoLuca

    YodoLuca

    Joined:
    May 16, 2018
    Posts:
    10
    Hello!

    I read in an old same topic's question that ML-Agents uses Adam as optimizer. I would like to discover more information about that:
    - Is adam used for all the supported main trainers? (ppo, sac and poca)
    - Can someone point me to the file in the repo where the optimizer is being declared and initialized?

    The first question is related to a message that appears when you resume the training of a network using the same configuration file but with different trainer type: for example, if we change from ppo to poca, it outputs that it can't find the adam optimizer status, while viceversa it says that can't find the value optimer status, so adam seems not there in poca, or hidden/wrapped in something else.

    The second question is for an eventual idea of changing the optimizer toward sgd with nesterov, that in some cases performs better than adam.

    Thank you very much!
     
  2. MidnightGameDeveloper

    MidnightGameDeveloper

    Joined:
    Apr 26, 2014
    Posts:
    122
    Have you already found out which optimizer is used for poca?
     
  3. YodoLuca

    YodoLuca

    Joined:
    May 16, 2018
    Posts:
    10
    kokimitsunami and tjumma like this.
  4. MidnightGameDeveloper

    MidnightGameDeveloper

    Joined:
    Apr 26, 2014
    Posts:
    122