Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Noisy Entropy

Discussion in 'ML-Agents' started by mateolopezareal, Jun 28, 2020.

  1. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    I am geeting these noisey entropies, I think that is because of beta, but how much has to decrease more or less the entropy in 1 Million steps? Captura.PNG
    The image is in 5Million steps, so I imagine I have to deacrease beta, but until when?
     
  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    Hi, is this using PPO or SAC?

    For PPO, beta should decay as long as you've set your
    learning_rate_schedule
    to linear. You can try lowering beta.

    What do your value and policy losses look like? Your agent might be having trouble converging.