Search Unity

  1. Unity 2019.4 has been released.
    Dismiss Notice
  2. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  3. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Noisy Entropy

Discussion in 'ML-Agents' started by mateolopezareal, Jun 28, 2020 at 5:23 PM.

  1. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    47
    I am geeting these noisey entropies, I think that is because of beta, but how much has to decrease more or less the entropy in 1 Million steps? Captura.PNG
    The image is in 5Million steps, so I imagine I have to deacrease beta, but until when?
     
  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    48
    Hi, is this using PPO or SAC?

    For PPO, beta should decay as long as you've set your
    learning_rate_schedule
    to linear. You can try lowering beta.

    What do your value and policy losses look like? Your agent might be having trouble converging.
     
unityunity