Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Noisy Entropy

Discussion in 'ML-Agents' started by mateolopezareal, Jun 28, 2020.

  1. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    I am geeting these noisey entropies, I think that is because of beta, but how much has to decrease more or less the entropy in 1 Million steps? Captura.PNG
    The image is in 5Million steps, so I imagine I have to deacrease beta, but until when?
     
  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    Hi, is this using PPO or SAC?

    For PPO, beta should decay as long as you've set your
    learning_rate_schedule
    to linear. You can try lowering beta.

    What do your value and policy losses look like? Your agent might be having trouble converging.