Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. We are looking for feedback on the experimental Unity Safe Mode which is aiming to help you resolve compilation errors faster during project startup.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Curiosity Value Estimate keeps increasing

Discussion in 'ML-Agents' started by Procuste, Jul 31, 2020.

  1. Procuste

    Procuste

    Joined:
    Feb 10, 2020
    Posts:
    11
    Hello,

    I've been training an agent to do a physic-related task in Unity using mlagents-learn, and as the task I'm interested about has a sparse-reward setup, I tried using curiosity.
    What I observed during the training using curiosity is that the quantity Curiosity Value Estimate keeps increasing (goes from 0 at the beginning of training to 4e4 at about 1M steps). The curiosity reward slowly drops to 0, and the agent doesn't seem to explore the environment.

    Any suggestions about what I'm doing wrong ?
     
  2. Procuste

    Procuste

    Joined:
    Feb 10, 2020
    Posts:
    11
    Ok so after some digging, I found out that the observation the agent sees is very incomplete relative to the task it needs to perfom. I will try adding more information in the observation and see if this was the cause of the problem observed. (It wouldn't surprise me if it is as an incomplete observation can't determine the futur, thus that's why the agent seemed forever curious)
     
unityunity