Search Unity

  1. All Pro and Enterprise subscribers: find helpful & inspiring creative, tech, and business know-how in the new Unity Success Hub. Sign in to stay up to date.
    Dismiss Notice
  2. Dismiss Notice

Help Wanted Why is curriculum learning designed to NOT reset the threshold average after a lesson change?

Discussion in 'ML-Agents' started by mcdenyer, Jun 6, 2021.

  1. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    20
    Was hoping someone could answer why on curriculum learning the Threshold is calculated using the average of all episodes and not just the episodes of the current lesson.

    This really screws up any curriculum that does not increase the threshold from lesson to lesson but rather decreases as I am trying to.

    I am trying to train to an agent to complete a task with fewer checkpoints which guide it from point a to point b so the cumulative reward is fewer. Why is curriculum learning designed for higher and higher amounts of rewards? Am I missing something?
     
    Last edited: Jun 7, 2021
  2. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    157
    The reward buffer should be cleared after the lesson is updated. Is this not happening for you?
     
    mcdenyer likes this.
  3. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    20
    I had a lesson where the reward threshold dropped significantly from the previous lesson. The lesson then went to the next lesson as soon as it had run min_lesson_length episodes. I double checked my curriculum config file to make sure I didn't have a typo and it was all correct. This must be something I did. Is there a way to retrieve the threshold average that is being calculated by the curriculum?
     
  4. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    157
    mcdenyer likes this.
  5. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    20
    It must be my mistake. I will try to replicate and post. Thank you for the help.
     
unityunity