Search Unity

Bug Academy EpisodeCount always return 1

Discussion in 'ML-Agents' started by Luke-Houlihan, Sep 4, 2020.

  1. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
    When displaying some stats on a UI canvas during training/inference I noticed the EpisodeCount always returns 1, while other properties on the academy return correctly.

    Here is the UI display code I'm using -
    Code (CSharp):
    1. using Unity.MLAgents;
    2. using UnityEngine;
    3. using UnityEngine.UI;
    4.  
    5. public class AcademyStats : MonoBehaviour
    6. {
    7.     public Text Episode;
    8.     public Text TotalSteps;
    9.  
    10.     // Update is called once per frame
    11.     public void Update()
    12.     {
    13.         if (Academy.IsInitialized)
    14.         {
    15.             Episode.text = "Episode: " + Academy.Instance.EpisodeCount;
    16.             TotalSteps.text = "Total Steps: " + Academy.Instance.TotalStepCount;
    17.         }
    18.     }
    19. }
    20.  
    When run the UI looks like this -



    You can see the agent resetting itself and the episode's completed by this agent being incremented. On the right however the academy episode stays the same.

    Version information:
    ml-agents: 0.19.0 (release_6),
    ml-agents-envs: 0.19.0,
    Communicator API: 1.0.0,
    TensorFlow: 2.3.0
    Unity: 2020.2.0a21.2837
     
    Last edited: Sep 4, 2020
  2. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @Luke-Houlihan,
    The episode property on the Academy class is for environment level episode counts when an entire environment reset is triggered from python. If you want to display the episode count for an agent please use Agent.CompletedEpisodes.
     
  3. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
    Ah that makes more sense, thank you!

    A quick follow up, would the environment reset automatically in a curriculum change or must that be done manually?
     
  4. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
    A quick explanation for anyone else who checks this out, you can see in my gif (bottom left) that Agent01's episodes increment with each reset of the agent. My incorrect assumption was that the Academy EpisodeCount (bottom right) was an aggregate of all agent episodes within the environment (like the total step count is for steps). However the environment has its own episode count that gets incremented as explained by christophergoy.
     
  5. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Yes, it is done manually through the python API
     
  6. nathan60107

    nathan60107

    Joined:
    Apr 18, 2021
    Posts:
    5
    If I create 16 independent fields for agents to train. Do I have to sum up all 16 CompletedEpisodes to get the total episode number of my training?
    I think this number should be available in the academy, not calculate by the user. And EpisodeCount seems useless because it always returns 1, which we don't care about.
     
  7. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @nathan60107,
    Academy.Episode count is incremented when the entire environment is reset from python. This only happens in certain situations like when you use curriculum training. Generally speaking the Academy.OnEnvironmentReset can be invoked by the Unity Environment / Game depending on how the game implements its end state.

    You can refer to this thread to get further information about this topic.
     
    nathan60107 likes this.