Search Unity

What are steps exactly?

Discussion in 'ML-Agents' started by Moonwilles, May 12, 2020.

  1. Moonwilles

    Moonwilles

    Joined:
    Apr 4, 2019
    Posts:
    10
    Hello :)

    Sorry for making a thread for something which I assume is so simple, but what exactly are steps?
    What is the correct definition of a step? I've tried looking at the documentation but was unable to find a proper definition of a step.

    Thanks!
     
    Moonwilles, May 12, 2020
    #1
    LohQ likes this.

  2. SurfingNerd

    SurfingNerd

    Joined:
    Apr 20, 2019
    Posts:
    8
    SurfingNerd, May 12, 2020
    #2

  3. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    There are actually two types of steps that are tracked by C# and python.

    On the C# side, there are the steps that are 1:1 with a fixed update call i.e. this step counter is incremented in the Agent every time a fixed update occurs. The 'max step' field in the agent script corresponds to the maximum number of these steps that an agent can exist for in an environment.

    Then, on the python side, there are are environment steps, each of which correspond to a `decision interval` (set in the decision requester component on an agent) elapsing. A decision interval is a set number of fixed updates (i.e. 5) that occur between an agent collecting an observation/changing its action. The 'max_steps' specified in the YAML config files corresponds to these steps.

    So, if you have a max step=3000 set in your agent script with a decision interval=5, the max number of environment steps per episode is 600. The reason these are on different counters is because python/C# have different cadences. Sorry, I know this is confusing. Let me know if I can clarify anything.

    TLDR;
    C# Step = 1 fixed update
    Python Step = 1 decision interval = k fixed updates
     
    andrewcoh_unity, May 14, 2020
    #3
    wikiwow, maxkcy, hk1ll3r and 6 others like this.

  4. Moonwilles

    Moonwilles

    Joined:
    Apr 4, 2019
    Posts:
    10
    I now have a better understanding! Thank you!
     
    Moonwilles, May 18, 2020
    #4

  5. Pr0gr1m

    Pr0gr1m

    Joined:
    Sep 11, 2021
    Posts:
    6
    Is there a way to count steps?
     
    Pr0gr1m, Jul 13, 2022
    #5

  6. hk1ll3r

    hk1ll3r

    Joined:
    Sep 13, 2018
    Posts:
    88
    Academy counts the C# steps. You can access them in code like this:
    Academy.Instance.StepCount // episode step count
    Academy.Instance.TotalStepCount // total training session step count
    Agent.StepCount // same as first line for agents that live throughout the episode

    You can also count your steps in fixedupdate calls of your agent or environment.
     
    hk1ll3r, Jul 18, 2022
    #6
    wikiwow likes this.