Search Unity

Question max_step with multiple agents

Discussion in 'ML-Agents' started by StewedHarry, Sep 7, 2020.

  1. StewedHarry

    StewedHarry

    Joined:
    Jan 20, 2020
    Posts:
    45
    I remember reading something a while ago, either on this forum or in the documentation, that you need to times the max step of each agent by the number of agents in the scene.

    Is this correct? and why?
     
  2. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @StewedHarry,
    In what context to do you want to do this? I’m not sure what you’re trying to accomplish. Could you elaborate?
     
  3. StewedHarry

    StewedHarry

    Joined:
    Jan 20, 2020
    Posts:
    45
    I'm trying to train an agent with other agents in the scene running on inference. I remember reading somewhere that either the max step or step count need to be adjusted to account for multiple agents in the scene. However, I can't find this in the documentation.
     
  4. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
    So there is a confusing variety of similar things ml-agents calls 'steps'. (I'm still figuring out all the relationships too so please correct me if any of these are wrong)(as of release_6)
    • Physics step - Called one time on each agent per fixed timestep using
      FixedUpdate()
    • Agent step - The same as a physics step, although potentially different because it can be incremented independently. Corresponds to an atomic change of the engine that happens between Agent decisions. Accessible using
      agentObject.StepCount
      (will get only that agent's current step count within the current episode)
    • Agent Max Step - The most steps an agent can possibly take before being automatically reset and starting a new episode, agent steps in an episode may be less than this if the agent is reset earlier via code (for example falling of the platform). Accessible using
      agentObject.MaxStep
    • Academy (Environment) steps - Total steps by all agents within unity are accessible using
      Academy.Instance.TotalStepCount
      or for all steps only in the current episode
      Academy.Instance.StepCount
    With that understanding I don't see a reason you would need to multiply max step by the number of agents unless you were trying to figure out how often a policy update occurs or something.
     
  5. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Max Step on the Agent will inform the agent of how many steps it can take in an episode. The summary values output by the python process are the addition of all of the agent steps in the current environments. So when you see a summary at 2000 steps, for example, that 2000 steps was calculated by adding all of the steps of each agent in the active environments together.

    In short, you should not have to do any complex math on the agent max step based on the number of agents.
     
  6. StewedHarry

    StewedHarry

    Joined:
    Jan 20, 2020
    Posts:
    45
    Ok great, I think I understand. Thanks for all the info.

    I wish I could remember where I came across the multiplication of step by the number of
    agents - it was one of the developers working on the ML-Agents.