Question max_step with multiple agents

I remember reading something a while ago, either on this forum or in the documentation, that you need to times the max step of each agent by the number of agents in the scene.

Is this correct? and why?

 

I'm trying to train an agent with other agents in the scene running on inference. I remember reading somewhere that either the max step or step count need to be adjusted to account for multiple agents in the scene. However, I can't find this in the documentation.

 

So there is a confusing variety of similar things ml-agents calls 'steps'. (I'm still figuring out all the relationships too so please correct me if any of these are wrong)(as of release_6)

• Physics step - Called one time on each agent per fixed timestep using

  FixedUpdate()

• Agent step - The same as a physics step, although potentially different because it can be incremented independently. Corresponds to an atomic change of the engine that happens between Agent decisions. Accessible using

  agentObject.StepCount

  (will get only that agent's current step count within the current episode)
• Agent Max Step - The most steps an agent can possibly take before being automatically reset and starting a new episode, agent steps in an episode may be less than this if the agent is reset earlier via code (for example falling of the platform). Accessible using

  agentObject.MaxStep

• Academy (Environment) steps - Total steps by all agents within unity are accessible using

  Academy.Instance.TotalStepCount

  or for all steps only in the current episode

  Academy.Instance.StepCount

With that understanding I don't see a reason you would need to multiply max step by the number of agents unless you were trying to figure out how often a policy update occurs or something.

 

Ok great, I think I understand. Thanks for all the info.

I wish I could remember where I came across the multiplication of step by the number of
agents - it was one of the developers working on the ML-Agents.