Does AgentReset() gets called when Max steps from the inspector has been reached?

Hello, I am a bit confused as to how are the agents and academy are being reset.

I have two conditions for resetting in the game I'm making using self-play

1: When all agents in a team are "killed" ie SetActive(false), I wrote a method which cycles through each agent and calls their agent.done() method. This condition works well and the agents are reactivated and moved to the current positions. AgentReset() method in the agent script has a "this.gameobject.setactive(true)" line.

2: i set the max steps to 500. However, when the max steps are reached, the agents do not reactivate. I tested this by "killing" one member of a team and leaving it there till it has reached its max steps. only those agents which are not deactivated move to the correct positions while the deactivated agents stay deactivated. Furthermore, it turns out that for the deactivated agents, when they have reached max steps, their agent reset methods are not being called at all.

It is strange that condition 1 works but condition 2 does not when to my understanding when the agent has reached its max step, it calls it's its own reset methods.

Can someone explain why cant my agents reset when it has reached max steps? thanks!

here is a snippet of my code:

This essentially checks condition 1 and resets all agents if it is met. this is not the agent script, its a separate monobehaviour script


this is the agent reset method in my agent script



I took reference from the soccer example hence some of the syntax and variables look familiar

 

ok so i managed to solve the issue but including this loop which checks at every fixedupdate() if each agent has reached its max step and if i has reset every agent:



not exactly an elegant solution as it keeps checking every frame, any ideas anyone?

 

(I'm assuming you're on the 0.14.1 release. If not, let me know and I'll find the relevant code).

This shouldn't be necessary - when the agent is stepped by the Academy, it will make the same check and mark the Agent done which will reset it too: https://github.com/Unity-Technologi...om.unity.ml-agents/Runtime/Agent.cs#L794-L798

However, if you deactivate an Agent, it will no longer get stepped by the Academy, so its step count won't increase:
https://github.com/Unity-Technologi...4.1/com.unity.ml-agents/Runtime/Agent.cs#L264
So maybe that's the cause of the confusion?

 

Hello, yeah I am on 0.14.1.

That explains why only the deactivated agents don't get reset after they reach their max step. its because their steps don't increase at all while being deactivated.

I understand whats going on now, thanks for the explanation and references to the code

 

Great, glad I could help!