Manually requesting for a decision from the agent

Hello everyone,


I have an agent where it will only execute the OnActionReceived(float[] vectorAction) if a condition variable holds true. So it would be something as follows:

OnActionReceived(float[] vectorAction)
{
if(condition)
{
//execute
}
}

I had the automatic stepping enabled and used a Decision Requester on the agent GameObject. However, I realized that this is not the best solution available because a lot of times the decisions may be requested and then the OnActionReceived is called but the condition variable holds false which makes this meaningless.

So I came across the following link:
https://forum.unity.com/threads/how-do-i-manually-call-collectobservations-and-agentaction.834385/

After reading this link I did the following inside the script which inherits from the Agent Class:

1. inside FixedUpdate() check if the condition is true
2. if it is true then do the following{
   RequestDecision();
   Academy.Instance.EnvironmentStep();
   }

The question is:

Considering that I will train several agents simultaneously and each agent will have its own condition variable (because not all agents will be ready to take a decision at the same time)

Does calling the Academy.Instance.EnvironmentStep() in one agent that had its condition=true, mean that the academy will interrupt the current step that the other agents are currently at? Would it interrupt the (Observation-Action-Reward) cycle of the other agents? Say the other agents where actually taking a decision in that same frame (i.e. they were in the cycle of Observation-Action-Reward) that the Academy.Instance.EnvironmentStep() was called by another agent, would it ruin anything for these agents' learning process?

Additionally, is it correct for me to check the condition in the FixedUpdate()? I am doing this because I read that an academy step is actually called by the FixedUpdate().

Finally, am I approaching this in the correct way or am I missing something?

Thank you for the time you took to read this.

 

I am also interested in the answer to your question. For now, the way I implemented this, is that I do not call EnvironmentStep() until I know that all of my agents have finished their move.

You don't need to call EnvironmentStep() in FixedUpdate(), since you call it on demand. It doesn't need to happen at any particular time.

 

Yes I think that is one way to get around this.
Thank you for the suggestion mate.