OnActionReceived() order of execution with respect to MonoBehaviour

Hi,
My agent have a script that manipulate it's movement during FixedUpdate (CatDynamics.cs). I have another script (CatAgent.cs) that inherits from Agent and therefore, have OnActionRecieved(). I want the actions received in CatAgent.cs (by policy or heuristic) that are stored in variables to propagate to CatDynamics.cs by reading these variables in CatDynamics.cs during the FixedUpdate().

Basically the order I want is:
OnActionReceived() (update variable in CatAgent.cs ) -> FixedUpdate() in CayDynmics.cs (Read the variables from CatAgent.cs)

During the same loop,
Does OnActionReceived() happens before any other MonoBehaviour FixedUpdate calls?

Thank you

 

Have you considered doing managed updates? I usually propagate all agent actions top-down from OnActionReceived. So in your case, CatAgent would have a reference to CatDynamics and you would replace CatDynamics' FixedUpdate method with a custom update method you'd call from OnActionReceived (which itself is invoked on FixedUpdate steps). I find this approach to be pretty solid, since it ensures a clear execution order. Also, you can pass down any action values directly this way and don't need to worry about storing them somewhere.

 

@henrypeteet thank you for taking the time to answer my question. As you recommended I am trying to avoid "breaking" the flow, as not knowing the internal of the system will sure make me break something there...
@mbaske , it is a great idea and I will probably use it, although I wished for a solution which doesn't require to change how the basic "cat" works (not using CatDynamics fixedupdate or so...)

 

Hi all

@henrypeteet @mbaske

I have an agent that runs actions in external software which return the result of these actions. This is done with callbacks.

So the logic is as follows:

Initialiaze()

{

// Send values to ext. software for the first time only just so I get a current state of values

}

OnEpisodeBegin()

{

// Randomize values to be sent to external software

// Randomize target value

}

OnActionReceived()

{

//Get the values from the action array, send them over. When the external calculations are done, a callback is triggered and I receive the result and Set the rewards

}

Of course I have the CollectObservations() and Heuristic() as well.
My logic is that I should 1) RequestDesicion() at the end of OnActionReceived or 2) Make AutomaticStepping=false in Intialize() and RequestDesicion(); EnvironmentStep(); at the end of OnActionReceived

None of the two works. Any Advice?

 

This usually works for me. If you put RequestDesicion() in your callback handler and the callback is triggered reliably, that should keep the agent loop running.

 

@mbaske Thank you for replying.
When I don't use the DesicionRequester and RequestDecisions manually my machine falls in something like an infinite loop and I have to shut down unity.
Also what a bout the stepping? When I turn it to false in Initialize(), then OnEpisodeBegin() never gets triggered.
Finally how do you combine the above with EndEpisode()?
Is there a chance you could share some example, or even some pseudo?

 

You can try it with the 3DBall example project. Remove the Decision Requester component. Then add RequestDecision(); to the agent's OnActionReceived and to the OnEpisodeBegin methods. RequestDecision(); should be the last line in both method bodies.

 

@mbaske I was told by a member of Unity Technologies that:

"RequestDecision should be called outside of Agent methods. For example, you cannot call it inside CollectObservations or OnActionReceive. In the latest release there should be some guardrails to prevent these recursions you mentioned."

 

Ah, good to know. Looking at the Agent code (API 1.2.0), calling RequestDecision() sets the m_RequestDecision flag, but doesn't seem to invoke anything right away. At least I didn't encounter any recursions yet, when I requested new decisions from within OnActionReceived.