agent starts to rotate in the axis that was not specified

Hey,

I'm new to Unity and ml-agents, and currently is making an cube agent moving toward a fixed target using dqn.
I set 6 discrete actions which is same as the official pushblock task. It was running okay in the beginning, but with the progress of the learning, the agent starts to rotate in x and z axis which was not specified. I'm not sure what the problem is. Plz help!
Thanks in advance and have a nice day!

Here is the code in OnActionReceived:

public float runSpeed=3;
public override void OnActionReceived(ActionBuffers actionBuffers)
{
var movement = Vector3.zero;
var rotate = Vector3.zero;
var action = actionBuffers.DiscreteActions[0];

switch (action)
{

case 1:
movement = transform.forward * 1f;
break;
case 2:
movement = transform.forward * -1f;
break;
case 3:
rotate = transform.up *1f;
break;
case 4:
rotate = transform.up *-1f;
break;
case 5:
movement = transform.right *-0.75f;
break;
case 6:
movement = transform.right *0.75f;
break;
}
transform.Rotate(rotate, Time.fixedDeltaTime * 200f);
rBody.AddForce(movement * runSpeed * Time.fixedDeltaTime, ForceMode.VelocityChange);

 

I think the problem is by applying the same action in a row, meaning applying the same force in many times, the agent will be able to be lifted up due to its mass.

Plz let me know if there are other reasons related to this thread!
Thanks!

 

you can tick the freeze rotation boxes for the x and z in the rigidbody window in the inspector for your object if you don't need those rotations at all.
if you want to reduce the number of times the actions happen adjust the decision requester so that it only does 1 action per step etc

 

hey Gary,

Thanks for the reply. I kept the freeze rotation of x and z ticked during learning, and also reduced the decision period to 1.
This behavior is still occuring

 

I managed to tune the RL hyperparameter and the capacity of nn, so the learning would not converge to strange local optima for repeating the same action all the time. Now everything is good!
Thanks