Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

agent starts to rotate in the axis that was not specified

Discussion in 'ML-Agents' started by koopagua, Jul 20, 2022.

  1. koopagua

    koopagua

    Joined:
    Jul 2, 2022
    Posts:
    4
    Hey,

    I'm new to Unity and ml-agents, and currently is making an cube agent moving toward a fixed target using dqn.
    I set 6 discrete actions which is same as the official pushblock task. It was running okay in the beginning, but with the progress of the learning, the agent starts to rotate in x and z axis which was not specified. I'm not sure what the problem is. Plz help!
    Thanks in advance and have a nice day!

    Here is the code in OnActionReceived:

    public float runSpeed=3;
    public override void OnActionReceived(ActionBuffers actionBuffers)
    {
    var movement = Vector3.zero;
    var rotate = Vector3.zero;
    var action = actionBuffers.DiscreteActions[0];

    switch (action)
    {

    case 1:
    movement = transform.forward * 1f;
    break;
    case 2:
    movement = transform.forward * -1f;
    break;
    case 3:
    rotate = transform.up *1f;
    break;
    case 4:
    rotate = transform.up *-1f;
    break;
    case 5:
    movement = transform.right *-0.75f;
    break;
    case 6:
    movement = transform.right *0.75f;
    break;
    }
    transform.Rotate(rotate, Time.fixedDeltaTime * 200f);
    rBody.AddForce(movement * runSpeed * Time.fixedDeltaTime, ForceMode.VelocityChange);
     
    Last edited: Jul 20, 2022
  2. koopagua

    koopagua

    Joined:
    Jul 2, 2022
    Posts:
    4
    I think the problem is by applying the same action in a row, meaning applying the same force in many times, the agent will be able to be lifted up due to its mass.

    Plz let me know if there are other reasons related to this thread!
    Thanks!
     
    Last edited: Jul 20, 2022
  3. smallg2023

    smallg2023

    Joined:
    Sep 2, 2018
    Posts:
    103
    you can tick the freeze rotation boxes for the x and z in the rigidbody window in the inspector for your object if you don't need those rotations at all.
    if you want to reduce the number of times the actions happen adjust the decision requester so that it only does 1 action per step etc
     
  4. koopagua

    koopagua

    Joined:
    Jul 2, 2022
    Posts:
    4
    hey Gary,

    Thanks for the reply. I kept the freeze rotation of x and z ticked during learning, and also reduced the decision period to 1.
    This behavior is still occuring :(
     
  5. koopagua

    koopagua

    Joined:
    Jul 2, 2022
    Posts:
    4
    I managed to tune the RL hyperparameter and the capacity of nn, so the learning would not converge to strange local optima for repeating the same action all the time. Now everything is good!
    Thanks :)