Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ML Agents and New Input System

Discussion in 'ML-Agents' started by Adham9, Jul 3, 2020.

  1. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    I have 6 actions mapped to 6 inputs buttons using the new input system. However, when I add another action along with its button, the agents keep repeating the same first actions forever! I noticed that since I am using a heuristic brain. I use a "Decision Requester" component.

    I suspect it has something to do with RequestAction() running underneath although I never call this function in my code.

    Any suggestions to prevent repetitive behavior are welcome.
     
    Last edited: Jul 4, 2020
  2. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    Can anybody advise why the RequestAction() is being called by itself? What is the purpose of such behavior? and how can I stop it?
     
  3. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    Are you retraining the agent with the 7 actions or are you using the heuristic function from the previous version of the action space? Is the behavior type set to default or heuristic in the behavior parameters script?
     
  4. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    Thank you andrewcoh for replying.

    I am not in the training phase at all. I only use the heuristic function for now. The behavior type is set to "Heuristic".

    Here is my heuristic function:-



    public override void Heuristic(float[] actionsOut)
    {
    controls.Enable();

    // Control animation states and speed
    ///ControlAnimation();

    actionsOut[0] = controls.Player.Move.ReadValue<Vector2>()[0];
    actionsOut[1] = controls.Player.Move.ReadValue<Vector2>()[1];

    ///Listen to Sprint Button
    if (controls.Player.Sprint.ReadValue<float>() == 1)
    {
    actionsOut[2] = 5f;
    }

    if (possess_the_ball)
    {

    if (controls.Player.Pass.ReadValue<float>() == 1)
    {
    anim.Play("kick_near");
    actionsOut[2] = 1f;

    } else if (controls.Player.LongBall.ReadValue<float>() == 1)
    {
    actionsOut[2] = 2f;

    } else if (controls.Player.ThroughBall.ReadValue<float>() == 1)
    {
    actionsOut[2] = 3f;

    } else if (controls.Player.ShootsAtGoal.ReadValue<float>() == 1)
    {
    actionsOut[2] = 4f;
    anim.Play("kick_far");
    }

    }
    else if (possess_the_ball == false && controls.Player.Pass.ReadValue<float>() == 1)
    {
    actionsOut[2] = 6;
    }

    }


    Notice the final else. When I comment it and reduce the number of actions by 1, the actions keep repeating. and when I uncomment it and increase the number of actions by 1, the actions repeat occasionally.

    Also, when I replace the last action with (and add the binding)


    else if (possess_the_ball == false && controls.Player.Tackle.ReadValue<float>() == 1) {

    instead of


    else if (possess_the_ball == false && controls.Player.Pass.ReadValue<float>() == 1)

    it keeps repeating forever.

    I tried to remove the parent if statement to test, but the problem persists"
    if (possess_the_ball)
    {
    "
    I tried to clear the GI cache, but with no success.

    It seems to me as a Cache problem as the code does not want to be changed!
     
    Last edited: Jul 9, 2020
    Azhagarasan likes this.
  5. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    So, when you say '6 actions' do you mean you have 6 action branches i.e. the array actionsOut should contain an entry for actionsOut[0..5] or that you have fewer action branches but with 6 possible values i.e. actionsOut[0] = [1..6]. Can you screenshot your behavior parameters script?
     
  6. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    I attached a screenshot of the behavior parameters. I use this component on two opposing teams differentiating them with team ID 1 and 2.

    This is my OnActionReceived() function

    Code (CSharp):
    1.   public override void OnActionReceived(float[] vectorAction)
    2.     {
    3.  
    4.         // continously monitor last touch
    5.         last_touch_player_team();
    6.  
    7.  
    8.         if (IsEngagementInPassIOrShootDone())
    9.         {
    10.             isEngagedInPassShoot = false;
    11.         }
    12.         AttachBallToPlayer();
    13.         //Defend();
    14.         ToggleDefenderWinsToFalse();
    15.  
    16.         MoveThePlayer(new Vector2(vectorAction[0], vectorAction[1]));
    17.  
    18.  
    19.         switch (vectorAction[2])
    20.         {
    21.  
    22.             case 0: break;
    23.             case 1:
    24.                 try
    25.                 {
    26.                     Pass(targetPlayer.GetSiblingIndex());
    27.                 }
    28.                 catch (System.Exception ex)
    29.                 {
    30.                     print("ERROR:  " + ex);
    31.                 }
    32.                 break;
    33.             case 2:
    34.                 LongBall(targetPlayer.GetSiblingIndex());
    35.                 break;
    36.             case 3:
    37.                 ThoroughBall(targetPlayer.GetSiblingIndex());
    38.                 break;
    39.             case 4:
    40.                 Shoot(lookingDirection);
    41.                 //KickBall();
    42.                 break;
    43.             case 5:
    44.                 Sprint(new Vector2(vectorAction[0], vectorAction[1]));
    45.                 break;
    46.             case 6:
    47.                 Defend();
    48.                 break;
    49.         }
    50. }
     

    Attached Files:

  7. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    Ah, I see. Can you turn the decision frequency on the decision requester down to 1 and try playing your agent?
     
  8. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    I turned the decision requester to 1 with "Take Actions Between Decisions" to false, and also to true, but the problem persists.
     
  9. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    Problem solved! I added an else statement for action # 0 in the heuristic method. I did not know that was necessary. Thanks for helping anyway.