ML Agents and New Input System

I have 6 actions mapped to 6 inputs buttons using the new input system. However, when I add another action along with its button, the agents keep repeating the same first actions forever! I noticed that since I am using a heuristic brain. I use a "Decision Requester" component.

I suspect it has something to do with RequestAction() running underneath although I never call this function in my code.

Any suggestions to prevent repetitive behavior are welcome.

 

Can anybody advise why the RequestAction() is being called by itself? What is the purpose of such behavior? and how can I stop it?

 

Thank you andrewcoh for replying.

I am not in the training phase at all. I only use the heuristic function for now. The behavior type is set to "Heuristic".

Here is my heuristic function:-

 

 public override void Heuristic(float[] actionsOut)

    {

        controls.Enable();



        // Control animation states and speed

        ///ControlAnimation();



        actionsOut[0] = controls.Player.Move.ReadValue<Vector2>()[0];

        actionsOut[1] = controls.Player.Move.ReadValue<Vector2>()[1];



        ///Listen to Sprint Button

        if (controls.Player.Sprint.ReadValue<float>() == 1)

        {

            actionsOut[2] = 5f;

        }



        if (possess_the_ball)

        {



            if (controls.Player.Pass.ReadValue<float>() == 1)

            {

                anim.Play("kick_near");

                actionsOut[2] = 1f;



            } else if (controls.Player.LongBall.ReadValue<float>() == 1)

            {

                actionsOut[2] = 2f;



            } else if (controls.Player.ThroughBall.ReadValue<float>() == 1)

            {

                actionsOut[2] = 3f;



            } else if (controls.Player.ShootsAtGoal.ReadValue<float>() == 1)

            {

                actionsOut[2] = 4f;

                anim.Play("kick_far");

            }

 

        }

     else if (possess_the_ball == false && controls.Player.Pass.ReadValue<float>() == 1)

        {

          actionsOut[2] = 6;

        }

 

    }

Notice the final else. When I comment it and reduce the number of actions by 1, the actions keep repeating. and when I uncomment it and increase the number of actions by 1, the actions repeat occasionally.

Also, when I replace the last action with (and add the binding)

else if (possess_the_ball == false && controls.Player.Tackle.ReadValue<float>() == 1) {

instead of

else if (possess_the_ball == false && controls.Player.Pass.ReadValue<float>() == 1)

it keeps repeating forever.

I tried to remove the parent if statement to test, but the problem persists"
if (possess_the_ball)
{
"
I tried to clear the GI cache, but with no success.

It seems to me as a Cache problem as the code does not want to be changed!

 

I attached a screenshot of the behavior parameters. I use this component on two opposing teams differentiating them with team ID 1 and 2.

This is my OnActionReceived() function

Code (CSharp):

1.   public override void OnActionReceived(float[] vectorAction)
2.     {
3.  
4.         // continously monitor last touch
5.         last_touch_player_team();
6.  
7.  
8.         if (IsEngagementInPassIOrShootDone())
9.         {
10.             isEngagedInPassShoot = false;
11.         }
12.         AttachBallToPlayer();
13.         //Defend();
14.         ToggleDefenderWinsToFalse();
15.  
16.         MoveThePlayer(new Vector2(vectorAction[0], vectorAction[1]));
17.  
18.  
19.         switch (vectorAction[2])
20.         {
21.  
22.             case 0: break;
23.             case 1:
24.                 try
25.                 {
26.                     Pass(targetPlayer.GetSiblingIndex());
27.                 }
28.                 catch (System.Exception ex)
29.                 {
30.                     print("ERROR:  " + ex);
31.                 }
32.                 break;
33.             case 2:
34.                 LongBall(targetPlayer.GetSiblingIndex());
35.                 break;
36.             case 3:
37.                 ThoroughBall(targetPlayer.GetSiblingIndex());
38.                 break;
39.             case 4:
40.                 Shoot(lookingDirection);
41.                 //KickBall();
42.                 break;
43.             case 5:
44.                 Sprint(new Vector2(vectorAction[0], vectorAction[1]));
45.                 break;
46.             case 6:
47.                 Defend();
48.                 break;
49.         }
50. }

 

I turned the decision requester to 1 with "Take Actions Between Decisions" to false, and also to true, but the problem persists.

 

Problem solved! I added an else statement for action # 0 in the heuristic method. I did not know that was necessary. Thanks for helping anyway.