Search Unity

Bug Agent not performing actions

Discussion in 'ML-Agents' started by Jakopro, Dec 3, 2023.

  1. Jakopro

    Jakopro

    Joined:
    May 30, 2021
    Posts:
    18
    I have a simple agent script attached to an object as in the image:
    upload_2023-12-3_20-47-11.png

    The goal of the agent would be to navigate the map and reach the ping cube. The agent is a ball controlled with velocity. I tested the heuristics and they worked perfectly. I am not getting any errors. However, when I start the mlagents in command prompt (it doesn't show any errors neither) and click play in my Unity project, my agent doesn't appear to move at all. I find it very confusing because heuristics work.

    Does anyone have any idea about what went wrong? I would really appreciate some help.

    My code:
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using Unity.MLAgents;
    5. using Unity.MLAgents.Actuators;
    6. using Unity.MLAgents.Sensors;
    7.  
    8. public class EnemyAgent : Agent
    9. {
    10.     public GameObject player;
    11.  
    12.     public int numRays = 20;
    13.     public float maxRayDistance = 100f;
    14.  
    15.     public Vector3 v3forceX;
    16.     public Vector3 v3forceY;
    17.  
    18.     public override void OnEpisodeBegin()
    19.     {
    20.         player = GameObject.Find("MLtarget");
    21.  
    22.         transform.position = new Vector3(34.2361298f, 10.85267067f, 0.0426945575f);
    23.         GetComponent<Rigidbody>().velocity = Vector3.zero;
    24.         GetComponent<Rigidbody>().angularVelocity = Vector3.zero;
    25.  
    26.         GetComponent<Rigidbody>().velocity += v3forceY;
    27.     }
    28.  
    29.     public override void CollectObservations(VectorSensor sensor)
    30.     {
    31.         sensor.AddObservation(transform.position);
    32.         sensor.AddObservation(player.gameObject.transform.position);
    33.  
    34.         sensor.AddObservation(GetComponent<Rigidbody>().velocity);
    35.         //sensor.AddObservation(player.gameObject.GetComponent<Rigidbody>().velocity);
    36.  
    37.         for (int i = 0; i < numRays; i++)
    38.         {
    39.             float angle = (360f / numRays) * i;
    40.             Vector3 direction = Quaternion.Euler(0, angle, 0) * transform.forward;
    41.  
    42.             if (Physics.Raycast(transform.position, direction, out RaycastHit hit, maxRayDistance))
    43.             {
    44.                 // Normalize the distance to a value between 0 and 1
    45.                 float normalizedDistance = hit.distance / maxRayDistance;
    46.                 sensor.AddObservation(normalizedDistance);
    47.  
    48.                 Debug.DrawLine(transform.position, hit.point, Color.red);
    49.             }
    50.             else
    51.             {
    52.                 // If no collision, set distance to 1 (max distance)
    53.                 sensor.AddObservation(1f);
    54.             }
    55.         }
    56.     }
    57.  
    58.     public override void OnActionReceived(ActionBuffers actions)
    59.     {
    60.         float moveX = actions.ContinuousActions[0];
    61.         float moveY = actions.ContinuousActions[1];
    62.  
    63.         GetComponent<Rigidbody>().velocity += moveY * v3forceY;
    64.         GetComponent<Rigidbody>().velocity += moveX * v3forceX;
    65.     }
    66.  
    67.     public override void Heuristic(in ActionBuffers actionsOut)
    68.     {
    69.         ActionSegment<float> continuousActions = actionsOut.ContinuousActions;
    70.  
    71.         continuousActions[0] = Input.GetAxisRaw("Horizontal");
    72.         continuousActions[1] = Input.GetAxisRaw("Vertical");
    73.     }
    74.  
    75.     private void OnTriggerEnter(Collider other)
    76.     {
    77.         if(other.tag == "Floor")
    78.         {
    79.             SetReward(-1f);
    80.         }
    81.         else
    82.         {
    83.             SetReward(1f);
    84.         }
    85.        
    86.         EndEpisode();
    87.     }
    88. }
    89.  
     
  2. smallg2023

    smallg2023

    Joined:
    Sep 2, 2018
    Posts:
    144
    it looks like it should be ok as you have a decision requester and the actions should be ok, have you done any debugging to see when the episodes start and end and what actions the agent is trying to perform etc?
    something that might be breaking it is the frame rate difference, perhaps the trigger is "triggering" unexpectedly due to physics issues at higher training speeds
     
  3. Jakopro

    Jakopro

    Joined:
    May 30, 2021
    Posts:
    18
    Thanks for replying! I have done some debugging and episode doesn't seem to end unexpectedly. I also tried giving some starting velocity to the agent and the agent did move in that direction a bit meaning everything is working except the fact that the agent doesn't perform any actions.

    Does my command prompt output help?
    upload_2023-12-4_11-40-45.png
     
  4. smallg2023

    smallg2023

    Joined:
    Sep 2, 2018
    Posts:
    144
    so what does moveX and moveY report after you have got the continuous actions in the onactionsrecieved function?
     
  5. Jakopro

    Jakopro

    Joined:
    May 30, 2021
    Posts:
    18
    Sorry for taking so long to reply, both moveX and moveY report zero.
     
  6. Jakopro

    Jakopro

    Joined:
    May 30, 2021
    Posts:
    18
    I finally made it work by installing a newer version of python and mlagents python. Thanks for helping!
    For anyone having the same issue, I used python 3.9.13.