Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Question The ml-agent doesn't move, the arrow controls don't even work.

Discussion in 'ML-Agents' started by MaggotDA, Apr 10, 2024.

  1. MaggotDA

    MaggotDA

    Joined:
    Dec 24, 2023
    Posts:
    5
    Hello!
    I'm learning to use the ML-Agents tool and I'm facing some problem...
    I wrote a class called "MoveToGoalAgent" in which the Agent (Cube) moves to the goal (Capsule). The "Heuristic" class tracks button clicks (arrows) and passes values to "ActionBuffers". In theory, my Agent should move by pressing keys (arrows) if I launch the project even without training the model, but this does not happen.
    My character only spawn (Rigidbody are on), fall to the panel and stand. Even if I start training model (in the hope that something will change): change on "Behavior type: Default" and running console "mlagents-learn --force --run-id=test1" - still nothing moves.

    Сode "MoveToGoalAgent":
    Code (CSharp):
    1. using System;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using System.Numerics;
    5. using Unity.MLAgents;
    6. using Unity.VisualScripting;
    7. using Unity.MLAgents.Sensors;
    8. using UnityEngine;
    9. using static UnityEngine.GraphicsBuffer;
    10. using UnityEngine.UIElements;
    11. using Unity.MLAgents.Actuators;
    12.  
    13. public class MoveToGoalAgent : Agent
    14. {
    15.     [SerializeField] private Transform targetTransform;
    16.     public override void OnEpisodeBegin()
    17.     {
    18.         transform.position = new UnityEngine.Vector3(0f,1f,0f);
    19.         //Y-axis = 1f to prevent the Agent from falling through the platform
    20.     }
    21.     public override void CollectObservations(VectorSensor sensor)
    22.     {
    23.         sensor.AddObservation(transform.position);
    24.         sensor.AddObservation(targetTransform.position);
    25.     }
    26.     public override void OnActionReceived(ActionBuffers actions)
    27.     {
    28.         float moveX = actions.ContinuousActions[0];
    29.         float moveZ = actions.ContinuousActions[1];
    30.         float moveSpeed = 1f;
    31.         transform.position += new UnityEngine.Vector3(moveX, 0, moveZ) * Time.deltaTime * moveSpeed;
    32.  
    33.         Debug.Log(moveX);
    34.         Debug.Log(moveZ);
    35.     }
    36.  
    37.     private void OnTriggerEnter(Collider other)
    38.     {
    39.         if (other.TryGetComponent<Wall>(out Wall wall))
    40.         {
    41.             SetReward(-1f);
    42.             Debug.Log("-1 reward");
    43.         }
    44.         if (other.TryGetComponent<Goal>(out Goal goal))
    45.         {
    46.             SetReward(+1f);
    47.             Debug.Log("+1 reward");
    48.             EndEpisode();
    49.         }
    50.     }
    51.     public override void Heuristic(in ActionBuffers actionsOut)
    52.     {
    53.         Debug.Log("Heuristic method work...");
    54.         ActionSegment<float> continuousActions = actionsOut.ContinuousActions;
    55.         continuousActions[0] = Input.GetAxisRaw("Horizontal");
    56.         continuousActions[1] = Input.GetAxisRaw("Vertical");
    57.     }
    58. }
    59.  
    Objects on scene:
    upload_2024-4-10_15-17-34.png

    Inspector of Agent (Cobe):
    upload_2024-4-10_15-40-15.png

    I’m trying to solve this problem in every possible way: I changed the weight of the object and its speed, but still the Agent does not move. I suspect that the Heuristic method does not work or that the ActionBuffers variables do not work correctly.
    ----------------------------------------------------------------
    Versions:
    ml-agents: 1.0.0,
    ml-agents-envs: 1.0.0,
    Communicator API: 1.5.0,
    PyTorch: 2.2.2+cu121

    CUDA: 12.4
    ----------------------------------------------------------------

    Thank you in advance!
     
  2. smallg2023

    smallg2023

    Joined:
    Sep 2, 2018
    Posts:
    154
    looks like you're not requesting a decision/action anywhere, even for heurisitics to work you will need to do so
    (i.e. attach decision requester component or call the relevant commands in your own control script(s))
     
    MaggotDA likes this.
  3. MaggotDA

    MaggotDA

    Joined:
    Dec 24, 2023
    Posts:
    5
    What kind of component is this? Which field should I enter in?
     
  4. smallg2023

    smallg2023

    Joined:
    Sep 2, 2018
    Posts:
    154
    it's a script that is supplied with MLA so you just attach it to your gameobject along with your agent script
     
    MaggotDA likes this.
  5. MaggotDA

    MaggotDA

    Joined:
    Dec 24, 2023
    Posts:
    5
    What is the name of this script and how to connect it to the project? Sorry that I don’t understand so much about ML-Agents.
     
  6. smallg2023

    smallg2023

    Joined:
    Sep 2, 2018
    Posts:
    154
    it's called "decision requester", you just add it as a new component to your gameobject - i.e. click your agent gameobject, press "add new component" and type "decision requester" (it'll already be in the project as part of ml-agents install)
     
    MaggotDA likes this.
  7. MaggotDA

    MaggotDA

    Joined:
    Dec 24, 2023
    Posts:
    5
    Oh my god, thank you so much!

    I didn’t even expect that only one script was missing...
    Unity should update its documentation.