Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Question MLAgent CollectObservations() disrupts agents movement

Discussion in 'ML-Agents' started by davogevo0416, Mar 9, 2023.

  1. davogevo0416

    davogevo0416

    Joined:
    Jan 8, 2022
    Posts:
    1
    Hey there,
    I have encountered a problem while trying to design an agent that would follow an Object. Agent works fine until i add any observation to it. After that it starts to show the same behaviour in every episode. The observations that i make are the agent position and the goalObject position.

    Here is my code

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. using Unity.MLAgents;
    6. using Unity.MLAgents.Actuators;
    7. using Unity.MLAgents.Sensors;
    8.  
    9. public class FishAgent : Agent
    10. {
    11.     [Range(0f, 1f)]
    12.     public float a;
    13.  
    14.     [Range(-1f, 1f)]
    15.     public float t, c;
    16.  
    17.     public int reward = 0;
    18.  
    19.     [SerializeField] private Transform targetTransform;
    20.  
    21.     public override void OnEpisodeBegin()
    22.     {
    23.         transform.position = new Vector3(515, 94, 597);
    24.     }
    25.  
    26.     private void FixedUpdate()
    27.     {
    28.         MoveFish(a, t, c);
    29.     }
    30.  
    31.     public override void CollectObservations(VectorSensor sensor)
    32.     {
    33.         sensor.AddObservation(transform.position);
    34.         sensor.AddObservation(targetTransform.position);
    35.     }
    36.  
    37.  
    38.  
    39.     public override void OnActionReceived(ActionBuffers actions)
    40.     {
    41.         a = (actions.ContinuousActions[0] + 1)/2;
    42.         t = actions.ContinuousActions[1];
    43.         c = actions.ContinuousActions[2];
    44.     }
    45.  
    46.     private void OnCollisionEnter(Collision other)
    47.     {
    48.         if (other.gameObject.layer == 11)
    49.         {
    50.             SetReward(1f);
    51.             EndEpisode();
    52.             reward++;
    53.         }
    54.         else if (other.gameObject.layer == 6)
    55.         {
    56.            reward--;
    57.            SetReward(-1f);
    58.             EndEpisode();
    59.         }
    60.     }
    61.  
    62.     private Vector3 inp;
    63.     public void MoveFish(float v, float h, float s)
    64.     {
    65.         inp = Vector3.Lerp(Vector3.zero, new Vector3(0, 0, v * 3.5f), 0.02f);
    66.         inp = transform.TransformDirection(inp);
    67.         transform.position += inp;
    68.  
    69.         transform.eulerAngles += new Vector3(0, (h * 90) * 0.02f, 0);
    70.         transform.eulerAngles += new Vector3((s * 90) * 0.02f, 0, 0);
    71.     }
    Here are my Behaviour parameters

    upload_2023-3-9_0-58-43.png

    Any help will be appreciated.