Resolved No action in OnActionReceived with Instantiate Prefab Agent

Hi. I have a model that has an agent which I want to take continues steps in the x/z direction on a plane. I have trained other models with this same code in different projects with no problem. The difference here is that I am instantiating the agent in the environment script from prefab that has the behavior script attached. Is it not possible to instantiate the agent automatically from a separate script?

The agent is instantiated and shows up in my training area, but upon training w/SAC or using heuristic, it does not move.

If I print ControlSignal in line 51, the value is 0. Also I tried setting the behavior type to default for training and heuristic when I was using the keyboard. Neither produce any action.

Here is my code for the agent.

Code (CSharp):

1. using System.Collections.Generic;
2. using UnityEngine;
3. using Unity.MLAgents;
4. using Unity.MLAgents.Sensors;
5.  
6. public class bunAgent : Agent
7. {
8.  
9.     private Rigidbody rBody;
10.     private bool isDead;
11.     public float bunSpeed = 10;
12.  
13.     // Start is called before the first frame update
14.     void Start()
15.     {
16.         rBody = GetComponent<Rigidbody>();
17.     }
18.  
19.     public override void OnEpisodeBegin() //how to "birth" the agent and target
20.     {
21.         if (this.transform.localPosition.y < -5f)
22.         {
23.             // If the Agent fell, zero its momentum
24.             this.rBody.angularVelocity = Vector3.zero;
25.             this.rBody.velocity = Vector3.zero;
26.             this.transform.localPosition = new Vector3(0, 1f, 0);
27.         }
28.     }
29.  
30.  
31.  
32.     public override void CollectObservations(VectorSensor sensor)
33.     {
34.         // // Target and Agent positions
35.         // sensor.AddObservation(Target.localPosition);
36.         // sensor.AddObservation(this.transform.localPosition);
37.         // // Agent velocity
38.         // sensor.AddObservation(rBody.velocity.x);
39.         // sensor.AddObservation(rBody.velocity.z);
40.     }
41.  
42.  
43.  
44.  
45.     public override void OnActionReceived(float[] vectorAction)
46.     {
47.         // Actions, size = 2
48.         Vector3 controlSignal = Vector3.zero;
49.         controlSignal.x = vectorAction[0];
50.         controlSignal.z = vectorAction[1];
51.         rBody.AddForce(controlSignal * bunSpeed);
52.  
53.         // Rewards
54.         SetReward(.01f);
55.  
56.         // Fell off platform
57.         if (this.transform.localPosition.y < -5)
58.         {
59.             SetReward(-1f);
60.             EndEpisode();
61.         }
62.     }
63.  
64.  

Here is my code that is instantiating the agent (a bit simplified to keep it short).

Code (CSharp):

1. using Unity.MLAgents;
2. using TMPro;
3. using System.Collections;
4. using System.Collections.Generic;
5. using UnityEngine;
6. using System;
7.  
8. public class forestEnv : MonoBehaviour
9. {
10.  
11.     [Header("Prefabs")]
12.     public GameObject Wild_rabbit_prefab;
13.  
14.     [Header("Hyperparameters")]
15.     public int numBunnies;
16.     public float spawnRange;
17.  
18.     private List<GameObject> spwndBunnies;
19.     private List<Tuple<Vector3, float>> occupiedPositions;
20.  
21.  
22.     // Start is called before the first frame update
23.     void Start()
24.     {
25.       ResetArea();
26.     }
27.  
28.     void ResetArea()
29.     {
30.       occupiedPositions = new List<Tuple<Vector3, float>>();
31.       ResetBunnies();
32.     }
33.  
34.     private void ResetBunnies()
35.     {
36.       if (spwndBunnies != null)
37.       {
38.         foreach (GameObject spwndBun in spwndBunnies.ToArray())
39.         {
40.           Destroy(spwndBun);
41.         }
42.       }
43.       spwndBunnies = new List<GameObject>();
44.       for (int i=0; i < numBunnies; i++)
45.       {
46.         GameObject bunInstance = Instantiate(Wild_rabbit_prefab, transform);
47.         RandomlyPlaceObject(bunInstance, spawnRange, 5);
48.         spwndBunnies.Add(bunInstance);
49.  
50.       }
51.     }
52.  

 

It turns out you need kinematic in the Rigidbody to be off.