Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Question Manually Requesting Decision in a loop

Discussion in 'ML-Agents' started by GeorgGrech, Mar 11, 2023.

  1. GeorgGrech

    GeorgGrech

    Joined:
    Nov 26, 2021
    Posts:
    6
    Hi.

    I have a list of of objects that I'm populating at runtime. I wanna be able to iterate through this list one by one, observe the objects properties, and then the agent decides whether it wants to take an action or check the next one.

    Below is the CollectObservations() method, so you can see what variables I'm observing

    Code (CSharp):
    1.     public override void CollectObservations(VectorSensor sensor)
    2.     {
    3.         try
    4.         {
    5.             if (targetTransform != null)
    6.             {
    7.                 sensor.AddObservation(targetPDistanceNormalised);
    8.                 sensor.AddObservation(targetBDistanceNormalised);
    9.                 sensor.AddOneHotObservation((int)targetType, numOfTypes);
    10.                 sensor.AddObservation(targetInvAmountLeftNormalised);
    11.  
    12.             }
    13.  
    14.             sensor.AddObservation((float)enemyPlayer.inventoryAmountFree / enemyPlayer.maxInventorySize); //Keep track of inventory
    15.         }
    16.  
    17.         catch
    18.         {
    19.             Debug.Log("Exception caught in observations");
    20.         }
    21.     }
    Below is the method which goes through the list and request a decision. I want this method to go through the list one by one and request a decision for each object, starting again if it goes through the entire list. If the agent takes a specific action, the loop should break, otherwise it tries again with the next object.

    Code (CSharp):
    1.    
    2. bool resourceSelecting;
    3. public void IterateAndChooseFromList(float minPDistance, float maxPDistance, float minBDistance, float maxBDistance)
    4.     {
    5.         resourceSelecting = true;
    6.         while (resourceSelecting)
    7.         {
    8.             foreach (ResourceData resourceData in resourcesTrackingList)
    9.             {
    10.                 Debug.Log("Observing next resource");
    11.                 targetTransform = resourceData.resourceObject;
    12.                 targetType = resourceData.type;
    13.                 targetPDistanceNormalised = (resourceData.distanceFromPlayer - minPDistance) / (maxPDistance - minPDistance);
    14.                 targetBDistanceNormalised = (resourceData.distanceFromPlayer - minBDistance) / (maxBDistance - minBDistance);
    15.                 targetInvAmountLeftNormalised = resourceData.invAmountLeft / enemyPlayer.maxInventorySize;
    16.  
    17.                 RequestDecision();
    18.  
    19.                 if (!resourceSelecting)
    20.                 {
    21.                     Debug.Log("Resource Selected. Breaking... 1");
    22.                     break;
    23.                 }
    24.             }
    25.         }  
    26.     }
    And here is the OnActionRecieved method to get an idea of the actions. When actionIndex is 0 or 1 the list checking should break, when it's 2, it should continue and check the next object.

    Code (CSharp):
    1.     public override void OnActionReceived(ActionBuffers actionBuffers)
    2.     {
    3.         int actionIndex = actionBuffers.DiscreteActions[0];
    4.  
    5.         if (actionIndex == 0 || actionIndex == 1)
    6.         {
    7.             resourceSelecting = false;
    8.  
    9.             if (actionIndex == 0) //0 means returns to base
    10.             {
    11.                 Debug.Log("Return to base");
    12.                 StartCoroutine(ReturnToBase());
    13.             }
    14.  
    15.             else if (actionIndex == 1)
    16.             {
    17.                 Debug.Log("Gathering resource");
    18.                 StartCoroutine(GatherResource(targetTransform));
    19.             }
    20.         }
    21.  
    22.         else //else if Action Index == 2, keep checking
    23.         {
    24.             Debug.Log("Check next resource.");
    25.         }
    26.     }
    Currently, this code hangs the project because of the while loop, but even if I remove that, from the debug messages I can see that the foreach doesn't pause and wait for an action when RequestDecision() is called.

    I'm not sure how else I can go about this. Any ideas would be appreciated!

    Thanks!
     
    Last edited: Mar 11, 2023