Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved Recurrent Warning: Fewer observations made than vector observation size

Discussion in 'ML-Agents' started by usaidpro, Mar 16, 2022.

  1. usaidpro

    usaidpro

    Joined:
    Feb 14, 2018
    Posts:
    13
    Hello,

    I am trying to implement a custom Agent in a custom environment where it has to navigate to a preset goal using its Transform position. However, I am getting the "Fewer observations (0) made than vector observation size (3). The observations will be padded." warning at a rate that seems to be every other observation.

    For example, if I log the expected observation once per Agent.CollectObservations(), after an episode I find 334 logged observations and 330 "fewer observation" warnings. Nothing in the observation is null - no objects are being instantiated/destroyed at runtime and the logging shows a consistent Vector3 observation.

    Here is the warning call stack:
    Code (Boo):
    1. Fewer observations (0) made than vector observation size (3). The observations will be padded.
    2. UnityEngine.Debug:LogWarningFormat (string,object[])
    3. Unity.MLAgents.Sensors.VectorSensor:Write (Unity.MLAgents.Sensors.ObservationWriter) (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Sensors/VectorSensor.cs:56)
    4. Unity.MLAgents.Policies.HeuristicPolicy:StepSensors (System.Collections.Generic.List`1<Unity.MLAgents.Sensors.ISensor>) (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Policies/HeuristicPolicy.cs:133)
    5. Unity.MLAgents.Policies.HeuristicPolicy:RequestDecision (Unity.MLAgents.AgentInfo,System.Collections.Generic.List`1<Unity.MLAgents.Sensors.ISensor>) (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Policies/HeuristicPolicy.cs:38)
    6. Unity.MLAgents.Agent:SendInfoToBrain () (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Agent.cs:1096)
    7. Unity.MLAgents.Agent:SendInfo () (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Agent.cs:1323)
    8. Unity.MLAgents.Academy:EnvironmentStep () (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Academy.cs:573)
    9. Unity.MLAgents.AcademyFixedUpdateStepper:FixedUpdate () (at Library/PackageCache/com.unity.ml-agents@c1b26d49e4/Runtime/Academy.cs:43)

    Here is my Agent.CollectObservations():
    Code (CSharp):
    1. public override void CollectObservations(VectorSensor sensor)
    2. {
    3.     // agentPos and goalDestination are preset and do not get deleted. Nothing here indicates Null or 0.0 when logging
    4.     sensor.AddObservation((agentPos.position - goalDestination.transform.position) / (goalRadius + 750.0f));
    5. }
     
    Last edited: Mar 16, 2022
  2. ChillX

    ChillX

    Joined:
    Jun 16, 2016
    Posts:
    145
    Does this log any errors ?

    Code (CSharp):
    1. public override void CollectObservations(VectorSensor sensor)
    2. {
    3.     // agentPos and goalDestination are preset and do not get deleted. Nothing here indicates Null or 0.0 when logging
    4.     try {
    5.     sensor.AddObservation((agentPos.position - goalDestination.transform.position) / (goalRadius + 750.0f));
    6.     } catch (Exception ex)
    7.     {
    8.          Debug.LogError(ex, this);
    9.     }
    10. }
     
  3. usaidpro

    usaidpro

    Joined:
    Feb 14, 2018
    Posts:
    13
    Adding the Exception/LogError above does not log any errors.
     
  4. ChillX

    ChillX

    Joined:
    Jun 16, 2016
    Posts:
    145
    Try add a breakpoint at the end of CollectObservations.
    Once it hits the breakpoint add a watch on Sensor.m_Observations.count
    (Private variable so have to use quickwatch to find it and add it)

    Next run let it run a few times and each time it hits the breakpoint check if Sensor.m_Observations.count == 3
     
    usaidpro likes this.
  5. usaidpro

    usaidpro

    Joined:
    Feb 14, 2018
    Posts:
    13
    Interesting. Looks like there may be a bug. Adding the breakpoint with a conditional checking if
    sensor.m_Observations.Count != 3
    never triggers. However, if I do
    sensor.m_Observations.Count == 3
    it does trigger correctly (since I have a Vector3 = 3-size observation).

    edit: Is it possible that there is some other observation size check before the overrided
    Agent.CollectObservations()
    is called?
     
  6. usaidpro

    usaidpro

    Joined:
    Feb 14, 2018
    Posts:
    13
    I got rid of the warning! It had nothing to do with my Agent script. Instead, it had something to do with the components I had on the GameObject. I reordered the components to match the picture on the ml-agents docs (picture at the "Final Agent Setup" section) and removed an extraneous VectorSensor component (which I do not think had any impact since it was set to size 2 but no errors/warnings) and it worked with no warnings!

    If anyone wants, I could reorder my components and try to figure out the exact cause of the warning. Now to test if this was impacting training - I have been experiencing really weird training where the agent learns but at a rate extremely slow for a simple problem (like just solving what essentially was a direct linear equation). Maybe the observations were being dropped right before writing to the buffer.

    If my training issues are solved with this and there had been some observation dropping, then I think there should be a specific warning/error about this. It was really weird to debug this because my
    Agent.CollectObservations()
    was returning the correct size. I'll mark this thread as Resolved in a few days if still training issues since then it's not related to the warning.

    edit: Still have training issues. Guess basic continuous pathfinding to a goal is pretty hard for RL to do within 2M timesteps. Gonna try curriculum learning and other things, but it is safe to assume that the warning is unrelated to my RL issues. Marking as resolved.
     
    Last edited: Mar 18, 2022