Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Resolved Giving an enum as an observation in CollectObservations()

Discussion in 'ML-Agents' started by Hsgngr, Jun 29, 2020.

  1. Hsgngr

    Hsgngr

    Joined:
    Dec 28, 2015
    Posts:
    31
    I would like to give the state of the agent as an observation. My enum is as follows:

    Code (CSharp):
    1.  public enum agentStatus
    2.     {
    3.         HEALTHY,
    4.         INFECTED,
    5.         RECOVERED
    6.     }
    I saw that in the FoodCollecterAgent example booleans are used to give the observation as below:

    Code (CSharp):
    1. sensor.AddObservation(System.Convert.ToInt32(m_Frozen));
    2. sensor.AddObservation(System.Convert.ToInt32(m_Shoot));
    What is the reason to convert booleans toInt32(). Should I do something similar for my enum ?

    For example:
    Code (CSharp):
    1. sensor.AddObservation(agentStatus));
     
  2. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    231
    System.Convert.ToInt32(bool) just returns 1 for true and 0 for false. Those values are then converted to float observations by VectorSensor. For observing enums, you could either use one-hot float encoding by having three observables for each enum state respectively. Set 1 for the currently active state and 0 (or -1) for the remaining two. Or you can try encoding the state in a single float observation, like e.g. HEALTHY = 1, RECOVERED = 0 and INFECTED = -1. Since you only have three states, the latter should work fine. But it might get harder for the agent to discern a larger number of states, if they're all encoded in a single float value.
     
    Hsgngr likes this.
  3. andrzej_

    andrzej_

    Joined:
    Dec 2, 2016
    Posts:
    74
    Yep ... one-hot encoding is basically a way to do booleans with numbers as the neural network ultimately takes arrays of floats as input, whether it's vector obsevations, visual observations or some custom values like in your case.
     
    Hsgngr likes this.
  4. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    Hsgngr likes this.
  5. Hsgngr

    Hsgngr

    Joined:
    Dec 28, 2015
    Posts:
    31
    So it resolved by its own. Turns out enums are encoded in the first place. Using this Debugging:

    Code (CSharp):
    1. Debug.Log("agentStatus: " + System.Convert.ToInt32(m_InfectionStatus));
    I got:

    upload_2020-6-29_19-44-20.png

    That means it is doing the encoding by itself.
     
  6. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    Hsgngr likes this.
  7. Hsgngr

    Hsgngr

    Joined:
    Dec 28, 2015
    Posts:
    31
    After you telling me I understood better. That documentation is gem. Therefore I changed my observation as follows:
    Code (CSharp):
    1. public enum agentStatus{ HEALTHY, INFECTED, RECOVERED } //3 different state
    2. const int NUM_ITEM_TYPES = (int)agentStatus.RECOVERED; //The last state in the enum (returns 2)
    3.  
    4. //Inside of the CollectObservation function:
    5. sensor.AddOneHotObservation((int)m_InfectionStatus, NUM_ITEM_TYPES);
     
  8. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    147
    That looks better, but I think you want
    NUM_ITEM_TYPES = 3
     
unityunity