Resolved Giving an enum as an observation in CollectObservations()

I would like to give the state of the agent as an observation. My enum is as follows:

Code (CSharp):

1.  public enum agentStatus
2.     {
3.         HEALTHY,
4.         INFECTED,
5.         RECOVERED
6.     }

I saw that in the FoodCollecterAgent example booleans are used to give the observation as below:

Code (CSharp):

1. sensor.AddObservation(System.Convert.ToInt32(m_Frozen));
2. sensor.AddObservation(System.Convert.ToInt32(m_Shoot));

What is the reason to convert booleans toInt32(). Should I do something similar for my enum ?

For example:

Code (CSharp):

1. sensor.AddObservation(agentStatus));

 

System.Convert.ToInt32(bool) just returns 1 for true and 0 for false. Those values are then converted to float observations by VectorSensor. For observing enums, you could either use one-hot float encoding by having three observables for each enum state respectively. Set 1 for the currently active state and 0 (or -1) for the remaining two. Or you can try encoding the state in a single float observation, like e.g. HEALTHY = 1, RECOVERED = 0 and INFECTED = -1. Since you only have three states, the latter should work fine. But it might get harder for the agent to discern a larger number of states, if they're all encoded in a single float value.

 

Yep ... one-hot encoding is basically a way to do booleans with numbers as the neural network ultimately takes arrays of floats as input, whether it's vector obsevations, visual observations or some custom values like in your case.

 

You're right that the int conversion is pointless. PR to remove it is here: https://github.com/Unity-Technologies/ml-agents/pull/4169

One other option for enums is to use an ObservableAttribute: https://github.com/Unity-Technologi...gn-Agents.md#observable-fields-and-properties
This will handle the one-hot encoding, and also is smart about enums that aren't sequential (like if you had

RECOVERED=1337

).

 

So it resolved by its own. Turns out enums are encoded in the first place. Using this Debugging:

Code (CSharp):

1. Debug.Log("agentStatus: " + System.Convert.ToInt32(m_InfectionStatus));

I got:



That means it is doing the encoding by itself.

 

Yes, it will convert the enum to a integer, but as @mbaske and @andrzej_ mentioned, one-hot encoding is a better way to handle the observation. See https://github.com/Unity-Technologi...s.md#one-hot-encoding-categorical-information for more information.

 

After you telling me I understood better. That documentation is gem. Therefore I changed my observation as follows:

Code (CSharp):

1. public enum agentStatus{ HEALTHY, INFECTED, RECOVERED } //3 different state
2. const int NUM_ITEM_TYPES = (int)agentStatus.RECOVERED; //The last state in the enum (returns 2)
3.  
4. //Inside of the CollectObservation function:
5. sensor.AddOneHotObservation((int)m_InfectionStatus, NUM_ITEM_TYPES);

 

That looks better, but I think you want

NUM_ITEM_TYPES = 3