Question Should using the BufferSensor to record varying numbers of other agents include self as an entry?

I am using a BufferSensor to record a varying number of enemies, which are themselves agents. So each agent gets a buffer/list of all the other agents' info.

Should each agent also include their own info in the buffer? Then, each agent can be given a unique id in their regular (vector sensor) observations, which is included in the buffer entry, allowing them to determine which entry in the buffer is themselves. Or should their own info be kept separate, as a regular, non-buffersensor observation (vector sensor)?

 

I think the question is "Should the agent know where it is, or only where the other agents are?" It's multiple instances of a single agent. If you add the observation of "location" each instance would know where every agent is including itself. If you didn't add it, then all the agents would know about the other agents, but not itself.

 

Yes, the agents definitely need to know info about themselves: their stats, how many cards they have, etc. What I am wondering though is WHERE to put the info about themselves. Should it go in the BufferSensor along with the other agents' info (with an ID in their VectorSensor observations so they know which is themselves), or should their own info go in VectorSensor while all others are in BufferSensor?

 

I could be wrong, but I don't believe it matters. I think the main difference in Buffer vs. Vector sensors is Vector is a defined size, where as the buffer sensor is variable length. So if there is 5 opponents, and always 5, you could use a vector. If the number of opponents is variable, you would need the buffer vector.

In the end I think ALL observations are flattened out to be inputs to the neural network, with no concern given to "where" they came from. For example the 3d ray sensors data would also be fed in. So, with that said, it probably should just go where you put the other ones just for consistency.

 

Buffer sensor observations pass through an attention layer before reaching the policy network so I wouldn't put anything the can't potentially be discarded in one. The attention layer learns which observations are most useful in the long run so the agent will train either way but you'd probably be slowing down training unless the added complexity is required.