Proper Usage of Buffer Sensor?

Hi all,

I am currently trying to use a buffer sensor in a dogfighting game for the AI to learn to avoid enemy projectiles (amongst other things). One thing I'm not sure of is if I had set it up correctly.

I added the buffer sensor to the gameobject which has all of the ML-Agent scripts and settings (I have attached an image showing how that looks).
Then inside **CollectObservations**:

Code (CSharp):

1.  public override void CollectObservations(VectorSensor sensor)
2.     {
3.        //omitted code about other basic observations
4.         for (int i = 0; i < bulletRadar.bullets.Count; i++)
5.         {
6.  
7.             Transform tempTran = bulletRadar.bullets[i].GetComponent<Transform>();
8.             Rigidbody2D tempRigid = bulletRadar.bullets[i].GetComponent<Rigidbody2D>();
9.             //normalized: x, y, velX, velY rotation around Z
10.             float[] temp = { tempTran.localPosition.x / xLength, tempTran.localPosition.y / yLength,    tempRigid.velocity.x / maxVel, tempRigid.velocity.y / maxVel, (transform.rotation.z % 360f) / 360f };
11.             bufferSensor.AppendObservation(temp);
12.         }
13.     }

So my question, do I just need to add the bullets into the bufferSensor on each CollectObservation call and that's it? Will the output of that bufferSensor be properly handled by ML-agents stuff?

 

Hopefully, it's not against rules, I will try bumping this post just once instead of reposting...

 

I'm working with a similar setup, and it does appear this is all you need to do. I'm not sure how to think about the buffer itself, how often it fills, what happens to old observations and at what rate. Any insights?

 

I believe the buffer is cleared every ML-Agents step and that you have to refill it each time collect observations is called. For your given problem, ordering the entries w.r.t distance to your agent would help it understand "threat" as well as the attention mechanism (that the NN uses with buffer sensor).

You can also use the forum search function to find threads that are related to buffer sensor questions. It also helped me reading the source code for buffer sensor.

 

Doesn't order not matter for the BufferSensor?

 

Order does not matter

 

Order doesn't matter and yes you would need to fill the buffer each time. If you want to carry observations from one step to another you would need to use the Stacked Vectors (or maybe a recurring neural network).

 

Yes, ordering doesn't matter unless the network contains convolutions. My apologies.