How can I use RayPerceptionComponent3D effectively?

Hi, I downloaded Karting Microgame and am trying to learn how to use ML- Agents package.
I updated ml- agents package to 1.0.2 and realized that it contains RayPerceptionComponent3D component for ray observations. I wonder how to reedit this CollectObservation function using this component?

Code (CSharp):

1. public override void CollectObservations()
2.         {
3.             AddVectorObs(kart.LocalSpeed());
4.  
5.             // Add an observation for direction of the agent to the next checkpoint.
6.             var next          = (checkpointIndex + 1) % Colliders.Length;
7.             var nextCollider  = Colliders[next];
8.             var direction     = (nextCollider.transform.position - kart.transform.position).normalized;
9.             AddVectorObs(Vector3.Dot(kart.Rigidbody.velocity.normalized, direction));
10.  
11.             if (ShowRaycasts)
12.             {
13.                 Debug.DrawLine(AgentSensorTransform.position, nextCollider.transform.position, Color.magenta);
14.             }
15.  
16.             for (int i = 0; i < Sensors.Length; i++)
17.             {
18.                 var current = Sensors[i];
19.                 var xform   = current.Transform;
20.                 var hit     = Physics.Raycast(AgentSensorTransform.position, xform.forward, out var hitInfo,
21.                     RaycastDistance, Mask, QueryTriggerInteraction.Ignore);
22.  
23.                 if (ShowRaycasts)
24.                 {
25.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance, Color.green);
26.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance * current.HitThreshold,
27.                         Color.red);
28.                 }
29.  
30.                 var hitDistance = (hit ? hitInfo.distance : RaycastDistance) / RaycastDistance;
31.                 AddVectorObs(hitDistance);
32.  
33.                 if (hitDistance < current.HitThreshold) {
34.                     AddReward(HitPenalty);
35.                     Done();
36.                     AgentReset();
37.                 }
38.             }
39.         }

I think there is no need to raycasts anymore when I use rayperceptioncomponent3D but
I don't know how to make this control if I need :
if (hitDistance < current.HitThreshold)

I have not found any code examples of this component, I would be glad if you could help.

 

No need to code if you use the RayPerceptionSensor3D Component. If you attach one to your agent all you need is configure the parameters in the editor; no need to add any code in the script since it will treat the raycasting as an observation by itself.

 

Okay, I understood that we don't need to collect observations of raycasts anymore.
But how to process these observations?
For example, we have to reset agent if it follows wrong raycast and crash, or we want to add reward if it follows correct raycast.
But now we can not do that.
I would be glad if you can explain with an example.

 

You are now in reward engineering. You can code that if the agent hits specific colliders then it receives -1 as a reward and then reset. Otherwise, you could give +1 if it reaches its final destination.

Or give it small rewards +0.01 or +0.001 when it reaches checkpoints and saves the +1 to the end.

It really depends on what you want to achieve.

 

We are doing what you say already. But we also want to use raycasts to process,
you can see this method used in unity example project's code that I shared first :

Code (CSharp):

1. for (int i = 0; i < Sensors.Length; i++)
2.             {
3.                 var current = Sensors[i];
4.                 var xform   = current.Transform;
5.                 var hit     = Physics.Raycast(AgentSensorTransform.position, xform.forward, out var hitInfo,
6.                     RaycastDistance, Mask, QueryTriggerInteraction.Ignore);
7.                 if (ShowRaycasts)
8.                 {
9.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance, Color.green);
10.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance * current.HitThreshold,
11.                         Color.red);
12.                 }
13.                 var hitDistance = (hit ? hitInfo.distance : RaycastDistance) / RaycastDistance;
14.                 AddVectorObs(hitDistance);
15.                 if (hitDistance < current.HitThreshold) {
16.                     AddReward(HitPenalty);
17.                     Done();
18.                     AgentReset();
19.                 }
20.             }

On the conclusion, we want to learn how to use this component and its parameters, events, features etc. for using in our future projects. There is no enough detailed knowledge about this. We would be glad if you can share detailed document with us.

Thank you.