Question Additional observation from RaySensor

I use RayPerceptionSensor2d. I have had some success in learning. But I have a few questions.
1) I'm not sure, but when ray encounters a detectable object, does it go no further? Judging by Gizmos it doesn't go any further. So I have to use 3 components of the ray sensor so that the agent sees the wall that is right behind the food?

2) Even though I looked at

https://github.com/Unity-Technologies/ml-agents/blob/08a66c4dc33340f4294643654bbe1dae74ca8ce7/com.unity.ml-agents/Runtime/Sensors/RayPerceptionSensor.cs#L164

but didn't really understand what kind of observations the Ray Sensor transmits to the python? I need to pass the following observations into python:

• Distance to object.
• The speed of the object (rigidbody2d)
• Weight of the object (rigidbody2d)

How can I do it? I would be very grateful for code examples

 

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Serialization;
3.  
4. namespace Unity.MLAgents.Sensors
5. {
6.     public class CustomRayPerceptionOutput : RayPerceptionOutput
7.     {
8.         public CustomRayPerceptionOutput()
9.         {
10.             CustomObservationSizePerRay = 3;
11.         }
12.  
13.         public override void GetCustomObservationData(RayOutput rayOutput, float[] buffer)
14.         {
15.             var rb = rayOutput.HitGameObject.GetComponent<Rigidbody>();
16.             if (rb != null)
17.             {
18.                 buffer[0] = rb.velocity.x;
19.                 buffer[1] = rb.velocity.z;
20.                 buffer[2] = rb.mass;
21.             }
22.         }
23.     }
24.  
25.     /// <summary>
26.     /// A component for Custom 3D Ray Perception.
27.     /// </summary>
28.     [AddComponentMenu("ML Agents/Custom Ray Perception Sensor 3D", (int)MenuGroup.Sensors)]
29.     public class RayPerceptionSensorComponentCustom3D : RayPerceptionSensorComponent3D
30.     {
31.         public override ISensor[] CreateSensors()
32.         {
33.             var rayPerceptionInput = GetRayPerceptionInput();
34.             var rayPerceptionOutput = new CustomRayPerceptionOutput();
35.  
36.             m_RaySensor = new RayPerceptionSensor(SensorName, rayPerceptionInput, rayPerceptionOutput);
37.  
38.             if (ObservationStacks != 1)
39.             {
40.                 var stackingSensor = new StackingSensor(m_RaySensor, ObservationStacks);
41.                 return new ISensor[] { stackingSensor };
42.             }
43.  
44.             return new ISensor[] { m_RaySensor };
45.         }
46.     }
47. }

Found this on the internet, if it worked in the project it would be very cool (but it doesn't work, no methods/fields like this). Who knows, maybe this is a new version? I don't really want to make a custom raycast when there is already a ready-made solution

 

you should be able to get the results with
https://docs.unity3d.com/Packages/c...ity.MLAgents.Sensors.RayPerceptionOutput.html

 

And how do I do that, an example would be greatly appreciated. Now I see a solution only to make changes to the package itself, which I do not really want to do for obvious reasons

 

bump

 

you need a reference to your perception ray (can pass this like a normal reference or get it with GetComponent etc)
i.e.

Code (CSharp):

1. [SerializeField]
2. RayPerceptionSensorComponent2D rayPerceptionSensor;

then in your CollectObservations function you can call something like

Code (CSharp):

1. // Get the ray perception output from the sensor
2.       RayPerceptionOutput rayPerceptionOutput = rayPerceptionSensor.RaySensor.RayPerceptionOutput;
3.       var rayOutputs = rayPerceptionOutput.RayOutputs;
4.  
5.       for (int i = 0; i < rayOutputs.Length; i++)
6.       {
7.         Debug.Log($"ray {i} hit something ? {rayOutputs[i].HasHit}");
8.         if (rayOutputs[i].HasHit)
9.         {
10.           Debug.Log($"hit gameobject = {rayOutputs[i].HitGameObject.name}");
11.         }
12.       }

so to get the info you asked for you can do something like this

Code (CSharp):

1. // Get the ray perception output from the sensor
2.       RayPerceptionOutput rayPerceptionOutput = rayPerceptionSensor.RaySensor.RayPerceptionOutput;
3.       RayPerceptionOutput.RayOutput[] rayOutputs = rayPerceptionOutput.RayOutputs;
4.  
5.       for (int i = 0; i < rayOutputs.Length; i++)
6.       {
7.         Debug.Log($"ray {i} hit something ? {rayOutputs[i].HasHit}");
8.         if (rayOutputs[i].HasHit)
9.         {
10.           Debug.Log($"hit gameobject = {rayOutputs[i].HitGameObject.name}");
11.           Debug.Log($"hit distance = {rayOutputs[i].HitFraction}");
12.           Rigidbody2D hitRB = rayOutputs[i].HitGameObject.GetComponentInParent<Rigidbody2D>();
13.           if (hitRB)
14.           {
15.             Debug.Log($"hit velo = {hitRB.velocity}");
16.             Debug.Log($"hit weight = {hitRB.mass}");
17.           }
18.         }
19.       }

(should just be a case of swapping the relevant commands to 3D if required)