Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

How can I use RayPerceptionComponent3D effectively?

Discussion in 'ML-Agents' started by Only1here, Jul 6, 2020.

  1. Only1here

    Only1here

    Joined:
    Mar 15, 2018
    Posts:
    3
    Hi, I downloaded Karting Microgame and am trying to learn how to use ML- Agents package.
    I updated ml- agents package to 1.0.2 and realized that it contains RayPerceptionComponent3D component for ray observations. I wonder how to reedit this CollectObservation function using this component?

    Code (CSharp):
    1. public override void CollectObservations()
    2.         {
    3.             AddVectorObs(kart.LocalSpeed());
    4.  
    5.             // Add an observation for direction of the agent to the next checkpoint.
    6.             var next          = (checkpointIndex + 1) % Colliders.Length;
    7.             var nextCollider  = Colliders[next];
    8.             var direction     = (nextCollider.transform.position - kart.transform.position).normalized;
    9.             AddVectorObs(Vector3.Dot(kart.Rigidbody.velocity.normalized, direction));
    10.  
    11.             if (ShowRaycasts)
    12.             {
    13.                 Debug.DrawLine(AgentSensorTransform.position, nextCollider.transform.position, Color.magenta);
    14.             }
    15.  
    16.             for (int i = 0; i < Sensors.Length; i++)
    17.             {
    18.                 var current = Sensors[i];
    19.                 var xform   = current.Transform;
    20.                 var hit     = Physics.Raycast(AgentSensorTransform.position, xform.forward, out var hitInfo,
    21.                     RaycastDistance, Mask, QueryTriggerInteraction.Ignore);
    22.  
    23.                 if (ShowRaycasts)
    24.                 {
    25.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance, Color.green);
    26.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance * current.HitThreshold,
    27.                         Color.red);
    28.                 }
    29.  
    30.                 var hitDistance = (hit ? hitInfo.distance : RaycastDistance) / RaycastDistance;
    31.                 AddVectorObs(hitDistance);
    32.  
    33.                 if (hitDistance < current.HitThreshold) {
    34.                     AddReward(HitPenalty);
    35.                     Done();
    36.                     AgentReset();
    37.                 }
    38.             }
    39.         }
    I think there is no need to raycasts anymore when I use rayperceptioncomponent3D but
    I don't know how to make this control if I need :
    if (hitDistance < current.HitThreshold)

    I have not found any code examples of this component, I would be glad if you could help.
     
  2. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    No need to code if you use the RayPerceptionSensor3D Component. If you attach one to your agent all you need is configure the parameters in the editor; no need to add any code in the script since it will treat the raycasting as an observation by itself.
     
  3. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    The observations collected by a raycast sensor is added internally to the agent's observations which has the raycast script attached. There is no coding necessary and nothing to account for in the behavior parameters script.
     
  4. Only1here

    Only1here

    Joined:
    Mar 15, 2018
    Posts:
    3
    Okay, I understood that we don't need to collect observations of raycasts anymore.
    But how to process these observations?
    For example, we have to reset agent if it follows wrong raycast and crash, or we want to add reward if it follows correct raycast.
    But now we can not do that.
    I would be glad if you can explain with an example.
     
  5. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    You are now in reward engineering. You can code that if the agent hits specific colliders then it receives -1 as a reward and then reset. Otherwise, you could give +1 if it reaches its final destination.

    Or give it small rewards +0.01 or +0.001 when it reaches checkpoints and saves the +1 to the end.

    It really depends on what you want to achieve.
     
  6. Only1here

    Only1here

    Joined:
    Mar 15, 2018
    Posts:
    3
    We are doing what you say already. But we also want to use raycasts to process,
    you can see this method used in unity example project's code that I shared first :

    Code (CSharp):
    1. for (int i = 0; i < Sensors.Length; i++)
    2.             {
    3.                 var current = Sensors[i];
    4.                 var xform   = current.Transform;
    5.                 var hit     = Physics.Raycast(AgentSensorTransform.position, xform.forward, out var hitInfo,
    6.                     RaycastDistance, Mask, QueryTriggerInteraction.Ignore);
    7.                 if (ShowRaycasts)
    8.                 {
    9.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance, Color.green);
    10.                     Debug.DrawRay(AgentSensorTransform.position, xform.forward * RaycastDistance * current.HitThreshold,
    11.                         Color.red);
    12.                 }
    13.                 var hitDistance = (hit ? hitInfo.distance : RaycastDistance) / RaycastDistance;
    14.                 AddVectorObs(hitDistance);
    15.                 if (hitDistance < current.HitThreshold) {
    16.                     AddReward(HitPenalty);
    17.                     Done();
    18.                     AgentReset();
    19.                 }
    20.             }
    On the conclusion, we want to learn how to use this component and its parameters, events, features etc. for using in our future projects. There is no enough detailed knowledge about this. We would be glad if you can share detailed document with us.

    Thank you.