Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

No visible gizmos for Ray Perception Sensor Component 3D

Discussion in 'ML-Agents' started by mdelisle, Feb 8, 2020.

  1. mdelisle

    mdelisle

    Joined:
    Feb 23, 2014
    Posts:
    3
    Hi,

    I'm not getting any gizmos / debug drawing in the editor for my Ray Perception Sensor Component 3D and I can't figure what I'm doing wrong.
    I do see regular raycast debug drawings I use for ground checks so I can only assume it's a problem with my setup here and not a general gizmo issue.

    Here is a screenshot of the editor view with the game running:
    upload_2020-2-8_17-33-25.png

    I do have the Use Child Sensors box checked:
    upload_2020-2-8_17-34-30.png

    Any idea what could be causing the issue?
     
  2. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    That's pretty strange. Do they display in the provided example scenes? WallJump and PushBlock are two examples that should have them. If that doesn't work, I'll try your version of the editor on Monday.

    Note that they only draw during play mode; there's still an open feature request to draw them outside of play mode.
     
  3. mdelisle

    mdelisle

    Joined:
    Feb 23, 2014
    Posts:
    3
    Thanks for your help!

    I tried in the WallJump scene as you advised and realized the rays are displayed, but only when running the real training. When I press play without the environment running, no raycasts are displayed.

    When trying the same thing on my scene, I can also see the debug drawing of the rays as I train my agent. Now the problem is that if this is the only way to get a preview of the rays, it is going to be difficult to tweak the positioning, as the game runs at high speed and as it's a little heavy to restart the environment every time I want to test an angle change.

    Do you have any advice on how to test without having to run the environment at the same time?
     
  4. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    Sorry, I don't have any other recommendations right now. We only create the
    RayPerceptionSensor (the thing that actually does the raycasts) when the Agent starts running, and the drawing code currently only works by processing its runtime results. This is something we can improve in the future, but don't have support at the moment.
     
  5. mdelisle

    mdelisle

    Joined:
    Feb 23, 2014
    Posts:
    3
    Understood. Thanks for your answer!
     
  6. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    Just a followup on this - I just merged a https://github.com/Unity-Technologies/ml-agents/pull/3484 which (amongst other things) will draw the RayPerception gizmos in non-play mode when the Agent is selected. It's on the master branch now and will be in the next release (in a few weeks).
     
  7. luchi1916

    luchi1916

    Joined:
    Jul 19, 2020
    Posts:
    2
    So I have the latest version of MLAgents and it looks like despite the update I am facing the exact same issue as OP
     
  8. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    Make sure the Agent (or its parent GameObject) is selected and that you have the RayPerceptionSensorComponents selected in the gizmos menu:

    upload_2020-8-11_18-34-29.png
     
    luchi1916 likes this.
  9. luchi1916

    luchi1916

    Joined:
    Jul 19, 2020
    Posts:
    2
    Ahh thanks. Sorry for the ignorance. This worked
     
    celion_unity likes this.
  10. SuhandhanRavee

    SuhandhanRavee

    Joined:
    Oct 24, 2020
    Posts:
    1
    Hello @celion_unity,

    I would like to render these RaySensor SphereCasts in my game mode too. Is there a way to do this?
     
  11. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    Hi,
    The debug information class that we use for drawing the gizmos isn't public, so you won't be able to use it directly. You can recreate the raycasts by doing something like

    Code (CSharp):
    1.  
    2.         var raySensorComponent = GetComponent<RayPerceptionSensorComponent3D>();
    3.         var input = raySensorComponent.GetRayPerceptionInput();
    4.         var outputs = RayPerceptionSensor.Perceive(input);
    5.         for(var rayIndex = 0; rayIndex < outputs.RayOutputs.Length; rayIndex++)
    6.         {
    7.             var extents = input.RayExtents(rayIndex);
    8.             Vector3 startPositionWorld = extents.StartPositionWorld;
    9.             Vector3 endPositionWorld = extents.EndPositionWorld;
    10.             var rayOutput = outputs.RayOutputs[rayIndex];
    11.             if (rayOutput.HasHit)
    12.             {
    13.                 Vector3 hitPosition = Vector3.Lerp(startPositionWorld, endPositionWorld, rayOutput.HitFraction);
    14.                 Debug.DrawLine(startPositionWorld, hitPosition, Color.red);
    15.             }
    16.             else
    17.             {
    18.                 Debug.DrawLine(startPositionWorld, endPositionWorld, Color.blue);
    19.             }
    20.  
    21.         }