Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

How to access RayPerceptionSensor3D data?

Discussion in 'ML-Agents' started by unity_nBMnid_GOQvclw, Mar 26, 2020.

  1. unity_nBMnid_GOQvclw

    unity_nBMnid_GOQvclw

    Joined:
    Feb 3, 2020
    Posts:
    5
    Can I access RayPerceptionSensor3D data from Agent class? I want to input it into a neural network implemented on C#.

    Thanks!
     
    unity_nBMnid_GOQvclw, Mar 26, 2020
    #1

  2. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @unity_nBMnid_GOQvclw,
    There isn't currently a way to get the data from the RayPerceptionComponent. But you can call Perceive directly and get the data back that way. Let me know if that covers your use case.
     
    christophergoy, Mar 26, 2020
    #2
    unity_nBMnid_GOQvclw likes this.

  3. LexVolkov

    LexVolkov

    Joined:
    Sep 14, 2014
    Posts:
    62
    And the truth is, there is no way to understand RayPerceptionComponent is working or not. There is no state tracking tool.
     
    LexVolkov, Mar 27, 2020
    #3

  4. AndrewGri

    AndrewGri

    Joined:
    Jan 30, 2020
    Posts:
    12
    if I call this directly, does it guarantee that the output matches to what RayPerceptionComponent passes to learning?
     
    AndrewGri, Mar 30, 2020
    #4

  5. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    289
    The output is equivalent but not in exactly the same form. You'd need to take the RayPerceptionOutput that Perceive() returns, and call RayOutput.ToFloatArray() for each RayOutput in RayPerceptionOutput.rayOutputs.
     
    celion_unity, Mar 30, 2020
    #5

  6. AndrewGri

    AndrewGri

    Joined:
    Jan 30, 2020
    Posts:
    12
    thank you! But how to fill input for Perceive(RayPerceptionInput input)? Can I take it from GetComponent<RayPerceptionSensorComponent3D>();?
     
    AndrewGri, Mar 30, 2020
    #6

  7. RealZST

    RealZST

    Joined:
    Feb 25, 2020
    Posts:
    5
    Hi. Do you know how to call 'Perceive' and fill the input now? I am facing the same problem. Looking forward to your reply. Thank you.
     
    RealZST, Apr 2, 2020
    #7

  8. AndrewGri

    AndrewGri

    Joined:
    Jan 30, 2020
    Posts:
    12
    not solved yet. Will try to dive deeper on weekends. In the same time I'd ask you to share solution if you find it earlier))
     
    AndrewGri, Apr 2, 2020
    #8

  9. RealZST

    RealZST

    Joined:
    Feb 25, 2020
    Posts:
    5
    I gave up and tried another way.
    In my case, I want to get the distance to the nearest obstacle to set the reward function. I gave up getting it from the RayPerceptionComponent. And use the "Physics.SphereCast" to access the data.
     
    RealZST, Apr 3, 2020
    #9

  10. celion_unity

    celion_unity

    Unity Technologies

    Joined:
    Jun 12, 2019
    Posts:
    289
    Yes, you should be able to use 
    GetComponent<RayPerceptionSensorComponent3D>();
    and then use the 
    GetRayPerceptionInput()
    method.

    This is assuming you're using 0.15.1 (or 0.15.0). In 0.14.0, you'd have to use the PerceiveStatic() which is the same basic idea but takes a lot of arguments instead of a single input structure, and doesn't give the outputs in as nice of a format.
     
    celion_unity, Apr 17, 2020
    #10
    unity_nBMnid_GOQvclw likes this.

  11. AndrewGri

    AndrewGri

    Joined:
    Jan 30, 2020
    Posts:
    12
    Thanks for the answer. Currently I implemented my own logic based on RayCast which lets my agent learn better then RayPerceptionSensor. Probably I get back to RayPerceptionSensor when it is more usable and gives more control.
     
    AndrewGri, Apr 19, 2020
    #11