Search Unity

Question How to add extra information from RayPerceptionSensor (Release 17)

Discussion in 'ML-Agents' started by WaxyMcRivers, May 27, 2021.

  1. WaxyMcRivers

    WaxyMcRivers

    Joined:
    May 9, 2016
    Posts:
    59
    Super happy to see that we can now get additional information from ray perception sensors using the RayPerceptionOutput.

    I'm looking to add information about the rigid body of a specific tagged object when its seen by a ray, do I have to manually handle the new number of inputs?

    For example: given I have 10 rays - I will have to add 10 additional vector sensor observations? Or is there a way to append this additional information to what is read in through the 
    m_RayPerceptionOutput.RayOutputs?[rayIndex].ToFloatArray(numDetectableTags, rayIndex, m_Observations)
    call in RayPerceptionSensor.cs?

    Thank you in advance.
     
    WaxyMcRivers, May 27, 2021
    #1

  2. ruoping_unity

    ruoping_unity

    Unity Technologies

    Joined:
    Jul 10, 2020
    Posts:
    134
    For now the easiest way is to grab the results from RayPerceptionOutput and add the additional observations as vector observations in Agent.CollectObservations().

    Unfortunately RayOutputs.ToFloatArray() wasn't designed to be customizable method. We're actively working on making an interface in Raycast sensor that can be overridden to collect custom observations, so stay tuned!
     
    ruoping_unity, May 27, 2021
    #2

  3. WaxyMcRivers

    WaxyMcRivers

    Joined:
    May 9, 2016
    Posts:
    59
    I very much appreciate your response - thank you.
     
    WaxyMcRivers, May 28, 2021
    #3

  4. ademord

    ademord

    Joined:
    Mar 22, 2021
    Posts:
    49
    when working with the gym wrapper, how do we pass these ray observations when we call GymEnvironment()? do we have to enable 
    allow_multiple_obs=True
    ?
     
    ademord, May 30, 2021
    #4