Search Unity

How to get expected reward or action probabilities during inference mode?

Discussion in 'ML-Agents' started by MarkTension, Oct 12, 2020.

  1. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    43
    Hi! I'd like to get expected reward or action probabilities during inference mode.
    Is there a place in the code (e.g. a barracuda script) where I can query this information from?
    Would be very helpful and interesting to see.
     
  2. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    Do you mean cumulative reward? Can you share your use case?

    You may be able to hack something together in com.unity.ml-agents/Runtime/Inference/ApplierImpl.cs
     
  3. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    43
    I'd be interested in just visualising in e.g. the UI how the expected reward changes depending on where the agent is in the environment, and also see how certain it is of its actions (by looking at the probability distribution of potential actions that the agent samples from).

    ApplierImpl.cs seems at first glance useful for this. I'll give it a try. Thanks for the idea!
     
  4. Houtamelo

    Houtamelo

    Joined:
    Jan 3, 2020
    Posts:
    31