Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Join us on March 30, 2023, between 5 am & 1 pm EST, in the Performance Profiling Dev Blitz Day 2023 - Q&A forum and Discord where you can connect with our teams behind the Memory and CPU Profilers.
    Dismiss Notice
New Forum User Notice Performance Profiling Dev Blitz Day 2023

How to get expected reward or action probabilities during inference mode?

Discussion in 'ML-Agents' started by MarkTension, Oct 12, 2020.

  1. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    42
    Hi! I'd like to get expected reward or action probabilities during inference mode.
    Is there a place in the code (e.g. a barracuda script) where I can query this information from?
    Would be very helpful and interesting to see.
     
    MarkTension, Oct 12, 2020
    #1

  2. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    Do you mean cumulative reward? Can you share your use case?

    You may be able to hack something together in com.unity.ml-agents/Runtime/Inference/ApplierImpl.cs
     
    andrewcoh_unity, Oct 13, 2020
    #2

  3. MarkTension

    MarkTension

    Joined:
    Aug 17, 2019
    Posts:
    42
    I'd be interested in just visualising in e.g. the UI how the expected reward changes depending on where the agent is in the environment, and also see how certain it is of its actions (by looking at the probability distribution of potential actions that the agent samples from).

    ApplierImpl.cs seems at first glance useful for this. I'll give it a try. Thanks for the idea!
     
    MarkTension, Oct 13, 2020
    #3

  4. Houtamelo

    Houtamelo

    Joined:
    Jan 3, 2020
    Posts:
    30
    Houtamelo, Sep 20, 2022
    #4