Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

How To Query The ValueFunction Of A Different Trained Brain While Training An Agent

Discussion in 'ML-Agents' started by Phong, Feb 20, 2020.

  1. Phong

    Phong

    Joined:
    Apr 12, 2010
    Posts:
    2,024
    Here is my problem. I have two brains:

    • A trained walk-brain that handles a running/walking/standing humanoid character
    • An untrained jump-brain that handles launch/airborne/land states of a jump.

    The jump-brain should learn to land in a pose that will lead to a successful/smooth transition to standing/walking/running. What I would like to do is query the value function of the walk-brain at the moment the character lands. If the value function of the walk-brain is high (good expected future reward), then the jump-brain gets a good reward. If the value function of the walk-brain is low, then the jump-brain gets small-negative reward (the walk-brain will probably fall over after landing).

    I need to be able to query the walk-brain (inference mode). While the jump-brain is being trained (learning mode). Both brains are on the same agent.

    Is there a way to rig a setup like this in an ML-Agents training scene?