Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Agents with dynamic Vector Action?

Discussion in 'ML-Agents' started by Gekigengar, Jul 18, 2020.

  1. Gekigengar

    Gekigengar

    Joined:
    Jan 20, 2013
    Posts:
    724
    How do I setup behavior vector action if the agent for example, have dynamic amount of limbs, where each limbs is controlled by 1 branch (Muscle Relax/Contract) (0-1f)?

    I need the agent to coordinate between all the available limbs to complete a task.

    Thank you
     
  2. BotAcademy

    BotAcademy

    Joined:
    May 15, 2020
    Posts:
    32
    From what I know, the size of the observation space has to be of a fixed size. I think it is an algorithmic limitation for both PPO and SAC.
     
    Gekigengar likes this.
  3. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    You'll need to commit to a fixed number of actions and observations, at least for now. If you want to generalize training for a variable number of limbs, you can define a large enough action space for the maximum number of limbs and give the agent some feedback about which limbs are present. Could be a simple one-hot encoded observation per optional limb. Your code would just ignore action values for non-existing limbs.
     
    Gekigengar likes this.
  4. Gekigengar

    Gekigengar

    Joined:
    Jan 20, 2013
    Posts:
    724
    Yeah, I fear that's the case..

    I'll try this for now, thanks for the suggestion!