Search Unity

Resolved Self-Play - How do the agents know who the teammates are?

Discussion in 'ML-Agents' started by MidnightGameDeveloper, Sep 30, 2020.

  1. MidnightGameDeveloper

    MidnightGameDeveloper

    Joined:
    Apr 26, 2014
    Posts:
    123
    Hello,

    I am trying to train intelligent adversaries using self-play. What I don't understand is how the agents know which tag (detected by raycast) belongs to the enemies and which are teammates since the teams that are learning getting switched during training. Do the agents know their Team ID or how do they learn who are teammates and who not?

    I am also wondering if there is a way to see which team is currently training?

    Are there any recommendations on which value should be set for stacked vectors for a quite dynamic game/environment where the agents drive cars? (I am using PPO)
     
    MidnightGameDeveloper, Sep 30, 2020
    #1

  2. henrypeteet

    henrypeteet

    Unity Technologies

    Joined:
    Aug 19, 2020
    Posts:
    37
    Thanks for the questions.

    1. Going off of the soccer example one way to represent symmetric teams is to tag each team's agents and include that tag under "Detectable Tags". But make sure to swap the input ordering based on team. In the soccer example it winds up looking like this for each agent, notice the flipped entries.
      • Blue: SoccerTags.png
      • Purple SoccerTagsPurple.png
    2. The team ID is not fed into the network by default
    3. I don't think there is a direct way to check which team is training but you can derive it from the number of steps taken and the value you set for `team_change` in your .yaml config
      • current_team = floor(steps / team_change) % num_teams
    4. I don't think we have a recommended value specifically for driving but others on the forum may be of more use.
     
    henrypeteet, Sep 30, 2020
    #2

  3. MidnightGameDeveloper

    MidnightGameDeveloper

    Joined:
    Apr 26, 2014
    Posts:
    123
    Hello,
    thank you for taking the time to answer my questions. I knew I must have missed something. Swapping the order of the "Detectable tags"-List based on the team makes total sense, I hope that this will improve the training in my project :).
     
    MidnightGameDeveloper, Oct 1, 2020
    #3