Search Unity

Can I add a reward in self-play which would make the game to be non-zero-sum?

Discussion in 'ML-Agents' started by OleguerCanal, Apr 10, 2020.

  1. OleguerCanal

    OleguerCanal

    Joined:
    Jul 21, 2015
    Posts:
    7
    I have a zero-sum game with very sparse reward and I am attempting to train 3 vs 3 agents (two teams).
    Seeing how long it is taking to learn, I would like to speed the training process by giving some rewards for doing certain actions. This would make the reward system to sum more than zero if we add both teams rewards.
    Would this be a problem? If so, why? Do you guys have ideas on how I could approach this problem?

    Thank you a lot!
     
    OleguerCanal, Apr 10, 2020
    #1

  2. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    You can do this but there are two things to keep in mind:

    1. The final reward must still be >0 for winning and <0 for losing for the ELO calculation to be acccurate.
    2. The agents don't ignore the objective of winning and losing to obtain the intermediate reward.

    If the agents seem to ignore winning and losing, you can also try using a curriculum.
     
    andrewcoh_unity, Apr 15, 2020
    #2

  3. OleguerCanal

    OleguerCanal

    Joined:
    Jul 21, 2015
    Posts:
    7
    I see, thanks!
     
    OleguerCanal, Apr 15, 2020
    #3