Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Train configuration setting in Self Play

Discussion in 'ML-Agents' started by Dream_Surpass, Jun 26, 2023.

  1. Dream_Surpass

    Dream_Surpass

    Joined:
    Dec 2, 2022
    Posts:
    18
    `Buffer size` and `team_change/swap_steps` setting in self play algorithm puzzles me a lot.

    What if switching team occurs when the number of experiences is not met the `buffer_size`?
    For example, I set `buffer_size` to 204800 and `team_change` to 100000. When the number of steps reaches 100000, it means the training side switches, but the steps does not meet buffer_size(update requirements).

    Any ideas? Thanks a lot.
     
  2. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
    Yes your interpretation is correct, a larger buffer_size than team_change would mean the agent would change sides and continue adding experiences to the buffer. The model updates would not happen until the buffer is full and would be updated based on the experiences from both sides.