Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question DodgeBall environment resets all decision steps on any win.

Discussion in 'ML-Agents' started by KristianTv, Jan 30, 2023.

  1. KristianTv

    KristianTv

    Joined:
    Oct 11, 2022
    Posts:
    12
    Hello,

    We are writing a master thesis where we are adapting the dodgeball-ml-agents environment to apply neuroevolution with python-NEAT. We are only using the elimination game mode and adapted it to 1v1 instead of 4v4.

    Environment repository: https://github.com/Hallahallan/Dodgeball-Bio-fMRI

    My issue is that the environment is configured i manner that lets all levels in the environment run and reset individually for a more RL based approach where gathering rewards from states and action is important. Each level resets after a agent has been eliminated or 5000 steps has been done. What happens is that for any victory, all decision steps is set to 0 which forces my algorithm to begin a new generation even though all levels are playing as nothing happened. This leads to some generations only lasting 1-3 seconds.

    Since we are using neuroevolution to develop the agents, we need the environments to run in sync where ideally they keep playing until one agent on each level is eliminated or 5000 timesteps has been reached, then reset in unison.

    I have tried commenting out game logic that ends games just to test how a single run would play out, but for the first win the decision steps is set to 0 and a new generation is started.

    The game controller is in this file:
    https://github.com/Hallahallan/Dodg...eball/Scripts/DodgeBallGameController.cs#L444

    Does anyone have any input on this or any tips on tailoring the environment better for neuroevolution?
     
    Last edited: Feb 1, 2023
    Hallahallan likes this.
  2. KristianTv

    KristianTv

    Joined:
    Oct 11, 2022
    Posts:
    12
    Bump: I made a discovery on what made the generations end so quickly. Rewrote the question as more specific.
     
  3. hughperkins

    hughperkins

    Joined:
    Dec 3, 2022
    Posts:
    191
    challenges with managing episode finishing in mlagents is one of the main things that pushed me to write my own interface between unity and python. You can check it out here
     
    KristianTv likes this.