Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Question Need help understanding ppo training results

Discussion in 'ML-Agents' started by wvm4lol, Jan 12, 2023.

  1. wvm4lol


    Dec 30, 2017
    I'm using ml agents for the first time for a school project, trying to make an AI play tetris.
    I've just been trying out changing different settings in both the game code/rewards and the training parameters but i'm not getting any decent results. Even after a lot of trial and error I'm only getting 2 types of results: 40 reward hist.PNG 54 reward hist.PNG 40 and 54 reward.PNG

    After getting the same type of results as in the top image (blue) i tried changing some things and now i get results like in the bottom image (red) and i have no clue what caused it.

    Github repo containing the project

    Attached Files:

  2. hughperkins


    Dec 3, 2022
    tetris is pretty hard... I'd start really simple. Just have one shape. And make it a simple shape. eg a 2x2 block. Get the agent to learn how to position that. Tetris sounds really ambitious though tbh.... I reckon you might want to simplify it a *lot*. What's the simplest game you can make with some kind of block placement? By the way, you dont say what your inputs are. Inputs are pretty fundamental. So, anyway, try to come up with *really* simple game, with really simple inputs. How about you just have a line of squares, instead of a grid. And your pieces are all lines, like 1x1, 2x1. Start from that.
  3. pradajohn


    Jan 19, 2017
    It is better if you post the hyperparameters for each plot. Also, how did you set the rewards rules?
    Just by seeing the plots, I can say:
    1) Maybe 1M simulation steps are not enough for your agent. Try to go to 10M
    2) it is clear that your agent is not learning. Try to change the reward rules