Question Need help understanding ppo training results

I'm using ml agents for the first time for a school project, trying to make an AI play tetris.
I've just been trying out changing different settings in both the game code/rewards and the training parameters but i'm not getting any decent results. Even after a lot of trial and error I'm only getting 2 types of results:

After getting the same type of results as in the top image (blue) i tried changing some things and now i get results like in the bottom image (red) and i have no clue what caused it.

Github repo containing the project

 

tetris is pretty hard... I'd start really simple. Just have one shape. And make it a simple shape. eg a 2x2 block. Get the agent to learn how to position that. Tetris sounds really ambitious though tbh.... I reckon you might want to simplify it a *lot*. What's the simplest game you can make with some kind of block placement? By the way, you dont say what your inputs are. Inputs are pretty fundamental. So, anyway, try to come up with *really* simple game, with really simple inputs. How about you just have a line of squares, instead of a grid. And your pieces are all lines, like 1x1, 2x1. Start from that.

 

It is better if you post the hyperparameters for each plot. Also, how did you set the rewards rules?
Just by seeing the plots, I can say:
1) Maybe 1M simulation steps are not enough for your agent. Try to go to 10M
2) it is clear that your agent is not learning. Try to change the reward rules