Question Using ML-Agents to train an AI to play classic tetris game

Hi everyone I am a beginner in artificial intelligence. I tried to train the AI models by setting the observations and rewards as follows:

Observations: line clear(0-4) number of holes bounded by four walls, boardstate score(higher the worse), sum of height of the each column
Rewards: line rewards when cleared lines(pow(2, lines cleared) * 10), -10 for game over, +1 for placing each tetromino

After training for serveral hours, the model can only move left and right to place the tetromino horizontally but not clearing serveral lines.

Can anyone suggest a better parameters for observations and rewards with the vector observation space size and number of stacked vectors? I am using discrete branches size of 2: 4 for rotation and 10 for positions and ML-Agents 19. Thank you.

 

Try using the inputs you are giving the AI to play the game yourself. Can you paly tetris, yourself, using only the inputs you are giving the AI? If you can't, what information are you missing?

 

Do you mean the discrete inputs settings? The AI can move the tetrominos normally and drop the tetrominos like normal gaming. But it does not play like human to clear lines. It just place the tetrominoes flatly to the left and to the right and in one episode it can hardly clear 1 line and never clear 2 lines or more. So I wonder how can I modify the agent's parameters(observations/rewards/vector observation space size/number of stacked vectors) to make it like human playing(clear 2 lines or more when it place one tetromino to get high score and avoid gameover for a long playing time).

 

No, I mean, the inputs that you are feeding to the AI. The observations of the world. If you only have those observations, can you play tetris yourself?

 

I can read input from the computer and call the rotate, move, drop functions to play tetris myself. Some of my script of tetris agent is like this. The rewards are called in the Tetris script.

Code (CSharp):

1. public class TetrisAgent : Agent
2. {
3.  
4.     public override void OnEpisodeBegin()
5.     {
6.         // Reset the environment.
7.         FindObjectOfType<Game>().Restart();
8.     }
9.  
10.  
11.     public override void CollectObservations(VectorSensor sensor)
12.     {
13.         // Generate observations.
14.  
15.         // num of rows cleared
16.         sensor.AddObservation(FindObjectOfType<Game>().numofRows);
17.  
18.         // fourWallHole
19.         sensor.AddObservation(FindObjectOfType<Game>().fourWallHole);
20.  
21.         // boardStateScore
22.         sensor.AddObservation(FindObjectOfType<Game>().boardStateScore);
23.  
24.         // sum of height
25.         sensor.AddObservation(FindObjectOfType<Game>().sumHeight);
26.  
27.  
28.     }
29.  
30.     public override void OnActionReceived(ActionBuffers actionBuffers) {
31.         // Rotation: 0-3, Position: 0-9
32.  
33.         for (int i = 0; i < actionBuffers.DiscreteActions[0]; i++) {
34.             GameObject.FindGameObjectWithTag("CurrentActiveTetromino").GetComponent<Tetris>().Rotate(1);
35.         }
36.  
37.         if (actionBuffers.DiscreteActions[1] < 5) {
38.             for (int i = 0; i < 5 - actionBuffers.DiscreteActions[1]; i++) {
39.                 GameObject.FindGameObjectWithTag("CurrentActiveTetromino").GetComponent<Tetris>().Move(Vector3.left);
40.             }
41.         } else {
42.             for (int i = 0; i < actionBuffers.DiscreteActions[1] - 5; i++) {
43.                 GameObject.FindGameObjectWithTag("CurrentActiveTetromino").GetComponent<Tetris>().Move(Vector3.right);
44.             }
45.         }
46.  
47.         GameObject.FindGameObjectWithTag("CurrentActiveTetromino").GetComponent<Tetris>().HardDrop();
48.  
49.     }

 

This code is for the output from the AI, or from yourself: you're showing code to receive your actions.

What inputs are you reading from the game? what observations are you feeding to yourself, and to the AI? What can the AI see? If you see wht the AI can see, can you play tetris.

 

you could use a 2d array for a grid representation of the board state (each element is 0 = empty, 1 = block), and an one hot encoding of the type of the next block.