Strategy board game AI

Hi avihay107, your implementation will vary wildly with how your game is written, but at a high level:

Your states should be the simplest representation of the board/game that is sufficient to play it. For instance, if your game takes place on a grid, you could represent it as a NxNxM matrix, where NxN is the dimensions of the board and M is the number of different types of pieces that may be on it.

A good reward function is sufficiently dense (i.e. a reward is given frequently enough) that the agent can "stumble upon" rewards in random play, but not overly dense as to influence gameplay. You may, for instance, give the agent a small reward for taking an enemy's piece and a big reward for winning; however, it will then be predisposed to try and take as many enemy's pieces as possible and not other strategies.

Note that Model-free Deep RL (what is used in ML-Agents) is a reactive technique (i.e. the algorithm chooses its next action based on what it sees), and does not plan ahead. It may have limited strategy ability compared to algorithms such as MCTS (Monte-Carlo Tree Search), which can plan ahead.

Stay tuned for some improvements for board games in ML-Agents - and good luck with your project.

 

The state isn't the possible actions, the Agent (a neural network) takes the state as input and based on that information, chooses an action. So it needs to have enough information so that it can make the right decision. For the reward function - if you're building a game, a useful way to think about it is a "score". If you're a human player, what should the game reward you (or punish you) for doing, so that the player with the largest score wins the game?

For the last question, you can call RequestDecision manually - see an example of that in our Bouncer environment: https://github.com/Unity-Technologi...ents/Examples/Bouncer/Scripts/BouncerAgent.cs