Search Unity

Question Best way to collect observations on player move

Discussion in 'ML-Agents' started by Adrian-S492, Apr 1, 2021.

  1. Adrian-S492

    Adrian-S492

    Joined:
    May 21, 2015
    Posts:
    8
    I've trained model for my turn-based game. Right now I'm wondering what is the best way to connect it to my existing game flow.

    In my case AI is giving tips on demand for the player. It means player plays on his own, but when he clicks "help" I'm calling RequestDecision() and providing player with suggested move.
    Right now player moves are not registred by the Agent, that means CollectObservations is called only when player ask for help and decision is made by the Agent. I use Stacked Vectors so this has especially bad influence on decision making as model is not fed with observations during player moves.

    I'm trying to find out what is the best way to fix it. I want to collect observations even when player is making the moves. I wonder if using Heuristic() is the only way to do it. I want to keep my code loosely coupled so it doesn't seem to be the best choice to move handling user input to Agent's code. Is there any better way to force agent to collect observations on player move?
     
    Adrian-S492, Apr 1, 2021
    #1

  2. ruoping_unity

    ruoping_unity

    Unity Technologies

    Joined:
    Jul 10, 2020
    Posts:
    134
    It sounds like you would want to run the trained agent for all steps just like the agent is playing the game, except that the output action from the agent is not actually used.

    Heuristic() is not the best way here since if you specify behavior type to Heuristic the agent model will not be used.

    I'd suggest you to call RequestDecision() in every step the actual player make a move, and that will result in calling CollectObservations() every step. And you can just ignore the output action or do nothing in OnActionReceived, and have you user input handled elsewhere.
     
    ruoping_unity, Apr 1, 2021
    #2
    Adrian-S492 likes this.

  3. Adrian-S492

    Adrian-S492

    Joined:
    May 21, 2015
    Posts:
    8
    Thanks, that popped into my mind, but I wasn't sure it's proper use ;)
     
    Adrian-S492, Apr 1, 2021
    #3