Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Unity 2022.2 is now available as the latest Tech release.
    Dismiss Notice
  3. We are making some changes to the DOTS forums.
    Dismiss Notice
  4. Join us on Dec 8, 2022, between 7 am & 7 pm EST, in the DOTS Dev Blitz Day 2022 - Q&A forum, Discord, and Unity3D Subreddit to learn more about DOTS directly from the Unity Developers.
    Dismiss Notice
  5. Have a look at our Games Focus blog post series which will show what Unity is doing for all game developers – now, next year, and in the future.
    Dismiss Notice

Need help with turn-based ML Agents player setup

Discussion in 'AI & Navigation Previews' started by Thejon, Dec 19, 2018.

  1. Thejon

    Thejon

    Joined:
    Nov 12, 2013
    Posts:
    11
    I am trying to make a turn-based game using ML Agents. I decided to start with something simple so I am trying to make "Connect Four". At the moment I am merely trying to get a "Player" brain to work properly and need some help.

    Based on the documentation, it looks like I should be able to use "On Demand Decision Making" for event based input. I have created a component that watches for any of the valid Inputs (the same keys as used for "Discrete Player Actions"), and then calls "RequestDecision" on whichever agent should be taking a turn.

    What I expected to happen:
    The documentation mentions that "RequestDecision" starts an "observation-decision-action-reward cycle", so I expected them to act accordingly and in this order:
    * observation -> "CollectObservations" should be invoked, I also wish to use "SetActionMask" here
    * decision -> determined by the keyboard input
    * action -> "AgentAction" should be invoked

    What actually happened:
    * "AgentAction" is invoked BEFORE "CollectObservations".
    * The move decision ALWAYS picks action index '0' first, for both players, regardless of my input.
    * The move I actually selected seems to be queued and used on subsequent actions, but not always.
    * I can't figure out if I should use some variant of Input's "GetKeyDown", "GetKey", or "GetKeyUp" as an event, because none of them seem to be consistently tied to the "Discrete Player Action's" inputs.

    Is anyone else experiencing similar problems? How am I supposed to have two competing player agents in the scene and make sure that only one of them responds to input at a time, and that the move they pick is actually the one that keyboard input specified?

    Environment:
    I am using Unity Version 2018.3.0f2 Personal
    I have updated to the current ML-Agent master commit (0.6.0)
     
  2. Thejon

    Thejon

    Joined:
    Nov 12, 2013
    Posts:
    11