Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Discussion Tips on Agent

Discussion in 'ML-Agents' started by vcinardo, Dec 6, 2022.

  1. vcinardo


    Dec 23, 2019
    Hello, I am having trouble getting my agent to learn and I believe it's due to some fundamental I am missing about deep reinforcement learning. In my current set up, I have an agent in a maze that will run into invisible tiles to stay on course. If they go off course, the agent will be punished, but rewarded slightly for staying on course (going towards a tile or checkpoint is good. Going towards a wall when seeing a tile/checkpoint is bad. Going off course, surrounded by tiles is bad). This is done using a Raycast.

    Here is the setup. White blocks are checkpoints, the one at the end is the goal. Those highlighted hitboxes are the tiles.

    MLAgents config file: default, sometimes I increase the max steps.

    Agent setup: Max steps is 100,000. Observations are what it sees north, south, east, and west. 1 for a tile/cheese, 0 for anything else (wall). Decision period is 20. I have attached the agent file that I have been working on.

    Hit wall = -20.0
    Go towards tile/cheese = 0.02
    Go in a direction other than tile/cheese or end up surrounded by walls = -0.03
    Hit a checkpoint = 10.0
    Hit a tile = 0.6

    If there are some glaring issues with my setup, please point me to some reference material that would help. Any advice on deep reinforcement learning or my setup would be greatly appreciated.

    Attached Files: