Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Imitation Learning isn't working as expected

Discussion in 'ML-Agents' started by luke90275, Apr 3, 2020.

  1. luke90275

    luke90275

    Joined:
    Mar 31, 2020
    Posts:
    3
    First off, I am assuming that step 3 in offline training in https://github.com/Unity-Technologi...mitation-Learning.md#recording-demonstrations is outdated. I therefore skipped this step. If this step is not outdates and I need to do something, please let me know.

    I then followed the steps and wrote this is in the config file.

    Code (csharp):
    1. Player 3 AI:
    2.     max_steps: 5.0e4
    3.     batch_size: 128
    4.     buffer_size: 2048
    5.     beta: 1.0e-2
    6.     hidden_units: 256
    7.     summary_freq: 2000
    8.     time_horizon: 64
    9.     num_layers: 2
    10.     behavioral_cloning:
    11.         demo_path: Assets/Demonstrations/demonstration 2.demo
    12.         strength: 1
    13.         steps: 200000
    Also the image of my agent script is attached at the bottom. However, when I train the

    However, when training the agent, it doesn't seem to follow the behavior of the demonstration at all. I could understand needing more data for the demonstration, but it looks as if its just acting randomly, not even trying to match the demonstration. Also, I don't seem to understand why you would need to train an agent who is trying to duplicate a behavior (behavioral cloning)- wouldn't the agent have all of the data needed? Finally when I get the brain from the player, the actions are just as sporadic as when I'm training the agent. Is step 3 important? Am I configuring the config incorrectly? Do I just need more training data or even less vector observations (currently have 22)? Any other ideas? Should I try online training instead?

    Thanks in advance!!!
     

    Attached Files:

    Last edited: Apr 3, 2020
  2. awjuliani

    awjuliani

    Unity Technologies

    Joined:
    Mar 1, 2017
    Posts:
    69
    Hello,

    The quality of the demonstration file is bound by how informative the observations being used are. If it is not possible to map from observations -> actions given the information provided, then it is unlikely that the agent can learn well from demonstrations. Can you provide a little more information about the problem and the way you have broken it into obs and actions?
     
  3. radiantboy

    radiantboy

    Joined:
    Nov 21, 2012
    Posts:
    1,631
    I cant get imitation to work either, it doesnt follow the demo in the slightest. ML in general has been severely disappointing, cant even train a bot to play a simple 2d game.
     
  4. ChillX

    ChillX

    Joined:
    Jun 16, 2016
    Posts:
    145
    Dude :) It takes patience and lots of time. :)

    Notice you are trying to get behavior cloning to work with just one single demo recording ???

    I get decent results for simple games when using 50 to 100 demo recordings.

    Also try GAIL.