Search Unity

How does Agent.GiveModel() affect the training?

Discussion in 'ML-Agents' started by donamin1, Mar 12, 2020.

  1. donamin1

    donamin1

    Joined:
    Apr 4, 2013
    Posts:
    26
    Hi

    I'd like to randomly assign some pre-trained models to some of my agents during runtime. I managed to do that using Agent.GiveModel(). But now it seems that even the agents with a pre-trained model are somehow affecting the training (I can tell by looking at my reward plot). Is that true? And if yes, how can I exclude those agents from interfering with the training process? In other words, I want the training to use experiences produced by the agents that do not have a pre-trained model.

    Thanks
     
  2. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    Can you clarify what you mean by affecting the training? The agents themselves are being picked up by the trainers and their models are being updated?

    On the agents with pretrained models, you can try switching behavior type from default to 'inference only.' This might be the functionality you're looking for.
     
  3. donamin1

    donamin1

    Joined:
    Apr 4, 2013
    Posts:
    26
    Hi

    I'm not sure if the models are updated by the trainers. But their experiences is taken into account while training the agents without any pre-trained models. I also tried switching to inference only and it didn't help. In the end, I had to change the behaviour name to something else and use a pre-trained model in "inference only" mode to hide the agents from the trainers.
     
  4. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    If you set an agent to 'Inference only', it shouldn't send data to the trainers i.e. it shouldn't be included in the reward. When are you calling Agent.GiveModel()?

    Can you try running by just setting the desired nn file in the model slot in the behavior parameters script, setting to inference only and not using GiveModel?
     
  5. donamin1

    donamin1

    Joined:
    Apr 4, 2013
    Posts:
    26
    It seems to be correct if I set the model without using GiveModel.
     
  6. andrewcoh_unity

    andrewcoh_unity

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    162
    And the only difference between working correctly and erring is that you call GiveModel with the .nn? How/Where are you calling it?
     
  7. donamin1

    donamin1

    Joined:
    Apr 4, 2013
    Posts:
    26
    Yes that's the only difference. I created a script that gets the behaviorparamters component of the agent and calls givemodel and this is done at agentreset so the agent has a new nn when a new episode begins.