Question How to make a transfer learning in ML-Agents?

Hello guys,

I'm wondering, if transfer learning in ML-Agents is possible now? Or how can we modify the scripts to make it possible? For example, in the official example of 2v2 soccer game. What if now I want to make a 3v3 soccer game, using the already trained model of 2v2 to make it train faster?

For now, my idea is to remove the first layer of trained model, and add a new layer of different number of inputs . And use the weights of trained model (except first layer) to initialize new weights for training. But I don't know how to do it in ML-Agents. Any instructions? Thanks a lot!

 

We don't currently have any support for modifying the layers or changing the observation or action sizes, but it's something we're doing research on.

However, once you've trained a 2v2 model, you can pass the

--initialize-from

argument to mlagents-learn to start training a new model using the weights of previous one as a starting point.

 

Thanks. Do we have to make sure that the observation and action sizes are same as before, when we use

--initialize-from

? Or it can adapt to it automatically?

 

Yes, the observation and action sizes (and other model parameters like layer sizes) need to be identical.

 

Okay thanks a lot!

 

Would planning for this and preloading input and output nodes ahead of time work as a workaround at all? It would probably affect performance and itteration times but would it work? So for E.g: Lets say im making an agent that needs say 4 observations for flight, 4 more for the first behviour i want and 5 observations for the second behvaviour i want. Can i start with the 4 observations but pass empty (or static -1) values for the other 9 until add those observations and actions when i make the environment more complex? Same with Actions

 

you might want to start a new training run but initialize it using an already-trained model. You may want to do this, for instance, if your environment changed and you want a new model, but the old behavior is still better than random. You can do this by specifying --initialize-from=<run-identifier>, where <run-identifier> is the old run ID.
For Example:

mlagents-learn config/trainer_config.yaml --initialize-from=first3DBallRun --run-id==second3DBallRun