Switching agent sparring partner mid training (fellow agent to scripted AI)

I'm currently training my adversarial agent. So far they're learning pretty well, but they haven't really show play pattern that I wish they have. They might if I continue training them for another tens of hours. My question is, is it possible to change their training partner to scripted AI mid training? Currently my scripted AI is still far better than them at playing the game and frankly I wish for my agents to play like my scripted AI (but better).

There's an option to re-train them from beginning using my scripted AI but I'm afraid, the opponent is too strong for them to learn anything, so I decided for them to play against themself until they can atleast damage the enemy. Now that they can actually do it, I want to try my original plan but I'm not sure how.

 

Maybe you can retrain with a curriculum? Perhaps dumbing down your scripted AI enough for agents to make initial progress, and then gradually increase the difficulty. Once they can match the scripted AI, you could switch to self-play for training agents beyond that skill level.
Also see https://forum.unity.com/threads/adding-self-play-in-mid-training.1081079/

 

That is a good idea. I thought of something similar but without scripted AI like learning to shoot a designated point, then to move while shooting etc. But it seems your suggestion is easier to implement. I haven't learn how to create a curriculum though. Any links (explained tutorial especially) that could help ?


On another note, i've trained my agent untill about 1mil steps (+- 800steps per episode) yet it's like how i explained above. Is that normal? or maybe i did something wrong

 

Please see https://github.com/gzrjzcx/ML-agents/blob/master/docs/Training-Curriculum-Learning.md