Question Teaching new input while keeping learned skills

How should I approach teaching my AI for shooter games if I want to use it on different maps? I trained it on a specific map, and everything is fine. Should I just continue feeding him new environments and hope for the best? or I should make separate brains for each map.

I know I can kinda test it myself I just decided to ask if there is an existing rule of thumb for such cases. I would really appreciate your help, thank you!

 

It depends partly on your goal, and partly on how different the environments are.

If your goal each time is to train an agent that can only solve one single map; and if your maps and tasks are very different, then training it on just a single map might work well. Note that this means you will have a zillion differnet models you need to juggle: one foreach map.

If you want to train a single model, that can solve various maps, and including NEW, unseen maps, then you'll want to train your model on different maps. During training, don't just feed it one map over and over again, but feed it different random maps each time.

One big question is: how similar are your tasks?

For example, in Vizdoom scenarios, https://github.com/Farama-Foundation/ViZDoom/tree/master/scenarios , many of the tasks are quite different. One is a maze. Some are shoot-everything-that-moves. One is pick up health packs. So, it might be hard for an agent to know what it has to do on each task, and it might be hard for one agent to learn all tasks.
- in this case, it might be better to have one agent for each *type* of task, and then train each agent on random maps with a similar task

Generally speaking, if you train a model for each task type, rather than for each specific map:
- you'll need fewer total models (one for each task type, rather than one for each map)
- each model might need to be bigger
- each model might take longer to learn

... but you'll have fewer models, and each model can handle new, unseen maps.

On the other hand, if you train a single model on a single map, if there is no randomness at all on that map, the aagent might simply memorize the map, I made a cartoon to illustrate this https://youtube.com/shorts/MB3UWmFGWvk?si=EnSIkaIECMiOmarE

 

Thank you, for such a quick response! It makes sense. Your channel is a goldmine! I am going to think through everything you said and everything you have on your channel.

 

Just one more question. What is more performant in terms of CPU usage (in actual game not while training) one large model or juggling multiple ones?

 

Smaller models will use less CPU.