Saving multiple intermediate models during training

Is there a built in way to save models at intermediate steps during training, say at every X number of steps? I am trying to make a video that shows the stages of development during the entire training process, and having intermediate models can show off behaviors that didn't quite make the final cut at the end of training (for example, imagine in the Pyramids example project having a model saved that understands to hit the switch but doesn't know to go look for the golden brick).

My current solution is to run multiple shorter training sessions and use --initialize-from=<previous_run>. Thanks for any advice!

 

Ah, the solution is using the

keep_checkpoints

hyperparameter.