What is the recommended way to change env parameter and hyper parameter values during training?

At different stages of training different hyper parameters work best.

For example learning rate should be decreased near the end of training.

ml-agents has learning_rate_schedule and curriculum that partially address this but it is hard ti know what values work best and each stage. So I end up running my training and experiment with values at different stages of training.

The problem here is that the initial parts of training get repeated each time I want to change hyper parameters for later stages of training. It wastes a lot of computation power and time.

I tried using --initialize-from to continue training as a new training session but the result is different than running one longer training session with curriculum. I suspect that some stats carry over when using curriculum in the same training session versus starting a new session with --initialize-from.

Can Unity ml-agents team shed some light on what would be the best way to change hyper parameters mid training? Hope my post makes sense.