Question Force epsilon to lower values than 0.1

As noticed in https://github.com/Unity-Technologies/ml-agents/issues/4984, the lowest value allowed for epsilon during training is 0.1. It does not decay to 0 as described in https://github.com/Unity-Technologi...hanced-agents-using-recurrent-neural-networks.

Is there a way to force lower values for epsilon?

Many thanks,

 

install ML Agents for local Debug / Modification

https://github.com/Unity-Technologi...advanced-local-installation-for-development-1

Then edit line 55 of ml-agents/ml-agents/mlagents/trainers/ppo/optimizer_torch.py

Code (CSharp):

1.  self.decay_epsilon = ModelUtils.DecayedValue(
2.             self.hyperparameters.epsilon_schedule,
3.             self.hyperparameters.epsilon,
4.            0.1,
5.             self.trainer_settings.max_steps,
6.         )
7.  

And change the 0.1 to whatever you want

Next reinstall your now modified ML Agents from the base directory where its installed

pip3 install -e ./ml-agents-envs
pip3 install -e ./ml-agents

Now you have a modified version with epsilon decay set to whatever you want it to be.

Note: there may be a good reason why Epsilon decay is clamped to 0.1
This I have no idea but you can always modify it. Test it out and then if it does not work as expected revert the changes in your local copy and reinstall it.

 

Thank you very much for the reply! I'll try it out.