Hyperparameters definition

Hello!!!

I would like to ask how we choose the value to set for each parameter in the configuration file.
In my case, I have an agent who have to navigate from one initial point to another.
Initial point (-5,-5)
Goal point (5,5)
I have read the documentation from the ML-Agents and I have made many changes to the parameter values
but the agent does not train correctly.

He can do 8 actions while he is moving.(up,down,right,left and on the sides in 4 directions)

Can someone tell me which changes I have to do, so as the agent can train correctly.
Here is the parameters
MyLearningBrain:
normalize: true
num_epoch: 8
time_horizon: 2048
batch_size: 1024
buffer_size: 10240
max_steps: 3.0e5
summary_freq: 5000
num_layers: 3
hidden_units: 512
reward_signals:
extrinsic:
strength: 1.0
gamma: 0.995

Thank you!!!

 

Hi, i am beginner as well, and i had the same problem.
i have just today achieved to find my first hyperparameter to train a network.

i did the abroach that i started with hyperparameter from the tutorials. those did not work.
the error rate stayed the same during training.
Then i picked parameter by parameter, and changed it's values within certain boundaries.
For example, in "hidden_units" can be drastical changed, starting with a low value, like 8, and doubling it for every test.
i have settet it up with a value for "max_steps" so it is done within a certain timeframe.
Then i made an executable build,
and scripted the executions over night like this:

sleep 180
mlagents-learn ML/config/hidden_01_1.yaml --env=./bin/linux/infectionLinux.x86_64 --run-id=hidden_01_1 --force
sleep 180
mlagents-learn ML/config/hidden_02_4.yaml --env=./bin/linux/infectionLinux.x86_64 --run-id=hidden_02_4 --force
sleep 180
mlagents-learn ML/config/hidden_03_32.yaml --env=./bin/linux/infectionLinux.x86_64 --run-id=hidden_03_32 --force
sleep 180
mlagents-learn ML/config/hidden_04_128.yaml --env=./bin/linux/infectionLinux.x86_64 --run-id=hidden_04_128 --force


Actually i was already thinking about to automate this.
building an algorithm that finds best hyperparameters.
But maybe it's just the case that we need to go over this obstacle one time in order what the parameters really mean, so it is easy in the future to choose those.

 

Think that's called a "grid search". Unfortunately, hyperparameters are interdependent, so you might need to run your search with multiple parameters. Which can take forever, as the search space grows exponentially with every additional parameter.

I've attempted something similar a while back. v0.5 actually, so the code's completely outdated now, but I thought I'd post the link anyway. Maybe it'll give you or anyone else some ideas on how you might want to try hacking the python end for automating the search.
https://github.com/mbaske/ml-agents-hyperparams

 

thank you Mathias for sharing !! Great work and great documentation as well. will take a deeper look into it within the next days. Hopefully i manage to understand and update to the current version. I am still quite a noob to Python & ML-Agents.

Oh, but 1 question pops up: Is the Tool helpful at all ?? If it would be - you might have maintained it ?

 

Thanks! It wasn't too helpful for me personally, but that's just because of the time it takes to run multiple training sessions with each one needing a couple of hours to finish. The original idea was to have a tool that's flexible enough for plugging in different search algorithms, beyond simple grid searches. So far though, I haven't had the need to dig deeply into hyperparameters - tweaking the example config files a bit usually works for me.