Question How can I tell if Agent is not learning vs. learning very slowly

Hello,

I've been having trouble with my latest AI I've been training. Mainly, I'm having difficulty deciding on whether or not I should give up on a training session and therefore many adjust how the inputs are preprocessed, hyperparameters, etc. and then start a new one.

My agent initially seems to learn a little bit, but the cumulative reward seems to flatten out after this, sometimes a little spike may occur after its done this for awhile, but still not anywhere close to my desired result. My problem is, I'm not sure if it will eventually learn if I just give it enough time. I can't just sit and wait forever obviously, at some point I have to call it quits and try something new.

How can I tell whether or not if I should give my agent more time to learn?

 

You cant really tell. Sometimes my ML takes 2 hours learning nothing and than has a fast increase.
Sometimes your hyperparamters are wrong and it will learn very slowly but will learn it eventually.
Sometimes you have a bug and dont notice.

I am taking a Deepl Learning introduction course on my College and it is sick how little changes in Hyperparamters change the outcome with simpler Neural Nets (basically no learning/converging vs 5 min)

So it depends. Try out simple changes one by one (dont change more than one parameter at a time), and let it run for a shorter period of time you can bare. Important is that you know it has no bugs (use Heuristic, Debug, etc.).
you surely use more instances and server build, I suppose?
You can build it and than run it on multiple machines with slightly changed parameters.

There is no workarround as far as I know. There is no formula for that. Grid seach and run

 

I've been training a scene for a month now, I've got 35 iterations and still haven't got the desirable result, taking a break at the moment ha-ha

 

1 iteration a day means that you have been training the next super A.I. brain.
Normally with 100 inputs I get 45 Million iterations in 10 hours

 

When it comes to deep learning, hyper parameters can make or break your training sessions. Especially in Reinforcement learning - there are so many hyper parameters and a single parameter can change how fast your AI learns and/or the quality of learning.

I can't give any code out, but there is a way to modify the configuration files and launch training sessions all through python. Doing a simple grid search can get you parameters that train your AI significantly faster than you imagined.

For example: Define all the parameter fields you want to test against along with the values for the fields, create all combinations of all parameters, for each combo: modify configuration yaml file, launch training session, train and end, do it all over again with next set of parameter combos. After some time with a few amount of epochs per training session, you should be able to find a good set of parameters for the task you're training on.

 

Taking into consideration the little amount of combinations for Unites DL, Grid Search is fine, but Random search is generally the better option and out perform Grid search.