ArgumentException: NaN increment passed to AddReward.

I was training my agents and suddenly during the training this error popped out, I do not find a reason why can this happen, there has to be a number in that place.
That is why my training was stable until this happen:

Code (CSharp):

1. 2020-06-24 16:59:39 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 3900000. Time Elapsed: 10656.156 s Mean Reward: -6.788. Std of Reward: 9.317. Training.
2. 2020-06-24 17:08:36 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4200000. Time Elapsed: 11193.373 s Mean Reward: -6.011. Std of Reward: 9.916. Training.
3. 2020-06-24 17:19:44 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4500000. Time Elapsed: 11861.222 s Mean Reward: -290.725. Std of Reward: 695.414. Training

 

Maybe a division by zero somewhere? That's where I usually get my NaNs from

 

The method where it happens is this:

Code (CSharp):

1. public float GetEntropyCompleta2()
2.     {
3.         int[] histogram = getHistogramaCompleto();
4.         double[] lista = new double[histogram.Length];
5.         double entropia = 0;
6.         int deads= histogram[0];
7.  
8.         for (int i = 1; i < histogram.Length; i++)
9.         {
10.             lista[i] = histogram[i];
11.             lista[i] = lista[i] / (144 - deads);
12.             if (lista[i] != 0)
13.             {
14.                 entropia += (lista[i] * Math.Log(lista[i], 3));
15.             }
16.         }
17.         return -(float)entropia;
18.     }

The int deads is never bigger than 144, and the values is lista can not be negative. Any ideas?

 

But can it be 144? That would give you a division by 0, resulting in a NaN in

lista[i]

.

 

It could be but is quite hard, it it happens I will let you know

 

Also note that Math.Log(0) will produce a NegativeInfinity, and 0 * NegativeInfinity results in a NaN.

 

There is an if before for that.

 

Sorry, I missed that part. I'm not sure how

lista[i] * Math.Log(lista[i], 3)

will behave if the value is very small (but non-zero) though.

 

I think that might be it.