Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ArgumentException: NaN increment passed to AddReward.

Discussion in 'ML-Agents' started by mateolopezareal, Jun 24, 2020.

  1. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    I was training my agents and suddenly during the training this error popped out, I do not find a reason why can this happen, there has to be a number in that place.
    That is why my training was stable until this happen:
    Code (CSharp):
    1. 2020-06-24 16:59:39 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 3900000. Time Elapsed: 10656.156 s Mean Reward: -6.788. Std of Reward: 9.317. Training.
    2. 2020-06-24 17:08:36 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4200000. Time Elapsed: 11193.373 s Mean Reward: -6.011. Std of Reward: 9.916. Training.
    3. 2020-06-24 17:19:44 INFO [stats.py:100] PruebaBAT7_CubeLearning: Step: 4500000. Time Elapsed: 11861.222 s Mean Reward: -290.725. Std of Reward: 695.414. Training
     
  2. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    Maybe a division by zero somewhere? That's where I usually get my NaNs from ;)
     
  3. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    The method where it happens is this:
    Code (CSharp):
    1. public float GetEntropyCompleta2()
    2.     {
    3.         int[] histogram = getHistogramaCompleto();
    4.         double[] lista = new double[histogram.Length];
    5.         double entropia = 0;
    6.         int deads= histogram[0];
    7.  
    8.         for (int i = 1; i < histogram.Length; i++)
    9.         {
    10.             lista[i] = histogram[i];
    11.             lista[i] = lista[i] / (144 - deads);
    12.             if (lista[i] != 0)
    13.             {
    14.                 entropia += (lista[i] * Math.Log(lista[i], 3));
    15.             }
    16.         }
    17.         return -(float)entropia;
    18.     }
    The int deads is never bigger than 144, and the values is lista can not be negative. Any ideas?
     
  4. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    But can it be 144? That would give you a division by 0, resulting in a NaN in
    lista[i]
    .
     
  5. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    It could be but is quite hard, it it happens I will let you know
     
  6. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    Also note that Math.Log(0) will produce a NegativeInfinity, and 0 * NegativeInfinity results in a NaN.
     
  7. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    There is an if before for that.
     
  8. celion_unity

    celion_unity

    Joined:
    Jun 12, 2019
    Posts:
    289
    Sorry, I missed that part. I'm not sure how
    lista[i] * Math.Log(lista[i], 3)
    will behave if the value is very small (but non-zero) though.
     
  9. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    I think that might be it.