Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Normalized observation values

Discussion in 'ML-Agents' started by Claytonious, Apr 25, 2020.

  1. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    907
    I've read here and there that it's generally good to keep values reported to VectorSensor from implementations of CollectObservations within the -1.0 to 1.0 range. I'm currently flaunting that advice and recording things naively such as:

    Code (CSharp):
    1. sensor.AddObservation(transform.position);
    where the position of the object can be anywhere from 0...5000 in x and z.

    How badly am I hurting myself by using values like that? How would things improve if I were to find a way to normalize this sort of thing, in the most general sense?

    Thanks.
     
  2. vincentpierre

    vincentpierre

    Joined:
    May 5, 2017
    Posts:
    160
    Hi,

    We have a hyperparameter called "normalize" https://github.com/Unity-Technologies/ml-agents/blob/master/config/trainer_config.yaml#L13 that will keep a running mean and standard deviation of the previous observations and normalize the observations automatically. However, this process is slow since the mean and standard deviation of the observations is constantly changing.
    The best thing is to make all observations in the range -1 to 1.
    If the observations are not normalized, the neural network could become unstable. With large weight values causing gradients as large that can put the neural network into an unrecoverable state.
    Not normalizing could prevent learning completely.
     
    ailuropoda0 and Claytonious like this.
  3. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    907
    That sounds pretty severe. Thanks for the clarification!