Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Are Ray Perception Sensors Observation Normalized?

Discussion in 'ML-Agents' started by mcdenyer, May 24, 2023.

  1. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    48
    I have been manually normalizing all my vector observations and wasn't using the normalization hyperparamter. I tried flipping on normalization in my config file and noticed a big impact on training so I'm wondering if it's normalizing my ray sensors which weren't before but I would've assumed these would be normalized already.
     
  2. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
    I believe when normalize is set to true all network inputs are normalized on the python side regardless of if they are already in the -1,1 space. This will result in worse performance than when you know the bounds of the inputs and can normalize it yourself.
     
  3. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    48
    Yes but I am talking about the RaySensor 2d Component that comes packaged with unity ML Agents.
     
  4. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    48
    I assume that they are in fact normalized but I am just puzzled how I am getting better results from using the python normalization versus doing my own manual normalizing.
     
  5. Luke-Houlihan

    Luke-Houlihan

    Joined:
    Jun 26, 2007
    Posts:
    303
  6. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    48
    I'm curious why they would not normalize these using the ray distance. I think I must be getting a large benefit from that.
     
  7. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    48
  8. GamerLordMat

    GamerLordMat

    Joined:
    Oct 10, 2019
    Posts:
    185
    @mcdenyer

    most of my bugs came from wrong normalization. If you want to use it without auto-norm than debug everything very carefully.

    regarding your question: I am pretty sure they normalize it.

    in my experience normalized by hand vs auto norm gives no real performance difference. The only thing that is better with manually is to achieve better generalization (if the agent doesnt go in Training to the end of the map, it will break while gaming when used there)
     
  9. mcdenyer

    mcdenyer

    Joined:
    Mar 4, 2014
    Posts:
    48
    @GamerLordMat I think you are correct. I think I am getting a boost on the first half of the level from the auto-normalization because the bounds of the vectors are more accurate for that portion of the level and I am then getting an observation space blowup on the second half of the map when it finally reaches it. The problem is I am struggling to even get to the second half of the map where my manual normalization would help out when I am using manual normalization.