Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Should I make my collected Vector Observations more relative?

Discussion in 'ML-Agents' started by JPhilipp, Jan 30, 2020.

  1. JPhilipp


    Oct 7, 2014
    Hi! For my target finding rigidbody helicopter, I'm collecting the following observations (plus some sensor ray components):

    Code (CSharp):
    1. AddVectorObs(target.position - transform.position);
    2. AddVectorObs(transform.rotation);
    3. AddVectorObs(body.velocity);
    4. AddVectorObs(body.angularVelocity);
    I'm wondering, should I somehow make all these observations more relative to the helicopter's forward, e.g. by using InverseTransformDirection on all of them -- if so what would be the best code for that? -- or does it not matter (as they're all equally absolute)? All the input actions, basically WASD-Space steering, are applied relatively.

    Any help appreciated, thanks!
    Last edited: Jan 30, 2020
  2. JPhilipp


    Oct 7, 2014
    For what it's worth I'm now using these (partially inspired by the CrawlerAgent example), though I'm not sure yet it's appropriate. Help appreciated.

    Code (CSharp):
    1. Vector3 directionToTarget = target.position - transform.position;
    2. Quaternion lookRotation = Quaternion.LookRotation(directionToTarget);
    3. Matrix4x4 targetDirectionMatrix = Matrix4x4.TRS(, lookRotation,;
    6. Vector3 relativeTargetPosition =
    7.     transform.InverseTransformDirection(target.position - transform.position);
    8. AddVectorObs(relativeTargetPosition);
    10. Vector3 forwardRelativeToLookRotationToTarget =
    11.     targetDirectionMatrix.inverse.MultiplyVector(transform.forward);
    12. AddVectorObs(forwardRelativeToLookRotationToTarget);
    14. Vector3 upRelativeToLookRotationToTarget =
    15.     targetDirectionMatrix.inverse.MultiplyVector(transform.up);
    16. AddVectorObs(upRelativeToLookRotationToTarget);
    18. Vector3 velocityRelativeToLookRotationToTarget =
    19.     targetDirectionMatrix.inverse.MultiplyVector(body.velocity);
    20. AddVectorObs(velocityRelativeToLookRotationToTarget);
    22. Vector3 angularVelocityRelativeToLookRotationToTarget =
    23.     targetDirectionMatrix.inverse.MultiplyVector(body.angularVelocity);
    24. AddVectorObs(angularVelocityRelativeToLookRotationToTarget);
  3. mbaske


    Dec 31, 2017
    My reason for always localizing velocities is that it reduces observation variability, and should therefore be easier to train with. Unless the agent has to be aware of global directions for some other purpose, which I haven't encountered yet. Let's say your helicopter flies towards an obstacle. Maybe it can evade that by turning left or right. Using global velocities would produce different observation values every time, depending on where the helicopter is coming from. But that's usually not relevant in this kind of situation.
    EDIT: The same is true for positions. Observing the relative position of an obstacle is more straight forward in my opinion than inferring it from two global positions for agent and obstacle.
    Last edited: Jan 31, 2020
    gomizako and JPhilipp like this.