Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Should I make my collected Vector Observations more relative?

Discussion in 'ML-Agents' started by JPhilipp, Jan 30, 2020.

  1. JPhilipp

    JPhilipp

    Joined:
    Oct 7, 2014
    Posts:
    56
    Hi! For my target finding rigidbody helicopter, I'm collecting the following observations (plus some sensor ray components):

    Code (CSharp):
    1. AddVectorObs(target.position - transform.position);
    2. AddVectorObs(transform.rotation);
    3. AddVectorObs(body.velocity);
    4. AddVectorObs(body.angularVelocity);
    I'm wondering, should I somehow make all these observations more relative to the helicopter's forward, e.g. by using InverseTransformDirection on all of them -- if so what would be the best code for that? -- or does it not matter (as they're all equally absolute)? All the input actions, basically WASD-Space steering, are applied relatively.

    Any help appreciated, thanks!
     
    Last edited: Jan 30, 2020
  2. JPhilipp

    JPhilipp

    Joined:
    Oct 7, 2014
    Posts:
    56
    For what it's worth I'm now using these (partially inspired by the CrawlerAgent example), though I'm not sure yet it's appropriate. Help appreciated.

    Code (CSharp):
    1. Vector3 directionToTarget = target.position - transform.position;
    2. Quaternion lookRotation = Quaternion.LookRotation(directionToTarget);
    3. Matrix4x4 targetDirectionMatrix = Matrix4x4.TRS(Vector3.zero, lookRotation, Vector3.one);
    4.  
    5.  
    6. Vector3 relativeTargetPosition =
    7.     transform.InverseTransformDirection(target.position - transform.position);
    8. AddVectorObs(relativeTargetPosition);
    9.  
    10. Vector3 forwardRelativeToLookRotationToTarget =
    11.     targetDirectionMatrix.inverse.MultiplyVector(transform.forward);
    12. AddVectorObs(forwardRelativeToLookRotationToTarget);
    13.  
    14. Vector3 upRelativeToLookRotationToTarget =
    15.     targetDirectionMatrix.inverse.MultiplyVector(transform.up);
    16. AddVectorObs(upRelativeToLookRotationToTarget);
    17.  
    18. Vector3 velocityRelativeToLookRotationToTarget =
    19.     targetDirectionMatrix.inverse.MultiplyVector(body.velocity);
    20. AddVectorObs(velocityRelativeToLookRotationToTarget);
    21.  
    22. Vector3 angularVelocityRelativeToLookRotationToTarget =
    23.     targetDirectionMatrix.inverse.MultiplyVector(body.angularVelocity);
    24. AddVectorObs(angularVelocityRelativeToLookRotationToTarget);
     
  3. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    My reason for always localizing velocities is that it reduces observation variability, and should therefore be easier to train with. Unless the agent has to be aware of global directions for some other purpose, which I haven't encountered yet. Let's say your helicopter flies towards an obstacle. Maybe it can evade that by turning left or right. Using global velocities would produce different observation values every time, depending on where the helicopter is coming from. But that's usually not relevant in this kind of situation.
    EDIT: The same is true for positions. Observing the relative position of an obstacle is more straight forward in my opinion than inferring it from two global positions for agent and obstacle.
     
    Last edited: Jan 31, 2020
    gomizako and JPhilipp like this.