Search Unity

How to train agent once an Action no longer affects outcome

Discussion in 'ML-Agents' started by CuckooOrange, Jun 10, 2020.

  1. CuckooOrange

    CuckooOrange

    Joined:
    Jun 10, 2020
    Posts:
    2
    Hello,
    My current setup is a 2D environment. There's a robot in the middle, and I am randomly firing projectiles at it from 8 fixed locations around it. I want to train the robot to jump over the projectiles and not get hit. My vectorActions are:
    [0]: 0 = no x movement, 1 = move left, 2 = move right
    [1]: 0 = jump button is not pressed, 1 = jump button is pressed
    The way the jump button works is that the longer I hold the jump button, the higher the jump (up to a certain height).

    So basically, during the jump arc, once the jump button is released, it doesn't matter if the jump button (vectorAction[1]) is 0 or 1 anymore. Would this affect the training? Should I be adding a boolean observation to essentially denote that the jump action no longer matters? I still want the move action to matter (vectorAction[0]). Also, if I need to "pad" an observation, do I basically just set that observation to be 0?
     
  2. CuckooOrange

    CuckooOrange

    Joined:
    Jun 10, 2020
    Posts:
    2
    So I found out I can set action masks by overriding CollectDiscreteActionMasks. I've also added an observation to denote that I've turned on the mask. Not sure if that was necessary though.
     
  3. MrWetsnow

    MrWetsnow

    Joined:
    Jan 5, 2020
    Posts:
    60
    You definitely do not need an observation about the mask.