Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

DecisionStep vs TerminalStep

Discussion in 'ML-Agents' started by cabesd, Jun 1, 2020.

  1. cabesd

    cabesd

    Joined:
    Feb 4, 2020
    Posts:
    6
    Hi all,
    I'm noticing that during runtime, some Agent ID information is being passed to both the DecisionStep and TerminalStep.
    To be more specific, in Python I can query
    Code (CSharp):
    1. DecisionStep.agent_id
    and
    Code (CSharp):
    1. TerminalStep.agent_id
    , and find that some cases some Agent IDs are on both lists.

    Which is the correct data point for the duplicated Agent ID?
    Is this a possible bug?

    See image below of an example of the 3DBall with 12 agent IDs.

    upload_2020-6-1_12-5-16.png

    I notice that the DecisionStep has a 0 reward for Agents 1, 7 and 9 but a different observation vector is seen on both objects. I would assume that the observation on the TerminalStep would be the actual terminal step.

    Is my assumption correct?

    Thanks!
     

    Attached Files:

  2. vincentpierre

    vincentpierre

    Unity Technologies

    Joined:
    May 5, 2017
    Posts:
    42
    Hi,

    If an AgentId is both in DecisionStep and TerminalStep, it means that the Agent reseted in Unity and immediately requested a decision.
    In your example, Agents 1, 7 and 9 had their episode terminated, started a new episode and requested a new decision. All in the same call to env.step()
     
  3. cabesd

    cabesd

    Joined:
    Feb 4, 2020
    Posts:
    6
    Hi vincent, thank you SO much for the clarification!!!
     
unityunity