Search Unity

What to do to restart the agent simulation loop if ResetOnDone is set to false

Discussion in 'ML-Agents' started by AngelClannad, Apr 29, 2020.

  1. AngelClannad

    AngelClannad

    Joined:
    Mar 16, 2019
    Posts:
    6
    I use the scene created in Getting Started with the 3D Balance Ball Environment.
    The target will respawn in a random position as the script ,in order to set the target's position by user definition(UI,etc),and let the boll roll to it in the inference,I set the Reset On Done to false to stop the repeat.Because if not, the target will respawn and the environment will be reset.
    As you can image,the boll stopped after it hit the target,and what I think is that if I change target's postion first and restart the simulation loop again,maybe the agent could roll to the target.
    I override the AgentOnDone() method and called AgentReset() in it ,it didn't work.
    Then I checked On Demand Decision in Unity Inspector and added RequestDecision() after AgentReset(),didn't work.
    So is there any method I didn't implay to restart the simulation loop?
     
  2. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @AngelClannad,
    From the link you posted it looks like you're using 0.13.1, is that correct?

    The AgentReset method is meant to be overridden by the user (you) and is called automatically. You should be calling the Done method when your agent reaches its goal. If ResetOnDone is disabled, you will only get a reset when the python process resets the unity environment.

    I'm not sure exactly what you are trying to accomplish either. Could you try to explain it a bit more?
     
  3. AngelClannad

    AngelClannad

    Joined:
    Mar 16, 2019
    Posts:
    6
    It is 0.13.1.
    I trained the model and got the .nn file,put the .nn file into Inspector.In the Play Mode,It worked well as the boll rolled to the target.
    When Reset On Done is enabled, after the boll hit the target,target and boll will respawn at a random position,and the boll will roll to the target again,right?
    What I want to accomplish is that in the play mode(inference),the boll and the targer start at a centain place B0 and T0(B for ball, T for target),then the boll rolls to the target and the boll stops at the position T0 where the target is. What is important is that the boll and tatget won't reset at that time.And then if the target's position chenges to T1 latter,the agent(boll) will start to roll from T0 to T1. Target from T1 to T2 and the boll track.
    What I think is that maybe skip the reset,maybe the agent thinks "hi,the target is there,go to it",when the target's postion changes,it thinks "where is the target now,oh it's here,go,to it,but how",after asking the brain for help,the agent knows how to roll to the target.Just like restart the simulation loop once or a RequestionDecision,but I couldn't accomplish that by the methods I mentioned above.
    Could you understand a bit more this time,sorry for the unspecific description.
     
  4. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @AngelClannad,
    It sounds like you need to add some game logic that is outside of the ML-Agents code. Your game should determine the rules of how you want the environment to behave, not the code flow of ml-agents. So if your ball is already able to roll towards a target, you should be able to write code to do exactly what you want when the ball collides with the target. Does that make sense?
     
  5. AngelClannad

    AngelClannad

    Joined:
    Mar 16, 2019
    Posts:
    6
    Sorry,maybe I should explain it in another way:
    During the training phase,while ResetOnDone is false,the agent moves randomly and may fall off the platform,and it is set Done as script.While ReseOnDone is false, the agent keeps falling and won't reset.
    So,I want to know whether there is a way to manualiy accomplish the simulation loop with ResetOnDone false and have the same effect as it is true.The doc says that "When you mark an Agent as done, it stops its activity until it is reset",so,how can I reset and continue training while ResetOnDone is false.
    And I really hope to get more explanation about AgentOnDone().Is It just a method for removing the agent from the scene?
     
  6. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @AngelClannad,
    AgentOnDone is called when Done is set on the Agent. If you want to add custom logic yourself and not rely on resetOnDone, you would do it there.