Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Done() doesn't call AgentReset() right away!

Discussion in 'ML-Agents' started by baramee1407, Feb 11, 2020.

  1. baramee1407

    baramee1407

    Joined:
    Oct 28, 2019
    Posts:
    4
    I'm using ML-Agents v0.10.1. I don't know if it has been change on newer version or not.

    But, I'm counting on the success rate of the car parking model so I modified my code in the inference mode to count the time that the car is parking successfully by increment the counter by 1 before calling Done(), but the increment is 3-4 times before the AgentReset() is called. Which make my success rate 200 - 400% (lol). Now, I use boolean flag to solve this and it woking.

    But, I'm curious that is this how it should behaved?
     
  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    You're correct, the agent doesn't actually reset until the next decision happens after the Done(). But this behavior will be changed to immediate reset in this month's release (and is already the case on master), so stay tuned!
     
  3. baramee1407

    baramee1407

    Joined:
    Oct 28, 2019
    Posts:
    4
    Okey, I understand. BTW, why do that in the first place? It make no sense at all to me. And is it effect the training process in some way?
     
  4. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    It was done this way to keep communication of the Done() flag in sync with actions and rewards - this is the way most reinforcement learning APIs are set up, to make training easier. Training shouldn't be affected with the new code.