Search Unity

Done() doesn't call AgentReset() right away!

Discussion in 'ML-Agents' started by baramee1407, Feb 11, 2020.

  1. baramee1407

    baramee1407

    Joined:
    Oct 28, 2019
    Posts:
    4
    I'm using ML-Agents v0.10.1. I don't know if it has been change on newer version or not.

    But, I'm counting on the success rate of the car parking model so I modified my code in the inference mode to count the time that the car is parking successfully by increment the counter by 1 before calling Done(), but the increment is 3-4 times before the AgentReset() is called. Which make my success rate 200 - 400% (lol). Now, I use boolean flag to solve this and it woking.

    But, I'm curious that is this how it should behaved?
     
    baramee1407, Feb 11, 2020
    #1

  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    You're correct, the agent doesn't actually reset until the next decision happens after the Done(). But this behavior will be changed to immediate reset in this month's release (and is already the case on master), so stay tuned!
     
    ervteng_unity, Feb 11, 2020
    #2

  3. baramee1407

    baramee1407

    Joined:
    Oct 28, 2019
    Posts:
    4
    Okey, I understand. BTW, why do that in the first place? It make no sense at all to me. And is it effect the training process in some way?
     
    baramee1407, Feb 12, 2020
    #3

  4. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    It was done this way to keep communication of the Done() flag in sync with actions and rewards - this is the way most reinforcement learning APIs are set up, to make training easier. Training shouldn't be affected with the new code.
     
    ervteng_unity, Feb 25, 2020
    #4