Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Python vs Unity Environment Reset

Discussion in 'ML-Agents' started by dewet99, Mar 23, 2023.

  1. dewet99


    Jun 13, 2021
    Hi there,

    I have a custom algorithm as well as a custom environment that uses MLAgents. I want to know whether signalling a reset should be handled on the Python side or the Unity side?

    For example my agent reaches the end of the episode, designated by it stepping the pre determined number of times inside the Unity environment. Should the proces then be:

    1. Agent reaches max number of steps
    2. EndEpisode() is called
    3. Agent is therefore in Terminal Steps on the Python side
    4. Python sees agent in terminal steps, sets done=True
    5. Python calls env.reset()
    6. env.reset() calls Academy.OnEnvironmentReset in the unity environment, which in turn calls the ResetScene() function
    7. Everything is reset, the next episode starts
    Or should it be
    1. Agent reaches max number of steps
    2. FixedUpdate contains a check like so (psuedocode):
      • if agentStep >= maxEpisodeStep:
      • EndEpisode()
      • ResetScene()
    3. Python sees agent in TerminalSteps
    4. Python calls env.reset()
    5. 6 and 7 of the previous happens
    Essentially, my questions comes down to the handling of ResetScene() between python and unity. Should ResetScene() ONLY be called when env.reset() is done from Python's side? And env.reset() is called when an agent is in terminalSteps, which is determined by whether EndEpisode was called on the agent.

    I hope my question was somewhat clear.