Search Unity

Question Training with an executable is way slower when using StableBaselines than ML-Agents

Discussion in 'ML-Agents' started by xYuye, Dec 4, 2023.

  1. xYuye

    xYuye

    Joined:
    Mar 24, 2016
    Posts:
    4
    Hi!

    I have built my RL environment and successfully trained it on a headless server by running:


    Code (CSharp):
    1. mlagents-learn config/MyConfig.yaml --env=executable.x86_64 --num-envs=8 --run-id=PPO --force
    and it runs at 600fps approximately, which is great. However, when I run the following code:

    Code (CSharp):
    1.  
    2. from mlagents_envs.environment import UnityEnvironment
    3. from mlagents_envs.envs.unity_gym_env import UnityToGymWrapper
    4. from stable_baselines3.common.vec_env import DummyVecEnv, SubprocVecEnv
    5. from stable_baselines3 import PPO
    6.  
    7.  
    8. def make_unity_env(env_directory, num_env, use_visual=False, start_index=0):
    9.     def make_env(rank):
    10.         def _thunk():
    11.             unity_env = UnityEnvironment(env_directory, base_port=5000 + rank)
    12.             env = UnityToGymWrapper(unity_env, allow_multiple_obs=False)
    13.             return env
    14.         return _thunk
    15.     return DummyVecEnv([make_env(i + start_index) for i in range(num_env)])
    16.  
    17. def main():
    18.     env = make_unity_env('executable.x86_64', num_env=8, use_visual=False, start_index=999)
    19.    
    20.     model = PPO(
    21.         "MlpPolicy",
    22.         env,
    23.         verbose=1
    24.     )
    25.  
    26.     model.learn(total_timesteps=10_000_000)
    27.  
    28. if __name__ == '__main__':
    29.     main()
    30.  
    Then it is way slower and runs at 40 fps approximately. Could anyone explain what is the main reason for that? and if it can be solved?

    Thank you!