Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

ML-Agents: Multiple environments (num-envs)

Discussion in 'ML-Agents' started by dracolytch, Jan 9, 2020.

  1. dracolytch


    Jan 1, 2016
    Hey folks,

    Getting into training with multiple simultaneous environments... Right now I have 8 environments going, and it's giving me a 2x speedup. CPU is at 15%, Memory at 40%, disk/network/gpu are all negligible. Any thoughts as to what's causing the bottleneck here? Also, below is a screenshot of the reward: Orange is 1 instance, blue is 8 instances. The problem is the same. Any clues as to why it's gone all... saw-tooth-y?

    CloudyVR and KarloE like this.
  2. SmartMediaNL


    Sep 29, 2016
    i like to know as well. at my system it seems only two cores (of total 8 cores /16 threads) are working hard. Increasing enviroments does not help (much) going from 8 to 16 only eats up more Ram nothing more. CPU keeps bouncing around 25 to 33% SSD is idle. I tried to increase buffer sizes in config file but nothing seems to help. I noticed 2 instances of Python running wish would explain the 2 core load. no idea on how to increase that to eight.
    CloudyVR likes this.
  3. C0dingschmuser


    Mar 23, 2021
    Thats because Pytorch is only configured to use 4 Threads at once. You can change that in venv\Lib\site-packages\mlagents\torch_utils\ at "get_num_threads_to_use()"

    In the "return max(min(num_cpus // 2, 4), 1) if num_cpus is not None else None"-Line change 4 to the Number of Threads you want
  4. ervteng_unity


    Unity Technologies

    Dec 6, 2018
    A note about PyTorch and CPU threads - for the small networks we're using in ML-Agents, increasing the number of threads that PyTorch uses will increase your CPU usage but it won't actually make it much faster o_O. This is because parallelizing small ops is less beneficial than with large ops (e.g. in the case of CNNs).

    As for the sawtooth problem - you're likely going to have to increase your summary frequency - it looks like many more short episodes are completing in between each summary write b/c of the increase in environments.