Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Effect of num-envs

Discussion in 'ML-Agents' started by Visuallization, Jul 7, 2023.

  1. Visuallization

    Visuallization

    Joined:
    Mar 14, 2015
    Posts:
    8
    Hello there,

    I am wondering about the effect of training option num-envs. Initially I thought it would only affect the speed of learning but it also seems to affect the accumulated reward. If I train with num-envs=8 I get much higher reward (10.000) than when I train with num-envs=1 (100). Can someone tell me how num-envs actually affects the reward and how I would get a similar good reward with only num-envs=1?
     
  2. Energymover

    Energymover

    Joined:
    Mar 28, 2023
    Posts:
    33
    Couple ways as I have seen, all the envs seem to share the batch & buffer, I found it helpful to increase it by multiplying the batch/buffer by the number of environments I run, this also seems to apply to the number of agents in a scene. I guess that makes sense since each agent is contributing to the single batch, that gets collected to the buffer, that feeds the update to the model weights. Also it is different because you are filling the buffers with different observations than you would if you only have one agent. The speed increase is because you have more observation sources contributing to the model updates.
     
  3. GamerLordMat

    GamerLordMat

    Joined:
    Oct 10, 2019
    Posts:
    185
    like Energymover wrote, as far as I know it is jsut about filling the buffer concurrently. The more agents you have the more diverse and faster the buffer will be filled, which leads to faster updates both in quality and speed.

    One problem can occur that if the Buffer is too small that the sequences seem too random to the ML, then you need a bigger buffer