Search Unity

Performance with large number of agents

Discussion in 'ML-Agents' started by AngrySamsquanch, Jan 26, 2021.

  1. AngrySamsquanch

    AngrySamsquanch

    Joined:
    Dec 28, 2014
    Posts:
    24
    I'm tinkering with a boids simulation using mlagents and the performance is passable on a 5600x with 512 agents, with about 60 fps in a build. I'm curious as to what the limits are to the number of simple agents I can have in a scene so I cranked the spawn count up to 1024 to watch the resulting slideshow. The performance profiler shows that the academy fixed update stepper is taking about 110 ms while running inference on a gpu (1070) in a build. Is there anything that I can do to improve this or have I hit a limit of what can be done on a modern consumer grade processor?

    https://imgur.com/a/B59cbGG


     
    AngrySamsquanch, Jan 26, 2021
    #1

  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    First off, really impressive demo! Do you know if it's dominated by AgentSendState.CollectObservations or ModelRunner.DecideAction?

    For the former, with that many (from what I see there) raycasts having to compute collisions, etc. at the same time, performance is going to take a hit. You'd have to either move to the new DOTS-based Unity Physics stack (huge change) or find a way to reduce the number of raycasts. We're working on a feature that might help with that (i.e. add the position of the closest fish rather than use raycasts) so stay tuned.

    For the latter, you could (on the trainer YAML) try reducing the number of hidden units and layers to improve inference performance.
     
    ervteng_unity, Jan 27, 2021
    #2

  3. AngrySamsquanch

    AngrySamsquanch

    Joined:
    Dec 28, 2014
    Posts:
    24
    The raycasts are only for obstacle collision, but you reminded me that they should only work on the layer with the obstacles. I've also reduced them down to a single sphere cast facing forward. The agents references are stored in a kd tree which is used to calculate the potions of the surrounding fish. Is the grid sensor intended to be used in a similar way?

    It seems that decide action is taking the most time. Is this directly influenced by the number of observations that the agent receives? How stable is dots atm? Would moving to dots help with having thousands of colliders and rigidbodies in a scene even if they're attached to an agent? Thanks.
     
    AngrySamsquanch, Jan 27, 2021
    #3
    OmarVector likes this.

  4. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    DOTS will improve all of these problems, but right now there's no official DOTS-ML-Agents integration.

    Decide action is most influenced by the number of observations collected, and the number of hidden units/layers your neural network is configured by. Since you've already cut down the number of obs, I'd try and reduce the number of layers/hidden units until the agents no longer learn. Also, do they all have the same decision interval?
     
    ervteng_unity, Feb 2, 2021
    #4

  5. AngrySamsquanch

    AngrySamsquanch

    Joined:
    Dec 28, 2014
    Posts:
    24
    I'm spawning the agents on every physics frame to minimize the number of agents that are requesting decisions at the same time. It still works out to about 25 agents updating per frame with 1024 agents in the scene. I think I'm at the minimum number of neurons that'll work (64 x 2), though I haven't had a chance to experiment much with slimmer, but deeper networks (32 x 3 for example) or SAC.

    I've added a demo here if you'd like to try it out.
     
    AngrySamsquanch, Feb 2, 2021
    #5

  6. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    Hmm yeah, I don't think you can make the network much smaller than that. SAC and PPO use the same policies, so the models will be the same performance.

    If you're on GPU inference it's likely you can do a lot more than 25 agents per frame, but that won't reduce the time per step. I think you're reaching the upper limit of what can be done currently, unfortunately.
     
    ervteng_unity, Feb 4, 2021
    #6