Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Maximising CPU utilization during training?

Discussion in 'ML-Agents' started by raithza, Aug 14, 2020.

  1. raithza

    raithza

    Joined:
    Dec 14, 2011
    Posts:
    21
    Hi there,

    I'm currently attempting a very involved training setup - 3 behaviours training simultaneously, 10 agents per behaviour per env, num-envs=4 on a PC that I custom built for ML experimentation (i9-9900K). Training is decently fast, but I can see the my CPU is still being underutilized - Each unity environment is at ~3% or so, and python sits at about 10%, meaning total CPU usage is at 20-40% at maximum according to Window's performance monitor. What's also interesting is that the Unity environments have significant framedrops when I look at them, even with timescale = 1 (presumeably because they are waiting for the python side of things?)

    Does anyone have any tips for me to just get more CPU utilization? I'm using what might be a small buffer size of 20480 (batch size 1024) per behaviour, should that be increased for the number of environments? should I be using more or fewer envs?

    It's interesting that I can run other programs no problem while training is running - I'd really like to try and max out the speed I get as each training run can take a week in this instance *cracks whip*.

    Any hints or suggestions welcome!

    P.S. I've recently update to ML-Agents 1.0 from 0.5 or so - the improvements in usability and interfaces are enormous, thanks so much for all the work that has been put into this. IMO one of the highest quality components in the entire Unity ecosystem.
     
    raithza, Aug 14, 2020
    #1
    unity_-DoCqyPS6-iU3A likes this.

  2. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    hi Raithza, there are a lot of factors that go into what you are describing (how the Unity engine runs, how your project / scene is setup). a few things:

    1. What would you be trying to achieve with better utilization? There might be much better and more faster ways to train than just increasing CPU utilization (i.e. how the observations are setup, rewards, different algorithms etc..). If your training is taking a week, more CPU utilization might not necessarily reduce that by much.

    2. The frame drops (as least when you visually inspect them in training) is likely due to the synchronous nature between the Python Trainer and Unity.

    And thank you for your feedback on usability/interfaces.

    Separately, would you be interested in utilizing cloud training? We are looking for preview customers (if so, drop me an email at jeffrey@unity3d.com).
     
    jeffrey_unity538, Aug 14, 2020
    #2

  3. raithza

    raithza

    Joined:
    Dec 14, 2011
    Posts:
    21
    Hey,

    Well, yes, the ultimate goal is to improve training times. I've been playing with Unity ML for quite a while now, you might recognise these purple guys:
    There are probably lots of ways to improve my training setup, I've certainly experimented a lot and looked at much of the available literature, but I'm still finding it hard to quickly train these guys the basics (eg balance) after which you have to train them the task at hand, such as throwing/catching in the above gif. Through all my tweaking what has consistently worked is just letting them train for a really high amount of steps. Getting the highest amount of steps out of the minimum amount of time is always going to be very valuable, which is why I'm looking at CPU utilization. I'm currently looking at the Marathon-envs Unity implementation to see if that implementation is drastically different to my own and maybe I can learn something there.


    I am hoping to convince my company to make a game with this tech, and cloud training might be very valuable, I'll definitely send you that email!
     
    raithza, Aug 17, 2020
    #3

  4. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    hi raithza - there is def a bit of an art in terms of setting up the training environment and also in terms of the hyperparameters used for training. I think your approach is reasonable. Would love to chat more about your experience setting up and if cloud training would be of use. Feel free to email me.
     
    jeffrey_unity538, Aug 17, 2020
    #4