Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

ML agents to heavy for my game

Discussion in 'ML-Agents' started by Dimi951, Apr 8, 2020.

  1. Dimi951

    Dimi951

    Joined:
    Oct 28, 2018
    Posts:
    1
    Hello everyone

    I have been working with ML agents since version 0.5 came out.
    But I still have some problems with the proformas of the Game.
    Let me sketch the situation.

    The game runs at ML-agent version 0.15.0 and Unity 2019.3.8f1
    You can compare the game to “Grid World” instead of a grid of 3*3 its 30*30
    Also there is not 1 player (ML agent) but 8.
    Because the CPU/GPU can’t handle 8 ML agents at the same time, they all take turns.

    But the problem I am having, is that the ML agents make the game drop to 15 fps when in game.
    I have tried the following steps to import the data:

    - Visual render texture set to exact 30*30 grayscale and let every stat of the grid be a shade of gray.

    - A Array of 900 (30*30) datapoints normalized (the creation of the Array only happens when it changes).

    Furthermore there are 19 datapoints more to play the game (such as game mode/specials/player data)

    The agent is trained in reinforcement learning and that part is working out. I think because the agent is getting better over time .

    Every agent also has 2 Desecrate actions ( 1: 21 options - 2:10.000 options)
    I have made the actions so that the first action is selecting the type off the action (example: building something/ replace a worker/do nothing ) and de second action is where to build it on the grid (100*100 positions) .
    Before I had those split up for X and Z, but with this method I can place masks and that really did made my agents learn faster.
    Definitely because the reward of an action comes some seconds after making the decision.
    I also hope that with giving the agent the masks it would understand the link between the map data and the masks.

    My question is; how do I make this agent lighter without making it stupid.
    I think I should split up my data into different shots (9 times 10*10 as a example), but I want to know what the best setting is. And I also want the agent to be aware of the map because it also changes by effect of other players and the game itself.

    Also to be clear I have I I7 3.4Ghz processor and a Nvidia GTX 1060 and 32GB of ram, so normally my PC should be more than good enough for this game that runs the game without ML agents at +/-120fps
    This profiler graph is from a vector observations of 900+19 ( no cam or render texture)
    8 agents ( one at a frame )

    ML-agents_on.png

    No ml agents:

    ML-agents_off.png

    Already thanks for taking a look at my problem. And sorry for my bad English
     
  2. AndrewGri

    AndrewGri

    Joined:
    Jan 30, 2020
    Posts:
    12
    It is just my speculation, but I think I will face with the same issue at some point. I assume that TensorFlow is used even for inference mode. If inferences run on CPU, then I would check if MKLDNN library is used, as it accelerates computing X times in comparison to native TensorFlow.
    However it is just my speculation. Good question to ML_agents developers, how inferences run.