Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ML agents to heavy for my game

Discussion in 'ML-Agents' started by Dimi951, Apr 8, 2020.

  1. Dimi951

    Dimi951

    Joined:
    Oct 28, 2018
    Posts:
    1
    Hello everyone

    I have been working with ML agents since version 0.5 came out.
    But I still have some problems with the proformas of the Game.
    Let me sketch the situation.

    The game runs at ML-agent version 0.15.0 and Unity 2019.3.8f1
    You can compare the game to “Grid World” instead of a grid of 3*3 its 30*30
    Also there is not 1 player (ML agent) but 8.
    Because the CPU/GPU can’t handle 8 ML agents at the same time, they all take turns.

    But the problem I am having, is that the ML agents make the game drop to 15 fps when in game.
    I have tried the following steps to import the data:

    - Visual render texture set to exact 30*30 grayscale and let every stat of the grid be a shade of gray.

    - A Array of 900 (30*30) datapoints normalized (the creation of the Array only happens when it changes).

    Furthermore there are 19 datapoints more to play the game (such as game mode/specials/player data)

    The agent is trained in reinforcement learning and that part is working out. I think because the agent is getting better over time .

    Every agent also has 2 Desecrate actions ( 1: 21 options - 2:10.000 options)
    I have made the actions so that the first action is selecting the type off the action (example: building something/ replace a worker/do nothing ) and de second action is where to build it on the grid (100*100 positions) .
    Before I had those split up for X and Z, but with this method I can place masks and that really did made my agents learn faster.
    Definitely because the reward of an action comes some seconds after making the decision.
    I also hope that with giving the agent the masks it would understand the link between the map data and the masks.

    My question is; how do I make this agent lighter without making it stupid.
    I think I should split up my data into different shots (9 times 10*10 as a example), but I want to know what the best setting is. And I also want the agent to be aware of the map because it also changes by effect of other players and the game itself.

    Also to be clear I have I I7 3.4Ghz processor and a Nvidia GTX 1060 and 32GB of ram, so normally my PC should be more than good enough for this game that runs the game without ML agents at +/-120fps
    This profiler graph is from a vector observations of 900+19 ( no cam or render texture)
    8 agents ( one at a frame )

    ML-agents_on.png

    No ml agents:

    ML-agents_off.png

    Already thanks for taking a look at my problem. And sorry for my bad English
     
  2. AndrewGri

    AndrewGri

    Joined:
    Jan 30, 2020
    Posts:
    12
    It is just my speculation, but I think I will face with the same issue at some point. I assume that TensorFlow is used even for inference mode. If inferences run on CPU, then I would check if MKLDNN library is used, as it accelerates computing X times in comparison to native TensorFlow.
    However it is just my speculation. Good question to ML_agents developers, how inferences run.