Search Unity

Question Multiple Purposed Agent

Discussion in 'ML-Agents' started by ilaydanil, May 19, 2020.

  1. ilaydanil

    ilaydanil

    Joined:
    May 17, 2020
    Posts:
    20
    Hey,
    I want my agent to make multiple actions like movement, shooting and choosing weapons.
    I already implemented methods for each of them.
    Do I need to create multiple agents for each action or can one agent deal with multiple things?

    When there is only one kind of action, I use OnActionReceived() function and I call the necessary method but I don't know what to do when there is multiple actions like I mentioned.
    Do I need to create multiple OnActionReceived() functions? How does it work?
     
  2. MrWetsnow

    MrWetsnow

    Joined:
    Jan 5, 2020
    Posts:
    60
    https://github.com/Unity-Technologi...onment-Design-Agents.md#discrete-action-space

    is what you are looking for. One agent. Assuming the agent doesn't need to do these actions at the same time, you can get away with just a single branch:

    branch = 0
    -------------
    0 = move right
    1 = move left
    2 = move back
    3 = move forward
    4 = shoot
    5 = pick gun
    6 = pick sword

    If you want the agent to be able to move and shoot at the same time, then you would have a branch for movement, and a branch for a second action.

    branch = 0
    -------------
    0 = move right
    1 = move left
    2 = move back
    3 = move forward

    branch = 1
    0 = shoot
    1 = pick gun
    2 = pick sword
     
  3. mbaske

    mbaske

    Joined:
    Dec 31, 2017
    Posts:
    473
    Training dedicated agents for different tasks can make sense if those tasks don't tightly depend on one another and/or happen on different time scales. Let's say you'd like to combine movement, path planning, shooting and choosing a weapon. Movement and shooting would probably have to be handled by a single agent, because movement directly affects aiming and both behaviors have to be performed at roughly the same short time interval. For those, different action branches should be the best option.
    You wouldn't want to update path planning and weapon selection quite as often though. Also, their relation to movement and shooting is rather one-sided: a path planning agent for instance might feed the moving agent a direction vector every 10 steps or so, but it doesn't need to be aware of every single step taken. The upside to having dedicated agents ist that they need to perform less complex tasks each, and therefore require smaller models which should make them easier and faster to train. Also, since the tasks are now modular, agents can be trained separately. You could train the path planning agent while movement itself is driven by a heuristic, or vice versa. Which can provide better control over training conditions.