Search Unity

Replicate OpenAI Five in ML-Agents

Discussion in 'ML-Agents' started by gdoai, Feb 2, 2021.

  1. gdoai

    gdoai

    Joined:
    Apr 14, 2020
    Posts:
    11
    Hello,

    We have just released a first mobile game which is using ML-Agents (https://itunes.apple.com/app/id1533555958)

    We are now working on a different type and more complex game (MOBA like).
    We would like to start by implementing some more advanced AI behaviours like the ones used by Open AI Five as described in their paper:


    As per our understanding of ML-Agents, we need to modify the structure of the Python learning pipeline in order to integrate those changes (embedding, post processing of results...).

    Is that correct?
    Do you have any alternative?
    Are you planning to develop such features? e.g. let the developers choose/create their own pipeline?

    Thanks.
     
    gdoai, Feb 2, 2021
    #1
    Cosimoyi likes this.

  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    Congrats on your game!

    At its core, OpenAI-5's inputs are just "observations" and outputs are just "actions". You can feed things like opponent/teammate information as observations, and let ML-Agents handle the embedding.

    With that said, we are building a feature that will let you put in variable # of observations through an attention mechanism (very similar to what OpenAI does in Five and Hide-and-Seek), so stay tuned for that. And a feature that will handle collaborative team behaviors better - so also stay tuned for that!
     
    ervteng_unity, Feb 2, 2021
    #2
    gdoai likes this.

  3. gdoai

    gdoai

    Joined:
    Apr 14, 2020
    Posts:
    11
    Thanks for your answer.
    However, we were planning to use embeddings in order to handle a variant number of nearby units and also describing modifiers (a variant number of abilities / characteristics).
    Example: your hero can be surrounded by several different units, each of them having a variant number of items, bonus or abilities.
    We are having hard times to implement it in the current state of ML-Agents apart from using fixed space length for each unit and using lots of zero padding.
    Could you guide us on the best approach?

    Looking fwd for the upcoming features! Do you have any rough ETA to share?

    Thanks!
     
    gdoai, Feb 2, 2021
    #3

  4. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    Unfortunately that's probably the best approach for now. The variable length feature has already been merged into the master branch on our open-source project (https://github.com/Unity-Technologies/ml-agents/pull/4909). If you'd like to use it now (experimental), you can check out the master branch. Otherwise it will be released by the next monthly release.
     
    ervteng_unity, Feb 16, 2021
    #4

  5. pinsir

    pinsir

    Joined:
    Dec 15, 2020
    Posts:
    2
    Hi, Gdoai. I'm also trying to make moba games. The AI part is tough to achieve.
    Would you mind to share your experience about how to build AI bots with ml-agents?
     
    pinsir, Mar 24, 2021
    #5