Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ML-Agents Roadmap Update (May 11, 2020) - Discussion Thread

Discussion in 'ML-Agents' started by jeffrey_unity538, May 11, 2020.

  1. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    Hi all,

    Please see release notes here on what was released in ML-Agents Release 1. Please see our guidelines on where to route issues.

    We would love to hear from you all on requests and features for ML-Agents in this thread!

    ML-Agents 3 Month Roadmap as of May 10, 2020 -
    • Continue development and gather feedback for Python trainers v1.0 & additional trainer enhancements
    • Improvements to the configuration files and structures
    • Improved Editor UI and workflows
    • Upcoming PhysX improvements and corresponding updates to our continuous control tasks
    • Explorations with other Unity packages and services such as Game Simulation, Input System, Cloud Build and AI Planner
    • Additional environments for research benchmarks, game templates, and industrial use cases
    • ML-Agents Cloud Preview Program
    • Evaluation of implementing PyTorch
    Previous roadmap updates:

    Jan 21, 2020
    Feb 14, 2020
    Mar 19, 2020
     
    Last edited: May 15, 2020
    StefanoCecere likes this.
  2. Deozaan

    Deozaan

    Joined:
    Oct 27, 2010
    Posts:
    707
    Copypasta?
     
  3. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    whoops thanks
     
  4. StefanoCecere

    StefanoCecere

    Joined:
    Jun 10, 2011
    Posts:
    210
    many are going PyTorch and indeed would be a nice option
    also a friendship with AI Planner would be huge! thank you!

    if i can add my 2 cents: i feel there is still too much "old stuff" around.. see teh docs and tutorials online.
    ML-agents has so much changed in latest years.
    i would consolidate everything onto Release 2, or even more release 3, since i see there are some big changes in master branch about configs and all
     
  5. Zephus

    Zephus

    Joined:
    May 25, 2015
    Posts:
    356
    There is one main problem I'm encountering when trying to teach ML-Agents to new people, most specifically students. It's way too hard to find information on how to actually implement your own algorithms. The documentation on the Python API exists, yes, but in my experience it's not thorough enough to get something up and running. So I have to run through the process with them every single time.

    As an example - imagine the task was to 'use a DQN with ML-Agents'. This task is given to someone who has only used your PPO algorithm so far, since that's the default. How is that person supposed to get the necessary information to implement this algorithm? In my experience it's scattered throughout the documentation, and even then it's not clear.

    What's missing is a tutorial on how to set up a very simple ML algorithm on your own. Similar to the 'build your own environment' page. I had to go through a lot of trial and error and I don't understand why this information is so hard to come by. You want people to use ML agents for research, right? Why not provide an example on how to actually use other algorithms not implemented by Unity?
     
    Adham9, gsprite, kpalko and 1 other person like this.
  6. Hsgngr

    Hsgngr

    Joined:
    Dec 28, 2015
    Posts:
    61
    Hey we would like to have one 2D example environment. All example environments are in 3D. If it is possible one very basic 2D environment can be really helpful for beginners as below:
    upload_2020-6-10_15-49-33.png
     
  7. andrzej_

    andrzej_

    Joined:
    Dec 2, 2016
    Posts:
    81
    I'd say that the OpenAI gym wrapper would be the best option for something like this. https://github.com/Unity-Technologies/ml-agents/tree/master/gym-unity
    I don't think ML-agents is intended to be a platform to develop novel RL algorithms, rather as a way of easily creating new environments and either using the prebuild tools like PPO, or use something that can work with gym. There are a few other projects as well, but gym is probably still most popular.
     
  8. lukem123

    lukem123

    Joined:
    Oct 30, 2019
    Posts:
    5
    Thanks Jeffrey! The additional environments is what I've been waiting for - sounds great! Could you explain more what the "industrial use cases" are or would be? The biggest problem I'm currently having is that all the example environments seem pretty trivial and simple and I've not been able to train any AI in a slightly more complex (but still relatively easy) environment. A big part of the problem is that MLAgents / Unity seems CPU bound so adding a good GPU doesn't seem to speed up things at all. It takes me about 12 hours for 1 million steps, so 1 to 3 million steps has been a practical upper bound for me and that limits the AI to only very simple behaviors.
     
    Adham9 likes this.
  9. gsprite

    gsprite

    Joined:
    Jun 30, 2020
    Posts:
    2
    I think multi agent support for gym wrapper should be added, having to run multiple envs instead of multi instances within an env adds significant overhead
     
    ZiadJ likes this.
  10. Hsgngr

    Hsgngr

    Joined:
    Dec 28, 2015
    Posts:
    61
    I would like to see support for spiking neural networks in Unity