ML-Agents Roadmap Update (May 11, 2020) - Discussion Thread

Copypasta?

 

many are going PyTorch and indeed would be a nice option
also a friendship with AI Planner would be huge! thank you!

if i can add my 2 cents: i feel there is still too much "old stuff" around.. see teh docs and tutorials online.
ML-agents has so much changed in latest years.
i would consolidate everything onto Release 2, or even more release 3, since i see there are some big changes in master branch about configs and all

 

There is one main problem I'm encountering when trying to teach ML-Agents to new people, most specifically students. It's way too hard to find information on how to actually implement your own algorithms. The documentation on the Python API exists, yes, but in my experience it's not thorough enough to get something up and running. So I have to run through the process with them every single time.

As an example - imagine the task was to 'use a DQN with ML-Agents'. This task is given to someone who has only used your PPO algorithm so far, since that's the default. How is that person supposed to get the necessary information to implement this algorithm? In my experience it's scattered throughout the documentation, and even then it's not clear.

What's missing is a tutorial on how to set up a very simple ML algorithm on your own. Similar to the 'build your own environment' page. I had to go through a lot of trial and error and I don't understand why this information is so hard to come by. You want people to use ML agents for research, right? Why not provide an example on how to actually use other algorithms not implemented by Unity?

 

Hey we would like to have one 2D example environment. All example environments are in 3D. If it is possible one very basic 2D environment can be really helpful for beginners as below:

 

I'd say that the OpenAI gym wrapper would be the best option for something like this. https://github.com/Unity-Technologies/ml-agents/tree/master/gym-unity
I don't think ML-agents is intended to be a platform to develop novel RL algorithms, rather as a way of easily creating new environments and either using the prebuild tools like PPO, or use something that can work with gym. There are a few other projects as well, but gym is probably still most popular.

 

Thanks Jeffrey! The additional environments is what I've been waiting for - sounds great! Could you explain more what the "industrial use cases" are or would be? The biggest problem I'm currently having is that all the example environments seem pretty trivial and simple and I've not been able to train any AI in a slightly more complex (but still relatively easy) environment. A big part of the problem is that MLAgents / Unity seems CPU bound so adding a good GPU doesn't seem to speed up things at all. It takes me about 12 hours for 1 million steps, so 1 to 3 million steps has been a practical upper bound for me and that limits the AI to only very simple behaviors.

 

I think multi agent support for gym wrapper should be added, having to run multiple envs instead of multi instances within an env adds significant overhead

 

I would like to see support for spiking neural networks in Unity