Hummingbird tutorial in ML Agents Version 2 and Unity 2020.3

If you want to follow the Hummingbird tutorial (https://learn.unity.com/course/ml-agents-hummingbirds) with ML Agents Version 2 and Unity 2020.3, here are the necessary steps. This assumes that you are already familiar with the tutorial.

1. Prepare Python
Install and activate a conda environment with Python 3.7, then:

pip install mlagents

pip install gym_unity

2. Version Check (yours may differ)
ML Agents Release 18 Version 2.1.0-exp.1 - June 09, 2021
Python Package 0.27

3. Prepare Unity
Install Unity Hub
In Hub, install Unity 2020.3
Download the Hummingbird project and the Hummingbird source from the tutorial. Open the Hummingbird project, then:

Edit -> Project Settings … -> Package Manager -> Enable Preview packages

Package Manager -> Unity Registry -> ML Agents Version 2.1.0-exp.1 -> Install

4. Update the training configuration yaml script
To update the training configuration script (contained in the Hummingbird source zip file), make a backup of the original and run from the command line:

 python -m mlagents.trainers.upgrade_config <path>/config.yaml <path>/new_config.yaml

Then use the new config file in the training.

5. Update the code
Actions now have type ActionBuffers instead of float[]. This affects the HummingbirdAgent Script which is attached to the Hummingbird prefab in two places: Agent.OnActionReceived and Agent.Heuristic

The beginning of Agent.OnActionReceived becomes

Code (CSharp):

1. public override void OnActionReceived(ActionBuffers actions)
2.     {
3.         // Don't take actions if frozen
4.         if (frozen) return;
5.  
6.         ActionSegment<float> vectorAction = actions.ContinuousActions;
7.  
8. # ...

The beginning of Agent.Heuristic becomes (note that I have renamed the parameter)

Code (CSharp):

1.  public override void Heuristic(in ActionBuffers actionsOutBuffer)
2.     {
3.  
4.         ActionSegment<float> actionsOut = actionsOutBuffer.ContinuousActions;
5.  
6.     # ...

Note that selecting "Burst" as Inference Device in the Behavior Parameters (also on the Hummingbird Prefab) component may speed up your training, compared to "CPU" (15% in my case).

With these changes I was able to run the training. More details on upgrading: https://github.com/Unity-Technologies/ml-agents/blob/release_17/docs/Migrating.md

Hopefully the tutorial which is quite useful will be updated at some point.

 

That's really useful info! Thank you so much!