Debugging Curriculum Learning - Best Practices?

Hello everyone,

I'm copying this together from the example-sourcecode of the unreleased master. So some of the terminology might be outdated.

Making use of curriculums enables us to change the environment by providing external parameters in the curriculum.json file.

To stick to the examples that file could look like this:

Code (JSON):

1. {
2.     "measure" : "progress",
3.     "thresholds" : [0.1, 0.3, 0.5],
4.     "min_lesson_length": 100,
5.     "signal_smoothing" : true,
6.     "parameters" :
7.     {
8.         "small_wall_height" : [1.5, 2.0, 2.5, 4.0]
9.     }
10. }

To access these values in Unity you can use the Academy-Singleton like this:

m_Academy.FloatProperties.GetPropertyWithDefault("small_wall_height", 4)

But how do you guys debug your code with a construct like this? E.g. have the property have different values to test your setup without calling mlagents-learn every time you want to test a different value?

I can think of multiple ways:

#1 change the default value in the argument of GetPropertyWithDefault
this is OK for small projects where every parameter is only used once in code. If you access the parameter in multiple places, you have to change the default value multiple times. That gets messy fast.

#2 call GetPropertyWithDefault at every episode start, and store parameters in variables
Check if it returns the default-value. If it can't get a value, overwrite the variables with debug-parameters.
This seems to be the go-to solution, but you loose the convinience of using the Academy-singleton. And you'll need an extra class.

#3 somehow write values into EnvironmentVariables without using sidechannels
In my opinion this would be the most elegant solution. Now, if that script parses a json-file directly, these variables are supplied hardcoded, or in the GUI is of secondary nature.
Has anybode written anything like this and could share their code?

#4 ?
I'm overlooking something, and everything is much easier than I think


Input to this is welcome

 

Hi,
I think the Solution #1 looks best to me at first glance. If you were not using ML-Agents but wanted to test your game with different values, you would probably have a way to set the height of the wall throughout the entire code. Wither with a static property or a game manager component.
We originally wanted to allow setting the environment parameters from C# but we realized this would cause confusion as C# could override a property set by Python during training.