Question Curriculum Training not increasing lesson.

Hello,
I am trying to use curriculum training in combination with self-play. The problem is that during the whole training process, the lesson does not change. The parameter is shown in Tensorboard but does not change from the value defined in the first lesson. I did some research and it looks like in older versions it was necessary to call Academy.Reset(). Is something similar still necessary in the newer versions? I am using ML Agents Version 1.0.4 from the package manager and my version information from the console is:

Version information:
ml-agents: 0.19.0,
ml-agents-envs: 0.19.0,
Communicator API: 1.0.0,
TensorFlow: 2.3.0

I don't use maxStep (I set it to 0) in the AgentClass, but I guess the threshold is depended on the maxStep set in the configuration file anyways. To access the parameter I only use a single line of code, I don't have a custom Academy Class.

Code (CSharp):

1.  
2. float obstacleScale = Academy.Instance.EnvironmentParameters.GetWithDefault("obstacle_size", 0);
3.  

Code (CSharp):

1. environment_parameters:
2.   obstacle_size:
3.     curriculum:
4.       - name: Lesson0 # The '-' is important as this is a list
5.         completion_criteria:
6.           measure: progress
7.           behavior: CarAgentBehavior
8.           signal_smoothing: true
9.           min_lesson_length: 0
10.           threshold: 0.1
11.         value: 0.0
12.       - name: Lesson1 # This is the start of the second lesson
13.         completion_criteria:
14.           measure: progress
15.           behavior: CarAgentBehavior
16.           signal_smoothing: true
17.           min_lesson_length: 0
18.           threshold: 0.3
19.         value: 5.0
20.       - name: Lesson2
21.         completion_criteria:
22.           measure: progress
23.           behavior: CarAgentBehavior
24.           signal_smoothing: true
25.           min_lesson_length: 100
26.           threshold: 0.5
27.         value: 10.0
28.       - name: Lesson3
29.         value: 12.5
30.  

EDIT:
So I am just trying to manage my lessons via C# for now, in a short test this worked quite well. The "issue" I have now is that it looks like the Academy.Instance.TotalStepCount are not the same as the steps in the Tensorboard. For example in Tensorboard the steps are already at 1.66 Mio while Academy.Instance.TotalStepCount is still at 1.45 Mio. Would be nice if someone can explain me the difference between them.

 

Hi, for testing I did set it to 500k. As you can see in the screenshot of Tensorboard the training stopped automatically at 500k as expected. Thanks for the link, that explains why there is a difference.

 

I attach my config file. No I haven't tried running the example environment yet. But I could give it a try.
One thing that I have also noticed, is that I can only train in the UnityEditor. When I try to use mlagents-learn with the --env=<env_name> parameter it starts my game/build successfully (.app) but it does not start training. I only get this message after some time: The Unity environment took too long to respond.

I have the correct scene as only scene in the build settings and it doesn't require any user interaction to start training. I think this must be a problem caused either through my project setup (I am using a Networking Solution for the multiplayer of my game) or my operating system macOS Catalina. Maybe it's the same cause for the problem with curriculum training.

 

Hi, I have changed the max_steps parameter for my testing yesterday, I did set it max_steps=500000 in my config file. Today I had another run with max_steps=5M also no increase in lesson. Should I do a test with the example environment with max_steps=50000?