Question TrainerConfigError: The option default was specified in your YAML file, but is invalid.

Hello Everyone, I am stuck . please help
I was following the Humming Birds Project on learn.unity.com

and at the point where i have to start training, when i type the command
mlagents-learn ./trainer_config.yaml --run-id hb_01
in anaconda prompt

I receive the following error

(ml-agents-1.0) C:\Users\Captain\Desktop\mlagents>mlagents-learn ./trainer_config.yaml --run-id hb_01
2020-06-15 19:28:04.676023: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-06-15 19:28:04.680877: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
WARNING:tensorflow:From d:\pf\anaconda3\envs\ml-agents-1.0\lib\site-packages\tensorflow\python\compat\v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Traceback (most recent call last):
File "d:\pf\anaconda3\envs\ml-agents-1.0\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "d:\pf\anaconda3\envs\ml-agents-1.0\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\pf\anaconda3\envs\ml-agents-1.0\Scripts\mlagents-learn.exe\__main__.py", line 7, in <module>
File "d:\pf\anaconda3\envs\ml-agents-1.0\lib\site-packages\mlagents\trainers\learn.py", line 322, in main
run_cli(parse_command_line())
File "d:\pf\anaconda3\envs\ml-agents-1.0\lib\site-packages\mlagents\trainers\learn.py", line 56, in parse_command_line
return RunOptions.from_argparse(args)
File "d:\pf\anaconda3\envs\ml-agents-1.0\lib\site-packages\mlagents\trainers\settings.py", line 351, in from_argparse
key
mlagents.trainers.exception.TrainerConfigError: The option default was specified in your YAML file, but is invalid.

 

Please see https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Migrating.md
- Trainer configuration format has changed, and using a "default" behavior name has been deprecated. (#3936)
You'll probably need to update your yaml file accordingly.

 

Thank You ! It worked !! @mbaske

 

What needs to be changed in the yaml file to fix this problem?

 

Yes, Please, how did you change this yaml file???

 

Please, someone, tell us how to change the yaml file.

 

Captain here ! as @mbaske said you should use that link, for me it took some time so here is the code:

Code (Boo):

1.  python -m mlagents.trainers.upgrade_config E:\yourfolder\trainer_config.yaml E:\yourfolder\trainer_new_config.yaml

You should give the absolute path of your configuration file *flies away

 

Try this:

behaviors:

  Hummingbird:

    trainer_type: ppo

    hyperparameters:

      batch_size: 2048

      buffer_size: 20480

      learning_rate: 0.0003

      beta: 0.005

      epsilon: 0.2

      lambd: 0.95

      num_epoch: 3

      learning_rate_schedule: linear

    network_settings:

      normalize: false

      hidden_units: 256

      num_layers: 2

      vis_encode_type: simple

    reward_signals:

      extrinsic:

        gamma: 0.99

        strength: 1.0

    keep_checkpoints: 5

    checkpoint_interval: 500000

    max_steps: 5000000

    time_horizon: 128

    summary_freq: 10000

    threaded: true

 

following the link
https://github.com/Unity-Technologies/ml-agents/blob/release_3_docs/docs/Migrating.md

For me (i.e migrating from release 1 to latest)

Open In anaconda prompt,
1. Activate your ML environment and then
2. Change the directory to the folder containing 'trainer_config.yaml' (downloaded form course material)
I did > cd desktop\mlagents as my file was in mlagents folder in desktop.

3. Run the command >python -m mlagents.trainers.upgrade_config [old yaml file name] [new file name]

I did > python -m mlagents.trainers.upgrade_config trainer_config.yaml abc.yaml

The new file abc.yaml is now the main file you need to work with, so delete the old 'trainer_config.yaml' and rename abc.yaml to trainer_config.yaml

now follow the tutorial.

PS : I am attaching the converted file with extension .txt rename it to .yaml that is rename trainer_config.txt to trainer_config.yaml and use this file.

Also if in anaconda prompt you have not changed to the directory (folder) where the trainer_config.yaml file is then

at [old yaml file name] and [new file name] you can give the exact locations of the file respectively
like, i would have done

python -m mlagents.trainers.upgrade_config C:\Users\Captain\Desktop\mlagents\trainer_config.yaml C:\Users\Captain\Desktop\mlagents\abc.yaml

Hope It Helps...

 

Thank you! this worked perfectly

 

thank you it worked

 

Thanks. Worked. Now I can continue with the course. I am curious how you figured it out, the Migrating.md page only guides on how to modify the C# script not the .yaml file, that is actually the issue. Anyway Thank you.

 

Code (CSharp):

1. behaviors:
2.   Hunter:
3.     trainer_type: ppo
4.     hyperparameters:
5.       batch_size: 512
6.       buffer_size: 20580
7.       learning_rate: 0.0003
8.       beta: 0.003
9.       epsilon: 0.2
10.       lambd: 0.95
11.       num_epoch: 3
12.       learning_rate_schedule: linear
13.     network_settings:
14.       normalize: true
15.       hidden_units: 256
16.       num_layers: 2
17.       vis_encode_type: simple
18.     reward_signals:
19.       extrinsic:
20.         gamma: 0.99
21.         strength: 1.0
22.     keep_checkpoints: 2
23.     max_steps: 3000000
24.     time_horizon: 1024
25.     summary_freq: 10000
26.     threaded: true
27.    
28. curriculum:
29.   Hunter:
30.     measure: reward
31.     thresholds: [4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4,4]
32.     min_lesson_length: 200
33.     parameters:
34.       killRange: [9, 8.5, 8, 7.5, 7, 6.5, 6, 5.5, 5, 4.5, 4, 3.5, 3, 2.5, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2,  2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2]
35.       praySpeed: [0, 0, 200, 200, 250, 250, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710, 720, 730, 740, 750, 760, 770, 780, 790, 800, 810, 820, 830, 840, 850, 860, 870, 880, 890, 900, 910, 920, 930, 940, 950, 960, 970, 980, 990, 1000]

Hello,
I am getting a similar error but with curriculum option:

mlagents.trainers.exception.TrainerConfigError: The option curriculum was specified in your YAML file, but is invalid.


Thank you!!

B

 

Can you please attach the yaml file or use the "code" option to keep the formatting? Indentation matters for yaml, so it's hard to tell from what you pasted.

Also, what version of ml-agents are you using? The format for curriculum changed in python version 0.18.0 and it looks like you're using the older format. An example of the new format is here. You can also try the migration script described here.

 

Hello,

Thanks, I edited my earlier post to include the code option of the yaml. I am on ml-agents version 1.0.4
I tried to change the original yaml file I have from my previous port in order to migrate to the new version, however I am getting this error:

Code (CSharp):

1. mlagents.trainers.exception.TrainerConfigError: Unsupported parameter randomization configuration 9, 8.5, 8, 7.5, 7, 6.5, 6, 5.5, 5, 4.5, 4, 3.5, 3, 2.5, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2,  2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2.

I am not sure the way I included my killRange and praySpeed values is correct.
This is the current yaml file I have after modifications :

Code (CSharp):

1. behaviors:
2.   Hunter:
3.     trainer_type: ppo
4.     hyperparameters:
5.       batch_size: 512
6.       buffer_size: 20580
7.       learning_rate: 0.0003
8.       beta: 0.003
9.       epsilon: 0.2
10.       lambd: 0.95
11.       num_epoch: 3
12.       learning_rate_schedule: linear
13.     network_settings:
14.       normalize: true
15.       hidden_units: 256
16.       num_layers: 2
17.       vis_encode_type: simple
18.     reward_signals:
19.       extrinsic:
20.         gamma: 0.99
21.         strength: 1.0
22.     keep_checkpoints: 2
23.     max_steps: 3000000
24.     time_horizon: 1024
25.     summary_freq: 10000
26.     threaded: true
27.  
28. environment_parameters:
29.   killRange:
30.     curriculum:
31.       - name: Lesson0
32.         completion_criteria:
33.           measure: reward
34.           behavior: Hunter
35.           min_lesson_length: 200
36.           threshold: 4
37.         value:9, 8.5, 8, 7.5, 7, 6.5, 6, 5.5, 5, 4.5, 4, 3.5, 3, 2.5, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2, 2, 2, 2, 2, 2,2, 2, 2, 2, 2,  2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2, 2, 2,2, 2, 2
38.          
39.   praySpeed:
40.     curriculum:
41.       - name: Lesson1
42.         completion_criteria:
43.           measure: reward
44.           behavior: Hunter
45.           min_lesson_length: 200
46.           threshold: 4
47.         value:0, 0, 200, 200, 250, 250, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710, 720, 730, 740, 750, 760, 770, 780, 790, 800, 810, 820, 830, 840, 850, 860, 870, 880, 890, 900, 910, 920, 930, 940, 950, 960, 970, 980, 990, 1000
48.        
49.  

Thanks so much,

B

 

Can someone help me out? I am stuck unable to train. thanks a lot!

 

This is what an example from when I did the Hummingbirds course looks like. I just found the variables and copied the values from the old trainer_config to this new one. I don't understand the curriculum bit, so I took it out. This most likely is not a novel solution to your pque

behaviors:
Hummingbird:
trainer_type: ppo

hyperparameters:
# Hyperparameters common to PPO and SAC
batch_size: 2048
buffer_size: 20480
learning_rate: 3.0e-4
learning_rate_schedule: linear

# PPO-specific hyperparameters
# Replaces the "PPO-specific hyperparameters" section above
beta: 5.0e-3
epsilon: 0.2
lambd: 0.95
num_epoch: 3

# Configuration of the neural network (common to PPO/SAC)
network_settings:
vis_encoder_type: simple
normalize: false
hidden_units: 128
num_layers: 2
# memory
memory:
sequence_length: 64
memory_size: 128

# Trainer configurations common to all trainers
max_steps: 5.0e7
time_horizon: 128
summary_freq: 10000
keep_checkpoints: 5
checkpoint_interval: 50000
threaded: true
init_path: null

 

Thanks Kruezkuemmelll, I appreciate that, however in my case I need the curriculum part to work and that's the one that's giving me error and I don't quite understand the syntax either.
Can you help me out celion_unity or anyone else?

Thanks a lot!

 

I tried with configuration parameters suggested and it work well. Anyway for me looks not to store any log (./config/summaries) for tensorboard. Someone know how to enable that feature?

 

The relevant files are made inside the ./results/ directory in your the root of your project.
Make sure that you're in your anaconda virtual environment (usually says "(base)" before the prompt. Then navigate to root of your project folder in the command line. A folder called "results" should have appeared after you started training. Then input `tensorboard --logdir results`.

 

This is definitely the most straight forward. When I tried it though it says "No module named mlagents.trainers.upgrade_config". Do you know how to fix that?

 

Thank you so much !!!

 

If you just want to follow along with the hummingbird tutorial, you can just install the version of ml-agents used in the tutorial:

Code (csharp):

1. pip install mlagents==0.16.0

 

Thank you, this worked for me using mlagents version 2.0.1

 

I did this but I am getting a key error for PPO more specifically the error is
Config file format version : version <= 0.16.X
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\envs\mlagents_r20\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\ProgramData\Anaconda3\envs\mlagents_r20\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "c:\users\artha\ml-agents\ml-agents\mlagents\trainers\upgrade_config.py", line 251, in <module>
main()
File "c:\users\artha\ml-agents\ml-agents\mlagents\trainers\upgrade_config.py", line 245, in main
new_config = convert(old_config, curriculum_config_dict, old_sampler_config_dict)
File "c:\users\artha\ml-agents\ml-agents\mlagents\trainers\upgrade_config.py", line 194, in convert
behavior_config_dict = convert_behaviors(config)
File "c:\users\artha\ml-agents\ml-agents\mlagents\trainers\upgrade_config.py", line 36, in convert_behaviors
hyperparam_cls = all_trainer_settings[trainer_type]
KeyError: 'ppo'

Any help is greatly appreciated