Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question How to adjust configuration file?

Discussion in 'ML-Agents' started by kashikoma, Mar 29, 2023.

  1. kashikoma

    kashikoma

    Joined:
    Nov 30, 2021
    Posts:
    1
    Hi all,
    When I try to use my own configuration file, an error is reported as follows:

    (ml_agents_envs)C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs>mlagents-learn config/ppo_default.yaml --run-id=fffff
    Traceback (most recent call last):
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\Scripts\mlagents-learn.exe\__main__.py", line 7, in <module>
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\learn.py", line 264, in main
    run_cli(parse_command_line())
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\learn.py", line 56, in parse_command_line
    return RunOptions.from_argparse(args)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 931, in from_argparse
    final_runoptions = RunOptions.from_dict(configured_dict)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 961, in from_dict
    return cattr.structure(options_dict, RunOptions)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 222, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 359, in structure_attrs_fromdict
    dispatch(type_)(val, type_) if type_ is not None else val
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 657, in dict_to_trainerdict
    cattr.structure(d, Dict[str, TrainerSettings])
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 222, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 410, in _structure_dict
    return {
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 411, in <dictcomp>
    key_conv(k, key_type): val_conv(v, val_type)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 711, in structure
    d_copy[key] = check_and_structure(key, val, t)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 45, in check_and_structure
    return cattr.structure(value, attr_fields_dict[key].type)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 222, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 215, in structure
    d_final[enum_key] = strict_to_cls(val, t)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 67, in strict_to_cls
    d_copy[key] = check_and_structure(key, val, t)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\mlagents\trainers\settings.py", line 45, in check_and_structure
    return cattr.structure(value, attr_fields_dict[key].type)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 222, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
    File "C:\Users\curef\Software\Anaconda3\envs\ml_agents_envs\lib\site-packages\cattr\converters.py", line 351, in structure_attrs_fromdict
    val = obj[name]
    TypeError: 'int' object is not subscriptable


    Then I found out that was caused by:
    curiosity:
    network_settings: 256


    After I delete the network_settings line, no more errors and warnings. So what's the error means? I still set other parameters under tag curiosity, so have I activate curiosity now? I'm sorry about know little about reinforcement learning.

    Additional question about I found that the game was too laggy during training. Is this normal? Or due to wrong hyper parameters?

    Thanks for any answers!
     
  2. DDe32

    DDe32

    Joined:
    Aug 17, 2022
    Posts:
    1
    I don't know if you already figured this out but i'm posting just in case someone gets stuck in here like i did following an outdated tutorial.

    I was getting a warning " 'encoding_size' should be replaced by 'network_settings' " so i replaced it and this error came out.

    The doc https://github.com/Unity-Technologi...figuration-File.md#curiosity-intrinsic-reward states that the value in 'network_settings' should be:

    Code (CSharp):
    1.  curiosity:
    2.         strength: 0.01
    3.         gamma: 0.99
    4.         learning_rate: 3.0e-4
    5.         network_settings:
    6.           normalize: false
    7.           hidden_units: 256
    8.           num_layers: 2
    9.           vis_encode_type: simple
    That's what the "TypeError: 'int' object is not subscriptable" error means, network_settings property is expecting an array (if you can call it like that) and you are giving it an int.

    As for the lag it may be normal. By default while learning the game is running at 20x the speed so that makes it appear like its "lagging". If you mean like lagging real hard it could be for a lot of reasons maybe you can decrease your buffer-size or num-epoch but it's going to hurt the learning performance.