Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved How do I train for more steps?

Discussion in 'ML-Agents' started by xofreshxo, Jul 10, 2020.

  1. xofreshxo

    xofreshxo

    Joined:
    Apr 12, 2017
    Posts:
    79
    Hello,
    I've been playing around with the ml-agents toolkit and one of my models needs to be trained for more than 500000 steps. I've used multiple example .yaml files, many of them with max_steps greater than 500000. However, whenever I start training, the console says under the hyperparameters that the max_steps is 500000. Each of my training runs ends after 500000. Thanks
     
  2. Adham9

    Adham9

    Joined:
    Nov 27, 2019
    Posts:
    18
    Are you sure you run the same the YAML config file you configured? Check the directory of your file and your command.
     
  3. BotAcademy

    BotAcademy

    Joined:
    May 15, 2020
    Posts:
    32
    Also make sure that the name you set in the Behavior Parameters -> Behavior Name is the same that you defined in the config. Otherwise it will take the default one (if using a version prior to ML-Agents Release 3)
     

    Attached Files:

  4. justingarrigus

    justingarrigus

    Joined:
    May 16, 2020
    Posts:
    2
    [EDIT: See reply below this one, fixed the problem]

    I have this same problem! Changing the max_steps in the base config.yaml doesn't seem to update the max_steps in the behavior once the program is run. For me, they get reset back to 500,000 (even though the number 500,000 doesn't occur anywhere in the project, in it's current state...). When I try to --resume training, ml-agents will create a new configuration.yaml (or update it, if it already exists), but the max_steps value of the Behaviour won't reflect my changes in the base file. I've tried resetting the project, changing different .yaml files, deleting the child .yaml files and re-running the base with the updated max_steps, but nothing seems to be working.
     
    Last edited: Jul 12, 2020
  5. justingarrigus

    justingarrigus

    Joined:
    May 16, 2020
    Posts:
    2
    Never mind, found out how to fix it! Here's what I did: in the base .yaml file you used to train, each "group" under the "behaviors:" area represent a behavior. The name of the behavior is defined in Unity under Behavior Name in the Behavior Parameters script of your agent. The name of the behavior you're changing (the one you want to extend the max_steps of) needs to match the one that appears in the .yaml file. If it doesn't match, then ml-agents won't say anything, but will resume with it's own default settings (like what BotAcademy said above).

    You can fix this by either:
    1.) Changing the name of your behavior in the .yaml to match the one in Unity.
    2.) Changing the name of your behavior in Unity to match the one in the .yaml (probably shouldn't do this if you want to resume training, since the results will be based off the Unity behavior name).
    3.) Or copying the behavior that already exists in the .yaml, pasting it below, and changing the name of the behavior to match what exists in Unity (this is what I did, since my training already took place with the behavior name).

    Hope this helps :)
     
    samdutter, Wuagliono98 and xofreshxo like this.
  6. xofreshxo

    xofreshxo

    Joined:
    Apr 12, 2017
    Posts:
    79
    @BotAcademy and @justingarrigus, I updated the behavior name in the .yaml file to match the Unity behavior and that fixed it! I was not aware this had to be edited, so its good to know. Thanks!
     
    samdutter and lin_cheng_TDSE like this.
  7. samdutter

    samdutter

    Joined:
    Jul 29, 2020
    Posts:
    10
    Hi there! Can you specify where the base config file is? I have searched all over for the main config file that is referenced.
     
  8. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    All of our yaml configs live here.
     
    samdutter likes this.
  9. samdutter

    samdutter

    Joined:
    Jul 29, 2020
    Posts:
    10
    Thanks for your response! I am coming back around to the project.

    My confusion stems from this error

    upload_2021-3-24_15-13-19.png

    I have created a config file to the best of my understanding.
    upload_2021-3-24_15-15-50.png
    This is the location in the project. I have created a virtual environment, aidojo.

    upload_2021-3-24_15-17-23.png

    This is the test behavior I have created. I just made a variation of the 3DBall yaml file.

    I am assuming I don't have things in the right location but have struggled finding any direction to go.
     
  10. ahsen35813

    ahsen35813

    Joined:
    Sep 15, 2019
    Posts:
    9
    Perfect, thank you very much. I had copied a previous project and started doing different stuff with it and forgot to change the name on the script and this was causing me a lot of trouble.
     
  11. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Did you change the name of the behavior in the yaml file you created?
     
  12. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    When can you post your mlagents-learn command and your yaml config
     
  13. samdutter

    samdutter

    Joined:
    Jul 29, 2020
    Posts:
    10
    The third screenshot provided shows the yaml file open in notepad.
     
  14. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    What about the mlagents-learn command?
     
    twobob likes this.
  15. samdutter

    samdutter

    Joined:
    Jul 29, 2020
    Posts:
    10
    Ah! I see why it was not working.

    upload_2021-4-1_17-37-58.png

    This is the command that I am now using. I was referencing a tutorial that had the backslashes switched.... They must have been working on a Mac or something?

    From now on I will just reference the Unity documentation directly.

    https://github.com/Unity-Technologies/ml-agents/blob/release_15_docs/docs/Readme.md

    For anyone searching for config file issues specifically, reference these pages
    upload_2021-4-1_17-40-47.png

    Thanks Christopher for your help!!
     
    christophergoy likes this.
  16. EarthHobbit

    EarthHobbit

    Joined:
    May 10, 2016
    Posts:
    44
    What I did, is changing the max steps in the yaml file located in the results folder, and giving the path to this yaml file as argument after the --resume command