Search Unity

Resolved Saved model

Discussion in 'ML-Agents' started by mateolopezareal, Jun 24, 2020.

  1. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    Why did it save the model before finishing the training? Is this normal?
    Code (CSharp):
    1. 2020-06-24 20:03:42 INFO [stats.py:100] Pruebath128v2_CubeLearning: Step: 2700000. Time Elapsed: 5059.919 s Mean Reward: -8.231. Std of Reward: 12.789. Training.
    2. 2020-06-24 20:12:52 INFO [stats.py:100] Pruebath128v2_CubeLearning: Step: 3000000. Time Elapsed: 5609.333 s Mean Reward: -7.589. Std of Reward: 12.291. Training.
    3. 2020-06-24 20:22:13 INFO [stats.py:100] Pruebath128v2_CubeLearning: Step: 3300000. Time Elapsed: 6170.680 s Mean Reward: -5.681. Std of Reward: 8.657. Training.
    4. 2020-06-24 20:31:01 INFO [trainer_controller.py:112] Saved Model
    5. 2020-06-24 20:31:41 INFO [stats.py:100] Pruebath128v2_CubeLearning: Step: 3600000. Time Elapsed: 6738.388 s Mean Reward: -62.983. Std of Reward: 89.105. Training.
    6. 2020-06-24 20:41:08 INFO [stats.py:100] Pruebath128v2_CubeLearning: Step: 3900000. Time Elapsed: 7305.699 s Mean Reward: -41.377. Std of Reward: 40.700. Training.
    7. 2020-06-24 20:51:15 INFO [stats.py:100] Pruebath128v2_CubeLearning: Step: 4200000. Time Elapsed: 7912.293 s Mean Reward: -10.417. Std of Reward: 12.720. Training.
     

    Attached Files:

    mateolopezareal, Jun 24, 2020
    #1

  2. ervteng_unity

    ervteng_unity

    Unity Technologies

    Joined:
    Dec 6, 2018
    Posts:
    150
    mlagents-learn periodically checkpoints the model, so if the program crashes or the process gets otherwise interrupted, you can use 
    --resume
    to resume training.
     
    ervteng_unity, Jun 24, 2020
    #2
    mateolopezareal likes this.

  3. mateolopezareal

    mateolopezareal

    Joined:
    Jun 4, 2020
    Posts:
    54
    Thx!
     
    mateolopezareal, Jun 24, 2020
    #3