Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ML-Agents Release 1 (April 30, 2020) - Discussion Thread

Discussion in 'ML-Agents' started by jeffrey_unity538, May 4, 2020.

  1. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    Hi everyone,

    We are very excited to have released ML-Agents Release 1 on April 30, 2020. This is a huge release in so many ways, specifically that we are moving the C# and communicator to v1.0!

    In the following days, we will be publishing a blog post on all the changes on ML-Agents Release 1 and what it means.

    In the meantime, please refer to the release notes for ML-Agents Release 1 here:
    https://github.com/Unity-Technologies/ml-agents/releases/tag/release_1

    As usual, please follow our guidelines on where to open issues, bugs, discussion. All feedback welcome and thank you again for your support of ML-Agents.


    Jeff
     
  2. bobben

    bobben

    Joined:
    Jan 22, 2013
    Posts:
    12
    Come on, work hard
     
  3. SSS135

    SSS135

    Joined:
    Nov 26, 2012
    Posts:
    65
    Hi,
    I am working with ML Agents for some time now. I like this project, it improved much since early versions. But I always end up modifying package's C# source code, throught not as much as I did before.
    For example, now I'm working on multi-goal reinforcement learning (goals are fixed actually, so there is just multiple rewards, one for each goal). I need to send multiple rewards to python side, so I've made a subclass of SideChannel. But there is a problem: I can't accurately synchronize these side rewards with main one without modifying C# package source. To solve this I modified Agent.SendInfoToBrain to call another function which generates and sends auxiliary rewards to python code.
    Before that, there was another problem, when I needed to disable ending of episode when agent's GameObject is deactivated. Also haven't found a way besides modifying C# source.
    Maybe these are just my specific problems and workaround isn't that bad, but I still decided to share.
    Keep up good work.
     
  4. TransformsDante

    TransformsDante

    Joined:
    Nov 4, 2019
    Posts:
    2
    Congratulations! Thank you for your hard work, each release has attacked major pain points, and it's great to see the project reaching a stable point.
     
    celion_unity likes this.
  5. deeznugglets

    deeznugglets

    Joined:
    Mar 19, 2020
    Posts:
    1
    Hey, I am having issues after the update to get Curriculum training to work...

    from terminal:
    mlagents-learn ~/ml-agents/config/ppo/trainer_config.yaml --run-id=some_id-curriculum

    I tried following the migration page on github but it still does not work:

    Added this to trianer_config.yaml, instead of having it in seperate file:
    curriculum:
    measure: reward
    thresholds: [0.1, 0.3, 0.5, 0.5, 0.5]
    min_lesson_length: 100
    signal_smoothing: true
    parameters:
    add_obstacle: [0, 1, 2, 3, 4, 5]

    Previously, my thresholds were at 0,9 for all of them and would reach the first after around 300k steps. Now it does not even seem to reach 0.1..

    I use:
    Code (CSharp):
    1.  stage = (int)Academy.Instance.EnvironmentParameters.GetWithDefault("add_obstacle", 0);
    2.  
    to retrieve the values, and later have a switch statements that initiate different obstacle GameOcjects.

    It worked perfectly fine before the update, but now it does not work at all.
     
  6. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    Can you open up a Github issue with the above?
     
  7. paypaytr

    paypaytr

    Joined:
    Jan 30, 2018
    Posts:
    5
    Since removal of notebooks ( which in my opinion sucks for beginners) I have updated all of my syntax for Rainbow DQN and made a repo about it.

    If any starter looking to just jump to it & use as reference here's it. No PG /AC algorithm yet but I will add as soon as possible.

    https://github.com/ugurkanates/MLAgents-Google-Collab
     
  8. vu1can

    vu1can

    Joined:
    Feb 8, 2020
    Posts:
    2
    Installation instructions don't seem to work
    after flowing the directions from "Installation.md" in the Release1 branch Docs folder
    - created a separate conda env
    - cloned Release1 branch
    - "pip3 install mlagents"

    >the command "mlagents-learn --help" reports "not recognized as an internal or external command"

    DId I miss something?
     
  9. vu1can

    vu1can

    Joined:
    Feb 8, 2020
    Posts:
    2
    humm.. worked second time. But, the config file has been changed. no PPO directory. looks like you have to specify "trainer_config.yaml" instead. Guess the user docs haven't caught up to release
     
  10. ilaydanil

    ilaydanil

    Joined:
    May 17, 2020
    Posts:
    20
    Hi, currently I am working on a Space Shooter game and I created an agent for it. But whenever I try to build it gives this error:
    UnityEditor.BuildPlayerWindow+BuildMethodException: 4 errors
    at UnityEditor.BuildPlayerWindow+DefaultBuildMethods.BuildPlayer (UnityEditor.BuildPlayerOptions options) [0x002bb] in <17b72532ee2c4da1b6f632d3f1705fe0>:0
    at UnityEditor.BuildPlayerWindow.CallBuildMethods (System.Boolean askForBuildLocation, UnityEditor.BuildOptions defaultBuildOptions) [0x00080] in <17b72532ee2c4da1b6f632d3f1705fe0>:0

    here is my code
    [code=CSharp]using System.Collections;
    using System.Collections.Generic;
    using UnityEngine;
    using Unity.MLAgents;

    public class SpaceshipAI : Agent
    {

    public static SpaceshipAI Instance;
    SpaceshipMovement movement;

    public void OnEpisodeBeginFunction()
    {
    OnEpisodeBegin();
    }

    private void Awake()
    {
    Instance = this;
    }

    private void Start()
    {
    movement = GetComponent<SpaceshipMovement>();
    }


    public override void OnActionReceived(float[] act)
    {
    Vector2 temp = new Vector2(act[0], act[1]);
    movement.movement = temp;

    }

    public void AddRewardFunction(float reward)
    {
    AddReward(reward);
    }

    public void EndEpisodeFunction()
    {
    EndEpisode();
    }

    }
    [/code]
     
  11. asatrian

    asatrian

    Joined:
    May 11, 2017
    Posts:
    8
    I did not see the docs explaining the tensorflowsharp.. does it mean it was not supported ?
     
  12. jeffrey_unity538

    jeffrey_unity538

    Unity Technologies

    Joined:
    Feb 15, 2018
    Posts:
    59
    Correct, we no longer support TF# but support Barracuda inference instead.
     
  13. asatrian

    asatrian

    Joined:
    May 11, 2017
    Posts:
    8
    I see thanks!
     
  14. oleg_v

    oleg_v

    Joined:
    Nov 10, 2017
    Posts:
    68
    Thank you for great product!
    Can't catch whats happening with online_bc/offline_bc trainers. There is no any mention in migration logs nor in docs. Are their not supported anymore? Are there any alternatives via gail or behavioral_cloning?