ML-Agents Release 1 (April 30, 2020) - Discussion Thread

Come on, work hard

 

Hi,
I am working with ML Agents for some time now. I like this project, it improved much since early versions. But I always end up modifying package's C# source code, throught not as much as I did before.
For example, now I'm working on multi-goal reinforcement learning (goals are fixed actually, so there is just multiple rewards, one for each goal). I need to send multiple rewards to python side, so I've made a subclass of SideChannel. But there is a problem: I can't accurately synchronize these side rewards with main one without modifying C# package source. To solve this I modified Agent.SendInfoToBrain to call another function which generates and sends auxiliary rewards to python code.
Before that, there was another problem, when I needed to disable ending of episode when agent's GameObject is deactivated. Also haven't found a way besides modifying C# source.
Maybe these are just my specific problems and workaround isn't that bad, but I still decided to share.
Keep up good work.

 

Congratulations! Thank you for your hard work, each release has attacked major pain points, and it's great to see the project reaching a stable point.

 

Hey, I am having issues after the update to get Curriculum training to work...

from terminal:
mlagents-learn ~/ml-agents/config/ppo/trainer_config.yaml --run-id=some_id-curriculum

I tried following the migration page on github but it still does not work:

Added this to trianer_config.yaml, instead of having it in seperate file:
curriculum:
measure: reward
thresholds: [0.1, 0.3, 0.5, 0.5, 0.5]
min_lesson_length: 100
signal_smoothing: true
parameters:
add_obstacle: [0, 1, 2, 3, 4, 5]

Previously, my thresholds were at 0,9 for all of them and would reach the first after around 300k steps. Now it does not even seem to reach 0.1..

I use:

Code (CSharp):

1.  stage = (int)Academy.Instance.EnvironmentParameters.GetWithDefault("add_obstacle", 0);
2.  

to retrieve the values, and later have a switch statements that initiate different obstacle GameOcjects.

It worked perfectly fine before the update, but now it does not work at all.

 

Since removal of notebooks ( which in my opinion sucks for beginners) I have updated all of my syntax for Rainbow DQN and made a repo about it.

If any starter looking to just jump to it & use as reference here's it. No PG /AC algorithm yet but I will add as soon as possible.

https://github.com/ugurkanates/MLAgents-Google-Collab

 

Installation instructions don't seem to work
after flowing the directions from "Installation.md" in the Release1 branch Docs folder
- created a separate conda env
- cloned Release1 branch
- "pip3 install mlagents"

>the command "mlagents-learn --help" reports "not recognized as an internal or external command"

DId I miss something?

 

humm.. worked second time. But, the config file has been changed. no PPO directory. looks like you have to specify "trainer_config.yaml" instead. Guess the user docs haven't caught up to release

 

Hi, currently I am working on a Space Shooter game and I created an agent for it. But whenever I try to build it gives this error:
UnityEditor.BuildPlayerWindow+BuildMethodException: 4 errors
at UnityEditor.BuildPlayerWindow+DefaultBuildMethods.BuildPlayer (UnityEditor.BuildPlayerOptions options) [0x002bb] in <17b72532ee2c4da1b6f632d3f1705fe0>:0
at UnityEditor.BuildPlayerWindow.CallBuildMethods (System.Boolean askForBuildLocation, UnityEditor.BuildOptions defaultBuildOptions) [0x00080] in <17b72532ee2c4da1b6f632d3f1705fe0>:0

here is my code

[code=CSharp]using System.Collections;

using System.Collections.Generic;

using UnityEngine;

using Unity.MLAgents;



public class SpaceshipAI : Agent

{



    public static SpaceshipAI Instance;

    SpaceshipMovement movement;



    public void OnEpisodeBeginFunction()

    {

        OnEpisodeBegin();

    }



    private void Awake()

    {

        Instance = this;

    }



    private void Start()

    {

        movement = GetComponent<SpaceshipMovement>();

    }





    public override void OnActionReceived(float[] act)

    {

        Vector2 temp = new Vector2(act[0], act[1]);

        movement.movement = temp;

       

    }



    public void AddRewardFunction(float reward)

    {

        AddReward(reward);

    }



    public void EndEpisodeFunction()

    {

        EndEpisode();

    }



}

[/code]

 

 

I did not see the docs explaining the tensorflowsharp.. does it mean it was not supported ?

 

I see thanks!

 

Thank you for great product!
Can't catch whats happening with online_bc/offline_bc trainers. There is no any mention in migration logs nor in docs. Are their not supported anymore? Are there any alternatives via gail or behavioral_cloning?