Problem with Multiple (unique) Agent Env (python only loads one behavior)

I am trying to train two different sets of agents in the same environment, but at training time, only one behavior parameter will be loaded in the python CMD window (one of the agent types will just stand inactive)...

Is there some kind of cmd or parameter that must be applied for multi (unique) agent envs?

REPRO STEPS
1: Add a single 3DBallHardNew agent to the 3DBall scene
2: Make Offset-step = true on the Hard agent
3: Set decision period to 6 (from 5) on the hard agent
4: initiate training

 

Hi,
Not sure I understand the issue but if a behavior is not appearing in the terminal, it probably means that the agents with that behavior have not yet requested any decisions. Make sure that the Agents with the missing behavior either have a decision requester component or that you call RequestDecision on them regularly. WallJump is an example of environment where the agents can have either one of two behaviors.
It is also possible that the agents with the missing behavior request decisions but end all their episodes too quickly. If the Agents have their episodes be too short, they will not be registered to the trainer.

 

This is exactly the problem. I am indeed requesting decisions for both agents, and both agent classes work fine if used individually. The sole difference of another type of agent being present causes the problem (the terminal just refuses to load more than one behavior).

Mostly I was wondering if there was some obvious setting or parameter that would cause this (or perhaps some updated setting that I have missed until now).



I'm actually on a strange version at the moment (I helped Celion out with some fix-testing) so I'm now wondering if this is actually just a bug.

 

"both agent classes work fine if used individually. " That sounds like a bug. Are you using different behavior names ?
Maybe re-cloning the repository will help?
Are the two agents in the same gameobject hierarchy ? If they are, maybe separate them in 2 different prefabs?
Which behavior has priority over the other (if both behavior are present, one of them is silenced, which one is it). Is there any difference between the two?
Maybe try the walljump environment? If it works then there might be something wrong with your environment.

 

Yep, am using different behavior names. I will try re-installing at some point today I think.

The agents have their own prefab. The prefabs are situated in the same hierarchy level, but one of the agents has its agent script located an extra layer deep within the prefab-specific hierarchy.

I have done some testing to determine what gives priority, and it seems to be that the first agent which requests an action determines the working class. But in debugging this, I discovered that I can get both behaviors to load IF they are both first requested on the same tick. One of my agent classes was using a starting offset, which allowed the other class to always request a decision first.

I checked the Wall Jump scene (it still works), and the only difference is that I have two different agent scripts/classes, whereas the wall jump uses the same class and swaps behaviors and models.

Note: it's always possible that there's something wrong with my environment, but I cannot imagine what given the agents work fine individually. Given the narrowed down cause (two unique agent classes making their first decisions on different steps) I think it might be an edge case that is not handled correctly.

 

This is super strange. Can you try with the balance ball environment ? In the 3DBall scene, spawn a 3DBall hard prefab mid training. Try with different offsets and decision periods if necessary. I tried it on master and it works as expected. If it does not work for you, it could be a bug in the version you are using. If it does work, then it probably is a problem with your environment...

 

I think I found the final ingredient that should let you reproduce this issue in the 3D ball env:

1: Add a single 3DBallHardNew agent to the 3DBall scene
2: Make Offset-step = true on the Hard agent
3: Set decision period to 6 (from 5) on the hard agent
4: initiate training

When they have the same decision interval, there seems to be no issue. But when there are different decision intervals and a starting offset, it often doesn't work. I say "often" (very strange indeed) because it seems like certain combinations or ratios of decision intervals can work. For example, if you set the hard-ball agent to have a decision interval of 2, both behaviors will load regardless of offset settings.

Do let me know if following those steps does not reproduce the issue...

 

Did a bit of playing around, the issue is definitely a strange one.

A sloppy fix to cut through the strangeness is simply adding a RequestDecision() to in Start(). The conditions where this issue occurs seem to vary inexplicably. Hopefully the steps in the above post will reliably cause the problem, else it must be my version.

 

Another ml-agents user in the discord followed the repro steps (mlagents version 14) and confirmed the problem.

EDIT: I went ahead and opened up a ticket for this issue as it has been confirmed reproducible across systems and versions:

https://github.com/Unity-Technologies/ml-agents/issues/3854

 

Thank you. As this looks like a bug, github seems to be the best place for it.