How to use Imitation Learning in version 0.13

Hello,
I would like to experiment with Imitation Learning (GAIL). I recently upgraded ML Agents from 0.10 to 0.13. But now that the old scriptable object Brains have been removed, there is no Player Brain anymore. Earlier we could create a Player Brain with keyboard keys mapped to different agent actions. I see that the Demonstration Recorder is still there. But how do we set up an agent to be human controlled so we can record it?

 

It seems that the Heuristic agents are recorded with the demonstration recorder. So one way of getting player actions recorded would be to have the human UI controls output actions into the Agent.Heuristic() method and let the Agent.AgentAction() handle the movement, etc like it does with a normal agent. I'm trying this out and it seems to be working (I see the demo recordings grow in size, etc). Don't know how to read the demo file to see if it's correct though, but looks promising.

Not sure if this is what the maintainers intended, but it seems ok so far.

 

I seem to be stuck in the same loop as you. Did you ever figure it out?

 

@luke90275 : I stopped experimenting with it, sorry. I did try opening the demo recording file using a hexdump tool but it didn't seem like simple observations and actions, there is some metadata in there as well. I am still using Heuristic() to have humans play as agents because my code was originally using Player Brains. This works ok, but haven't played with recordings.

@AnthonyRUnity : Those are what I was following along with as well. The docs say that the recorder will record an Agent's observations and actions, etc. But there is no way to have a human control an Agent anymore. If we want to train agents to imitate human play, I don't understand how to have a human play, now that Player Brains are gone. Like mentioned earlier, you can kind of use the Agent Heuristic mode and create a UI which captures player input and send those actions in Heuristic() so they are acted on by AgentAction() (and presumably recorded), but I don't know if that was the intended way of doing this.