Question about Walker/Crawler examples

Hi all!

I have simple questions about walkers/crawler examples:
1) They got a TouchedTarget function that adds 1 revard of points, it is only initialized, but it is not called anywhere, how do I understand this?
2) is it possible to run imitation training, for example with mixamo animation, where can I find how to run exactly this kind of examples? Or how to run Imitation Learning this examples?
3) If I want to train him on different surfaces (for example, ice or spoles) to walk, how can training be built? Do I need to use raycast to pass the surface type as observation?

thanks all!

 

Some thoughts on your questions:

1) TouchedTarget : From what I can see its unused. Add a breakpoint and run the example. In scene view move the Dynamic Target object right next to the crawler and see if the breakpoint gets hit

2) Imitation learning for a physics character rig to try and imitate animation sequences is not easy.
The following is TorchSharp based (not yet ported to ML Agents) but it took me well over an year to build that. It uses a dense reward method that I kind of called delta between heuristics and agent. Kind of similar concept to GAIL but quite different.
For what you are trying to do imitation learning is probably not worth the effort.
https://forum.unity.com/threads/post-your-ml-agents-project.1005134/#post-7861711

3) Different surface = different friction and / or surface flatness level.
Friction: Simplest way for friction I'm guessing might be to use rigid bodies.
https://docs.unity3d.com/Manual/class-PhysicMaterial.html

Surface flatness level: As for surface flatness level you could just model that on a mesh. Or just get mapmagic and create a terrain with perlin noise applied to its surface. Also if using mapmagic you could regenerate a new surface for each training run.

If you want to do both Friction and Surface flatness level (like rocky vs mud vs ice / metal sheet) you would have to give it some thought though.

 

There are a couple of resources for training from animations, e.g.:
https://github.com/sebastianstarke/AI4Animation
https://xbpeng.github.io/projects/DeepMimic/index.html
And here's one using ML-Agents and physics:
https://github.com/Unity-Technologies/marathon-envs
I think Joe Booth tried a similar approach to what ChillX describes, namely having dense rewards for minimizing the delta between animated states and agent physics.

Here's my usual approach for generating demo files for imitation learning / GAIL:
I have two versions of my character, 1) the animated mixamo rig, 2) a copy of the rig with articulation body physics (they're better than configurable joints for this purpose, because they're meant to be nested, so you can keep the skeleton hierarchy as is).
Joint rotations can be set through continuous agent actions. In order to record a demonstration, there has to be a heuristic function which exactly mimicks the animated rig. It needs to read the animated local rotations and convert them to action values. And vice versa, those agent actions have to be re-converted to target angles for the physics joints.
Depending on how many DOFs my agent joints have, it can be tricky to constrain the possible rotation angles in a meaningful way. Let's say a humanoid character can articulate its body parts on all three axes - simply using quaternions would allow for a lot of 'impossible' postures. In that case, it can help to do a swing/twist decomposition of the animated rotations, with angle limits for the different rotation components.
I think the most challenging part with this approach is tweaking the physics rig so it actually 'works' with a given animation. I found this to be near impossible with humanoid rigs, because they tend to lose balance almost immediately. However, you can cheat a little by applying a counter force to the hips. Or train explicitly for keeping balance. Quadrupeds are easier in this regard, simply because they are more stable.

For touching surfaces, my agents observe a one-hot isTouching value for each foot, set via OnCollisionEnter/Exit methods on the foot colliders. If agents need to handle different friction types, they also have to observe the relative motion of their feet. Together with the isTouching value, they should be able to infer how slippery a floor is. For tilted surfaces, I usually have my agents observe their inclination as a Vector3:
Transform t = agent_root.transform;
Vector3 inclination = new Vector3(t.right.y, t.up.y, t.forward.y);