Mecanim: Direct control/creation of animation state machine from script?

It might just be me, but the workflow for animation states&properties is a big pain.
When adding lots of similar animations, it takes too long to create a new parameter, create the states, hook up transitions, set transition properties, parameterize transitions, and any other logic that might be needed. For many purposes, I've considered using the legacy animation system, as it was much easier to hook up arbitrary, similar animations. I've looked for a way to do something similar with mechanim for a long time, but I can't find any way to play an AnimationClip on some layer at runtime by reference.

I'd like to be able to just play arbitrary animation clips using the Mecanim system. It honestly shouldn't require someone to hook up a bunch of parameters and animations individually to make them available to play. I'd potentially like to load animations at runtime and play them. It just seems werid that the animation system is so heavily locked down compared to everything else. Even Shaders can be created/compiled at runtime. I could even piece together a bunch of shaders from text snippets. Why can't I do some similar creation of AnimationController states/machines? Why can't I swap the AnimationClip on some state from a script? Why does the animation system have to be so relentlessly locked down from being controlled from scripts at runtime? Parameters are too indirect to have good control over animations in many situations, and are not flexible enough to provide easily maintainable solutions (all possible parameters must still be defined in the editor).

I feel like I've missed some major API call that's provided, or sort of trick to making it easier to work with. While I like the power of mecanim, especially the humanoid animation retargeting, from what I can see the interface mechanim exposes is too locked-down. It would be much easier to use, as a programmer, if there were more direct ways to define/control the animations.

Let me clarify with a somewhat hypothetical example:
Lets say I have a bunch of animations in the project, all which would be compatable with a number of different rigs.
Basic movement type animations: Run, Walk, etc
Lots of attack type animations: AttackKnife00, AttackKnife01, AttackKnife02, etc, AttackSword00, AttackSword01,etc, AttackBow00, AttackBow01, etc AttackGreatsword00, AttackGreatsword01, etc, AttackSpellcast00, AttackSpellcast01, etc
And some other, arbitrary animations that might need to be played in a lot of contexts: Emote00, Emote01, Emote02, etc, MiscAction000, MiscAction001, etc

And I want to be able to play some of them on arbitrary animators (for example, through an event system, from a text script written by a designer) which have the supported rig, but don't want to have to define all of the animations on each controller.

Would there be a way to play an arbitrary animation (provided by data from a text file or something), without having a Mecanim state machine set up specifically to play that animation? It's fairly easy to look up an AnimationClip by name, so that's not an issue. Getting the AnimationClip into the Mecanim state machines is the problem, since I can't do it from script, it has to be set up in the editor, and making sure every controller has 100+ parameters and animations would be very difficult.

The Mechanim state machine can be useful for some cases, but being forced to hook all of the animations up through states makes it very difficult to utilize in certain situations where there would be a large number of states.

 

You can play a clip directly with Playables:

Spoiler

Code (csharp):

1.  
2.         PlayableGraph graph = PlayableGraph.Create();
3.         AnimationPlayableOutput animOutput = AnimationPlayableOutput.Create(graph, "AnimOutput", animator);
4.         AnimationLayerMixerPlayable mixer = AnimationLayerMixerPlayable.Create(graph, 2);
5.         animOutput.SetSourcePlayable(mixer);
6.         animOutput.SetSourceInputPort(0);
7.  
8.         int layer = 0;
9.         AnimationClipPlayable animationClipPlayable = AnimationClipPlayable.Create(graph, animationClip);
10.         mixer.ConnectInput(layer, animationClipPlayable, 0);
11.         mixer.SetInputWeight(layer, 1);
12.  
13.         graph.Play();
14.  

That'll override the graph and not play the animation controller--to play both you'll need to create an AnimationLayerMixerPlayable and mix in the original graph from the Animator. It's not a concise API, but you can build on top of it.

There are editor APIs to modify the graphs (everything the animator window uses is a public API), but they're all editor APIs. No idea why the graph is read-only at runtime.

 

You posted while I was editing my post for clarification.

Would the PlayableGraph let me bypass the state machine's parameters/transitions and just play animations directly?

--Edit--
Yes, I think this exactly what I wanted. I'm doing some more research into it, but I think I can make this work. I might end up replacing all of my animation stuff with something that uses these custom graphs.

Two concerns I have are:
-If there are lots of animated objects that play the same kinds of animations, can the Graphs be shared easily, without the actual animations playing being shared (say, all of my emoteable characters draw from one emote playgraph, and if I tell one of them to play some emote, the one I play it is the only one that plays it, and the rest of the animators will continue what they were doing?)
-What kind of overhead does using this playgraph have? Could 100-1000 animated objects with different playgraphs be extant at the same time on an average PC (Given the renderers are effecient enough)?

 

That code will play raw AnimationClips, and the rest of the Playable API lets you do most or all of the other things an Animator graph does, like mixing clips and working with masks. It won't do anything special inside an Animator graph itself, it sits around it or in place of it.

 

I eventually did stumble upon the manual pages for it at
https://docs.unity3d.com/Manual/Playables-Graph.html

This system is amazing, I wish that there was more documentation and examples for it.
Thank you for bringing my attention to it, I have a new project to work on for a few weeks now.

 

I think there was a pretty extensive Unity blog post about playables several months ago, maybe a year ago.

 

There's a new blog entry here.

And an old one, before official release, here.

I think those are the main ones ....

 

@Mecanim-Dev

Just some questions since I've been playing around with the systems:

1. How necessary is it for users to provide the indexes for the connections of the nodes? Is there no way for these indexes to be tracked by the system, so a call could be
graph.Connect(idle, mixer);
graph.Connect(run, mixer);
instead of
graph.Connect(idle, 0, mixer, 0);
graph.Connect(run, 0, mixer, 1);

It seems kinda pointless to provide these indexes, it's something the system should take care of itself.


2. Since all of the types that are being used are structs (making them value types) doesn't this cause excessive copies and prevent changes when passed by reference, or is there some way to tag the struct so it will still get passed by reference?

3. Can these graphs be reused across multiple objects? Would it be as simple as adding multiple outputs to one graph for each target animator, or would each in a group of animated objects need its own unique copy of an animation graph?

I do like that this Playables system is exposed, but there needs to be a lot more extensive documentation and examples for it. That was one thing I really liked about Unity, and I understand that it's very newly exposed stuff, but I feel like I'm working with Unreal Engine with the lack of documentation for playables

There should also be a lot more documentation/examples on how to use the basic
AnimationMixerPlayable/AnimationLayerMixerPlayable/AnimationClipPlayable/AnimationPlayableOutput classes, rather than what I percieve as a focus on making custom playable types- It seems more useful to hook the built-in playables up in different ways since they basically seem to cover any use case I can imagine.

 

(I'm not Mecanim-Dev, but they only respond once in a while...)

You can use PlayableExtensions.AddInput, which is just Connect that uses the next unused input port, so you have

mixer.Connect(idle, 0);
mixer.Connect(run, 0);

All it does is call GetInputCount and use that as the destination port. There's no default arg for the source port, but it's almost always 0.

It would be nice if it would return the index it used, so you can say int outputPort = mixer.Connect(idle);

> 2. Since all of the types that are being used are structs (making them value types) doesn't this cause excessive copies and prevent changes when passed by reference, or is there some way to tag the struct so it will still get passed by reference?

Nope, it'll always copy it every time. It's not a deep copy, though. A Playable under the hood really only contains a PlayableHandle, which is just an IntPtr and an int, so copying it should be very cheap. That's why changes to one copy of a playable struct are visible in others--you're actually making changes to the native object that it refers to.

> 3. Can these graphs be reused across multiple objects? Would it be as simple as adding multiple outputs to one graph for each target animator, or would each in a group of animated objects need its own unique copy of an animation graph?

You can't output a graph to multiple Animators. I've tried it, and Unity flipped out briefly and then crashed.

It would help a lot to have a bunch of examples in one place in the documentation. There aren't a lot of examples right now, and they're scattered between obscure blog posts and forum posts (and the worst place of all for documentation, videos).

I haven't found any good examples of real uses of ScriptPlayable. All of the examples are of the "create three big, ugly classes to let you control a single attribute" format, which isn't a practical use at all, and I haven't found any examples of how it works with ports, eg. to read data from a connected AnimatorControllerPlayable's output and spit out data to another node's input. That's the whole point of a graph, but every example looks like this, which isn't using the graph connections at all.

 

I think the main use of ScriptPlayable is for cutscene use in Timeline. It can be generally applied to things that needs to happen over time, but you'd not need to wrap those things as a playable.

Being able to deep copy an existing playable graph would be a very powerful addition! I'm not quite sure what that'd involve.

 

I had tried making one graph drive two Animators, for a cutscene clip that has animation for two characters and drives both of them in sync, and that crashed. I don't recall exactly how I tried it, but I think I created two AnimationPlayableOutputs and connected them both to the AnimationLayerMixerPlayable. If this is intended to work I can try to repro it.

 

They weren't inputs to anything, I don't think the API would allow that (different interfaces). I created two AnimationPlayableOutputs, made the same AnimatorControllerPlayable (not AnimationLayerMixerPlayable, that's one step further down) an input to both of them, and added the outputs to the graph. I don't have the code, since it was just a quick test to see what would happen, but I think it was roughly just

graph = PlayableGraph.Create();
playable = AnimatorControllerPlayable.Create(graph, animationController);

output1 = AnimationPlayableOutput.Create(graph, "output1", animator1);
output1.SetSourcePlayable(playable);

output2 = AnimationPlayableOutput.Create(graph, "output2", animator2);
output2.SetSourcePlayable(playable);

graph.Play();

 

So I've gotten busy with other stuff over the past week, and havent had a chance to check this thread and work on learning this stuff.

@NoiseFloorDev
Thanks for the answers above.

@Mecanim-Dev
I saw that example page you linked above. I find some of it is pretty useful, but I think that there should be built in methods to create some of the common constructs (like making a blend tree AnimationMixerPlayable directly from an array of AnimationClips, masking animations or changing the blend mode to additive).
I could create some extensions for some of these for myself, but I think that some procedures for creating the really common constructs should be easily provided by the libraries, with the ability to go deeper if more complexity is needed. I couldn't find the place to apply masks/change animation blending myself, and I don't see any of it in the examples.

Another example (I think on a different blog post somewhere) had a system built with the graph system, that was similar to the legacy animation system. There was no source code for it, but I'd be interested in looking at it (and using/modifying it) if it is available.

 

Masks are still editor-only (baked into animation layers) and not part of the playable API, as far as I know...

The extension docs are there, just not inline with the extended classes, eg. https://docs.unity3d.com/ScriptReference/Playables.PlayableExtensions.html. If there are extension methods that are completely missing from the docs then it would be nice to get a pointer to them.

 

You can add Mask. AnimationLayerMixerPlayable (the mixer that corresponds to the Animator Controller's layers) has a SetLayerMaskFromAvatarMask method to do just that. I've tried it, it works exactly the same as the one in the controller.

Now, that method could've been named "SetMask", I don't know what's with the long-windedness. I'm still hearing of people using 80 character max line width, and this API tears through that in a single method call.

 

That's about at the outer limit, but not unreasonable. Formatting code to 80 characters is just silly--and of course people can do what they like with their own code, but they can't expect the rest of the world to accomodate their silliness. A reasonable modern limit is 110-120 characters.

I see why I didn't see it: it's in UnityEngine.Animations and I was only searching in UnityEngine. It's not an extension method, though, it's just undocumented.

 

Well, great, more undocumented stuff.

Will most of this be properly supported/documented with 2017.2? Or what is the expected version?

 

I spent a while the other day just trying to figure out where Texture2D.LoadImage went--it was nowhere in the docs so I was worried that it had been suddenly removed, just to find that it was another victim of undocumented-extension-itis.

Putting everything in extensions seems to be a bit of a fad these days, so this is getting worse. Please make your documentation system support something before using it everywhere. It seems like decompiling is the only way to find out about a lot of things now.

 

If you want to author the state controller not at runtime, you can https://docs.unity3d.com/ScriptReference/Animations.AnimatorController.html

If you have a base “framework” of animator controller (idle, movement, attack states) and want to use other animation clips in runtime. You should use https://docs.unity3d.com/ScriptReference/AnimatorOverrideController.html
It can be created and swap in runtime

Not sure why devs hype for playables immediately

 

You just pointed out two of the reasons. Creating states, parameters, and transitions can only be done in the editor and is a lot of effort which is usually completely unnecessary when all you really want to do is get an AnimationClip in your script and play it as desired. And override controllers still require you to waste that time and then they force your scripts to use even more magic strings than usual.

 

Guidance and possibly a request for @Mecanim-Dev

We've got a character with a mecanim that is built to handle about 200 potential animations with multiple transitions in/out of various anims and sub-state machines. We receive an input from the server at the start of the level and may only need a handful of these anims, but we also could need them all. And since we're building for WebGL, we are memory limited and want to only pull in the anims actually being used via addressables so we reduce download/memory.

The solution I wanted to do was create the mecanim but have the clips be null, then once we've received the input for which anims we actually need, we retrieve those and plug them into the mecanim using an AnimationOverrideController. But since the only way we can access states is by the clips name, that isn't possible. If there were a way to access the states by name, then it would work... could that become a thing? [Request]
If not, the alternative is to put stub animations in the mecanim that are properly identifiable by name, then override them that way. (the real anims are anywhere between 2mb-16mb each, whereas the stubs are only 2kb each)

Alternatively, the Playable system seems somewhat scriptable... but is it intended to completely create and replace a mecanim with multiple sub states and lots of animations and transitions throughout? It seems like that would be somewhat monolithic to write in code.

Thoughts? Suggestions? Thanks!!

 

@skwsk8

The Playables API is entirely scripted (it has no Editor Window or Animator Controller equivalent) and can completely replace Animator Controllers, though it can also play and blend between multiple Animator Controllers if you still want to use them for some things. I built Animancer (link in my signature) using the Playables API based around the principle that you should never need to cram all your animation configuration into one centralised place, you can just give any script a reference to an Animation Clip and play it on demand.

I agree with you on overriding animations by state rather than by clip name. Back when I did still have to use Animator Controllers I used overrides a couple of times and every single time it would have been ideal to just leave states blank to be overridden. That's also the most common issue I've seen people run into when trying to figure out how overrides work (they try to do it based on state names and it has no effect).