Search Unity

Mecanim: Direct control/creation of animation state machine from script?

Discussion in 'Animation' started by ninjapretzel, Aug 13, 2017.

  1. ninjapretzel

    ninjapretzel

    Joined:
    Jul 19, 2011
    Posts:
    26
    It might just be me, but the workflow for animation states&properties is a big pain.
    When adding lots of similar animations, it takes too long to create a new parameter, create the states, hook up transitions, set transition properties, parameterize transitions, and any other logic that might be needed. For many purposes, I've considered using the legacy animation system, as it was much easier to hook up arbitrary, similar animations. I've looked for a way to do something similar with mechanim for a long time, but I can't find any way to play an AnimationClip on some layer at runtime by reference.

    I'd like to be able to just play arbitrary animation clips using the Mecanim system. It honestly shouldn't require someone to hook up a bunch of parameters and animations individually to make them available to play. I'd potentially like to load animations at runtime and play them. It just seems werid that the animation system is so heavily locked down compared to everything else. Even Shaders can be created/compiled at runtime. I could even piece together a bunch of shaders from text snippets. Why can't I do some similar creation of AnimationController states/machines? Why can't I swap the AnimationClip on some state from a script? Why does the animation system have to be so relentlessly locked down from being controlled from scripts at runtime? Parameters are too indirect to have good control over animations in many situations, and are not flexible enough to provide easily maintainable solutions (all possible parameters must still be defined in the editor).

    I feel like I've missed some major API call that's provided, or sort of trick to making it easier to work with. While I like the power of mecanim, especially the humanoid animation retargeting, from what I can see the interface mechanim exposes is too locked-down. It would be much easier to use, as a programmer, if there were more direct ways to define/control the animations.

    Let me clarify with a somewhat hypothetical example:
    Lets say I have a bunch of animations in the project, all which would be compatable with a number of different rigs.
    Basic movement type animations: Run, Walk, etc
    Lots of attack type animations: AttackKnife00, AttackKnife01, AttackKnife02, etc, AttackSword00, AttackSword01,etc, AttackBow00, AttackBow01, etc AttackGreatsword00, AttackGreatsword01, etc, AttackSpellcast00, AttackSpellcast01, etc
    And some other, arbitrary animations that might need to be played in a lot of contexts: Emote00, Emote01, Emote02, etc, MiscAction000, MiscAction001, etc

    And I want to be able to play some of them on arbitrary animators (for example, through an event system, from a text script written by a designer) which have the supported rig, but don't want to have to define all of the animations on each controller.

    Would there be a way to play an arbitrary animation (provided by data from a text file or something), without having a Mecanim state machine set up specifically to play that animation? It's fairly easy to look up an AnimationClip by name, so that's not an issue. Getting the AnimationClip into the Mecanim state machines is the problem, since I can't do it from script, it has to be set up in the editor, and making sure every controller has 100+ parameters and animations would be very difficult.

    The Mechanim state machine can be useful for some cases, but being forced to hook all of the animations up through states makes it very difficult to utilize in certain situations where there would be a large number of states.
     
    Last edited: Aug 13, 2017
    ModLunar likes this.
  2. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    You can play a clip directly with Playables:

    Code (csharp):
    1.  
    2.         PlayableGraph graph = PlayableGraph.Create();
    3.         AnimationPlayableOutput animOutput = AnimationPlayableOutput.Create(graph, "AnimOutput", animator);
    4.         AnimationLayerMixerPlayable mixer = AnimationLayerMixerPlayable.Create(graph, 2);
    5.         animOutput.SetSourcePlayable(mixer);
    6.         animOutput.SetSourceInputPort(0);
    7.  
    8.         int layer = 0;
    9.         AnimationClipPlayable animationClipPlayable = AnimationClipPlayable.Create(graph, animationClip);
    10.         mixer.ConnectInput(layer, animationClipPlayable, 0);
    11.         mixer.SetInputWeight(layer, 1);
    12.  
    13.         graph.Play();
    14.  

    That'll override the graph and not play the animation controller--to play both you'll need to create an AnimationLayerMixerPlayable and mix in the original graph from the Animator. It's not a concise API, but you can build on top of it.

    There are editor APIs to modify the graphs (everything the animator window uses is a public API), but they're all editor APIs. No idea why the graph is read-only at runtime.
     
  3. ninjapretzel

    ninjapretzel

    Joined:
    Jul 19, 2011
    Posts:
    26
    You posted while I was editing my post for clarification.

    Would the PlayableGraph let me bypass the state machine's parameters/transitions and just play animations directly?

    --Edit--
    Yes, I think this exactly what I wanted. I'm doing some more research into it, but I think I can make this work. I might end up replacing all of my animation stuff with something that uses these custom graphs.

    Two concerns I have are:
    -If there are lots of animated objects that play the same kinds of animations, can the Graphs be shared easily, without the actual animations playing being shared (say, all of my emoteable characters draw from one emote playgraph, and if I tell one of them to play some emote, the one I play it is the only one that plays it, and the rest of the animators will continue what they were doing?)
    -What kind of overhead does using this playgraph have? Could 100-1000 animated objects with different playgraphs be extant at the same time on an average PC (Given the renderers are effecient enough)?
     
    Last edited: Aug 13, 2017
  4. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    That code will play raw AnimationClips, and the rest of the Playable API lets you do most or all of the other things an Animator graph does, like mixing clips and working with masks. It won't do anything special inside an Animator graph itself, it sits around it or in place of it.
     
  5. ninjapretzel

    ninjapretzel

    Joined:
    Jul 19, 2011
    Posts:
    26
    I eventually did stumble upon the manual pages for it at
    https://docs.unity3d.com/Manual/Playables-Graph.html

    This system is amazing, I wish that there was more documentation and examples for it.
    Thank you for bringing my attention to it, I have a new project to work on for a few weeks now.:)
     
  6. theANMATOR2b

    theANMATOR2b

    Joined:
    Jul 12, 2014
    Posts:
    7,782
    I think there was a pretty extensive Unity blog post about playables several months ago, maybe a year ago.
     
  7. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    5,026
    There's a new blog entry here.

    And an old one, before official release, here.

    I think those are the main ones ....
     
    theANMATOR2b and Mecanim-Dev like this.
  8. Mecanim-Dev

    Mecanim-Dev

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    1,666
    yes those two blog post are the official one
     
  9. ninjapretzel

    ninjapretzel

    Joined:
    Jul 19, 2011
    Posts:
    26
    @Mecanim-Dev

    Just some questions since I've been playing around with the systems:

    1. How necessary is it for users to provide the indexes for the connections of the nodes? Is there no way for these indexes to be tracked by the system, so a call could be
    graph.Connect(idle, mixer);
    graph.Connect(run, mixer);
    instead of
    graph.Connect(idle, 0, mixer, 0);
    graph.Connect(run, 0, mixer, 1);

    It seems kinda pointless to provide these indexes, it's something the system should take care of itself.


    2. Since all of the types that are being used are structs (making them value types) doesn't this cause excessive copies and prevent changes when passed by reference, or is there some way to tag the struct so it will still get passed by reference?

    3. Can these graphs be reused across multiple objects? Would it be as simple as adding multiple outputs to one graph for each target animator, or would each in a group of animated objects need its own unique copy of an animation graph?

    I do like that this Playables system is exposed, but there needs to be a lot more extensive documentation and examples for it. That was one thing I really liked about Unity, and I understand that it's very newly exposed stuff, but I feel like I'm working with Unreal Engine with the lack of documentation for playables :)

    There should also be a lot more documentation/examples on how to use the basic
    AnimationMixerPlayable/AnimationLayerMixerPlayable/AnimationClipPlayable/AnimationPlayableOutput classes, rather than what I percieve as a focus on making custom playable types- It seems more useful to hook the built-in playables up in different ways since they basically seem to cover any use case I can imagine.
     
  10. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    (I'm not Mecanim-Dev, but they only respond once in a while...)

    You can use PlayableExtensions.AddInput, which is just Connect that uses the next unused input port, so you have

    mixer.Connect(idle, 0);
    mixer.Connect(run, 0);

    All it does is call GetInputCount and use that as the destination port. There's no default arg for the source port, but it's almost always 0.

    It would be nice if it would return the index it used, so you can say int outputPort = mixer.Connect(idle);

    > 2. Since all of the types that are being used are structs (making them value types) doesn't this cause excessive copies and prevent changes when passed by reference, or is there some way to tag the struct so it will still get passed by reference?

    Nope, it'll always copy it every time. It's not a deep copy, though. A Playable under the hood really only contains a PlayableHandle, which is just an IntPtr and an int, so copying it should be very cheap. That's why changes to one copy of a playable struct are visible in others--you're actually making changes to the native object that it refers to.

    > 3. Can these graphs be reused across multiple objects? Would it be as simple as adding multiple outputs to one graph for each target animator, or would each in a group of animated objects need its own unique copy of an animation graph?

    You can't output a graph to multiple Animators. I've tried it, and Unity flipped out briefly and then crashed.

    It would help a lot to have a bunch of examples in one place in the documentation. There aren't a lot of examples right now, and they're scattered between obscure blog posts and forum posts (and the worst place of all for documentation, videos).

    I haven't found any good examples of real uses of ScriptPlayable. All of the examples are of the "create three big, ugly classes to let you control a single attribute" format, which isn't a practical use at all, and I haven't found any examples of how it works with ports, eg. to read data from a connected AnimatorControllerPlayable's output and spit out data to another node's input. That's the whole point of a graph, but every example looks like this, which isn't using the graph connections at all.
     
    Mecanim-Dev and hopeful like this.
  11. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,471
    I think the main use of ScriptPlayable is for cutscene use in Timeline. It can be generally applied to things that needs to happen over time, but you'd not need to wrap those things as a playable.

    Being able to deep copy an existing playable graph would be a very powerful addition! I'm not quite sure what that'd involve.
     
  12. Mecanim-Dev

    Mecanim-Dev

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    1,666
    @NoiseFloorDev Nice you did a great job answering all thoses questions. Thanks a lot!

    The multiple output are best used when you want a graph to drive an animation, audio and video output in sync, like you usually do in cut scene, but could be used at runtime too if needed for other use.

    The ScriptPlayable is there to help you to organize you're code, it like attaching a behaviour on a Playable, so rather than having all your managing code in one big MonoBehaviour you can split it in many smaller behaviour that will be invoked only when the attached playable is evaluated, like state machine behaviour for statemachine

    If you haven't see this page take a look, there is some examples
    https://docs.unity3d.com/Manual/Playables-Examples.html
     
  13. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    I had tried making one graph drive two Animators, for a cutscene clip that has animation for two characters and drives both of them in sync, and that crashed. I don't recall exactly how I tried it, but I think I created two AnimationPlayableOutputs and connected them both to the AnimationLayerMixerPlayable. If this is intended to work I can try to repro it.
     
  14. Mecanim-Dev

    Mecanim-Dev

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    1,666
    Playable Outputs should never be connected as input on anything,
    This is not supported, an AnimationPlayableOutputs should be never connected as an input to another playable.
    But it shouldn't crash. If you still have the project somewhere could you log a bug with this project?
     
  15. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    They weren't inputs to anything, I don't think the API would allow that (different interfaces). I created two AnimationPlayableOutputs, made the same AnimatorControllerPlayable (not AnimationLayerMixerPlayable, that's one step further down) an input to both of them, and added the outputs to the graph. I don't have the code, since it was just a quick test to see what would happen, but I think it was roughly just

    graph = PlayableGraph.Create();
    playable = AnimatorControllerPlayable.Create(graph, animationController);

    output1 = AnimationPlayableOutput.Create(graph, "output1", animator1);
    output1.SetSourcePlayable(playable);

    output2 = AnimationPlayableOutput.Create(graph, "output2", animator2);
    output2.SetSourcePlayable(playable);

    graph.Play();
     
  16. ninjapretzel

    ninjapretzel

    Joined:
    Jul 19, 2011
    Posts:
    26
    So I've gotten busy with other stuff over the past week, and havent had a chance to check this thread and work on learning this stuff.

    @NoiseFloorDev
    Thanks for the answers above.

    @Mecanim-Dev
    I saw that example page you linked above. I find some of it is pretty useful, but I think that there should be built in methods to create some of the common constructs (like making a blend tree AnimationMixerPlayable directly from an array of AnimationClips, masking animations or changing the blend mode to additive).
    I could create some extensions for some of these for myself, but I think that some procedures for creating the really common constructs should be easily provided by the libraries, with the ability to go deeper if more complexity is needed. I couldn't find the place to apply masks/change animation blending myself, and I don't see any of it in the examples.

    Another example (I think on a different blog post somewhere) had a system built with the graph system, that was similar to the legacy animation system. There was no source code for it, but I'd be interested in looking at it (and using/modifying it) if it is available.
     
    Last edited: Aug 25, 2017
  17. Mecanim-Dev

    Mecanim-Dev

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    1,666
    Unfortunatelly there is a know issue with our doc tools and extension method currently, and the playable are implemented with extension method so all the doc for extension method doesn't appear right now in the documentation. :(
     
  18. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    Masks are still editor-only (baked into animation layers) and not part of the playable API, as far as I know...

    The extension docs are there, just not inline with the extended classes, eg. https://docs.unity3d.com/ScriptReference/Playables.PlayableExtensions.html. If there are extension methods that are completely missing from the docs then it would be nice to get a pointer to them.
     
  19. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,471
    You can add Mask. AnimationLayerMixerPlayable (the mixer that corresponds to the Animator Controller's layers) has a SetLayerMaskFromAvatarMask method to do just that. I've tried it, it works exactly the same as the one in the controller.

    Now, that method could've been named "SetMask", I don't know what's with the long-windedness. I'm still hearing of people using 80 character max line width, and this API tears through that in a single method call.
     
  20. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    That's about at the outer limit, but not unreasonable. Formatting code to 80 characters is just silly--and of course people can do what they like with their own code, but they can't expect the rest of the world to accomodate their silliness. :) A reasonable modern limit is 110-120 characters.

    I see why I didn't see it: it's in UnityEngine.Animations and I was only searching in UnityEngine. It's not an extension method, though, it's just undocumented.
     
  21. ninjapretzel

    ninjapretzel

    Joined:
    Jul 19, 2011
    Posts:
    26
    Well, great, more undocumented stuff.

    Will most of this be properly supported/documented with 2017.2? Or what is the expected version?
     
    Roy-Massaad1 likes this.
  22. NoiseFloorDev

    NoiseFloorDev

    Joined:
    May 13, 2017
    Posts:
    104
    I spent a while the other day just trying to figure out where Texture2D.LoadImage went--it was nowhere in the docs so I was worried that it had been suddenly removed, just to find that it was another victim of undocumented-extension-itis.

    Putting everything in extensions seems to be a bit of a fad these days, so this is getting worse. Please make your documentation system support something before using it everywhere. It seems like decompiling is the only way to find out about a lot of things now.
     
  23. Mecanim-Dev

    Mecanim-Dev

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    1,666
    We know that the documentation is lacking right now, it was all written but at the last minute for 2017.1 everything was moved and the documentation got overwritten without us to notice.

    We are working at fixing this right now in 2017.3 and we will backport it as soon as it ready.
     
    hopeful likes this.
  24. Mecanim-Dev

    Mecanim-Dev

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    1,666
    It was made like this on purpose. The runtime representation of this object is not the same one, so there is a conversion involved in the process and also playable object cannot keep reference to Unity.Object easily.

    Animations.AnimationLayerMixerPlayable.SetLayerMaskFromAvatarMask(uint,AvatarMask)
    summary: Sets the layer's current mask.
    layerIndex: The layer's index.
    mask: The AvatarMask used to create the new LayerMask.
    description: This function generates a layer mask from the specified AvatarMask, and applies it to the specified Layer index. If you change the AvatarMask, you need to call this function again to update the layer's mask.

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4.  
    5. using UnityEngine.Experimental.Director;
    6.  
    7. public class LayerMixerPlayable : MonoBehaviour
    8. {
    9.     public AnimationClip clip1;
    10.     public AnimationClip clip2;
    11.     public Transform leftShoulder;
    12.  
    13.     PlayableGraph m_Graph;
    14.     PlayableHandle m_Mixer;
    15.  
    16.     public float mixLevel = 0.5f;
    17.  
    18.     AvatarMask mask;
    19.  
    20.     public void Start()
    21.     {
    22.         Animator animator = GetComponent<Animator>();
    23.  
    24.         mask = new AvatarMask();
    25.         mask.AddTransformPath(leftShoulder, true);
    26.  
    27.         m_Graph = PlayableGraph.CreateGraph();
    28.         AnimationPlayableOutput playableOutput = m_Graph.CreateAnimationOutput("LayerMixer", animator);
    29.         m_Mixer = m_Graph.CreateAnimationLayerMixerPlayable(2);
    30.         playableOutput.sourcePlayable = m_Mixer;
    31.         m_Graph.Play();
    32.  
    33.         m_Graph.Connect(m_Graph.CreateAnimationClipPlayable(clip1), 0, m_Mixer, 0);
    34.         m_Graph.Connect(m_Graph.CreateAnimationClipPlayable(clip2), 0, m_Mixer, 1);
    35.  
    36.         // Create two layers, second is setup to override the first layer and affect only left shoulder and childs
    37.         m_Mixer.SetInputWeight(0, 1.0f);
    38.         m_Mixer.SetInputWeight(1, mixLevel);
    39.  
    40.         AnimationLayerMixerPlayable layerMixerPlayable = m_Mixer.GetObject<AnimationLayerMixerPlayable>();
    41.         layerMixerPlayable.SetLayerMaskFromAvatarMask(1, mask);
    42.     }
    43.  
    44.     public void Update()
    45.     {
    46.         m_Mixer.SetInputWeight(1, mixLevel);
    47.     }
    48.  
    49.     public void OnDestroy()
    50.     {
    51.         m_Graph.Destroy();
    52.     }
    53. }
     
  25. EthanF4D

    EthanF4D

    Joined:
    Jan 9, 2018
    Posts:
    13
  26. Kybernetik

    Kybernetik

    Joined:
    Jan 3, 2013
    Posts:
    757
    You just pointed out two of the reasons. Creating states, parameters, and transitions can only be done in the editor and is a lot of effort which is usually completely unnecessary when all you really want to do is get an AnimationClip in your script and play it as desired. And override controllers still require you to waste that time and then they force your scripts to use even more magic strings than usual.
     
    ModLunar likes this.
  27. skwsk8

    skwsk8

    Joined:
    Jul 6, 2014
    Posts:
    20
    Guidance and possibly a request for @Mecanim-Dev

    We've got a character with a mecanim that is built to handle about 200 potential animations with multiple transitions in/out of various anims and sub-state machines. We receive an input from the server at the start of the level and may only need a handful of these anims, but we also could need them all. And since we're building for WebGL, we are memory limited and want to only pull in the anims actually being used via addressables so we reduce download/memory.

    The solution I wanted to do was create the mecanim but have the clips be null, then once we've received the input for which anims we actually need, we retrieve those and plug them into the mecanim using an AnimationOverrideController. But since the only way we can access states is by the clips name, that isn't possible. If there were a way to access the states by name, then it would work... could that become a thing? :) [Request]
    If not, the alternative is to put stub animations in the mecanim that are properly identifiable by name, then override them that way. (the real anims are anywhere between 2mb-16mb each, whereas the stubs are only 2kb each)

    Alternatively, the Playable system seems somewhat scriptable... but is it intended to completely create and replace a mecanim with multiple sub states and lots of animations and transitions throughout? It seems like that would be somewhat monolithic to write in code.

    Thoughts? Suggestions? Thanks!!
     
  28. Kybernetik

    Kybernetik

    Joined:
    Jan 3, 2013
    Posts:
    757
    @skwsk8

    The Playables API is entirely scripted (it has no Editor Window or Animator Controller equivalent) and can completely replace Animator Controllers, though it can also play and blend between multiple Animator Controllers if you still want to use them for some things. I built Animancer (link in my signature) using the Playables API based around the principle that you should never need to cram all your animation configuration into one centralised place, you can just give any script a reference to an Animation Clip and play it on demand.

    I agree with you on overriding animations by state rather than by clip name. Back when I did still have to use Animator Controllers I used overrides a couple of times and every single time it would have been ideal to just leave states blank to be overridden. That's also the most common issue I've seen people run into when trying to figure out how overrides work (they try to do it based on state names and it has no effect).
     
    hopeful likes this.
unityunity