Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Unite Now - Kinematica – Democratizing Motion Matching for Character Animation

Discussion in 'Animation Previews' started by AskCarol, Apr 28, 2020.

  1. AskCarol

    AskCarol

    Joined:
    Nov 12, 2019
    Posts:
    234
    Unite Now Session

    In this session you’ll learn how to get started with Kinematica and implement it within your animation pipeline. Based on Motion Matching, Kinematica uses raw animation clips or motion capture data – not animation graphs – to produce character poses automatically, based on your description of the intended movement.

    Update (May 29th)
    The demo is now available! We've made the GitHub repository public as well.
    The repo address is - https://github.com/Unity-Technologies/Kinematica_Demo

    The release can be downloaded here.


    When
    Thu, May 7, 9AM PT


    Where
    You can watch the session here.
    Make sure to sign up to get access to all Unite Now content.


    ______________________


    Together with this Unite Now session we are also hosting a Q&A here in this thread to answer as many of your questions as possible.


    A team of product experts from across Unity will be available starting after the session. Our Community team will continue to field questions to foster an ongoing discussion with the community. Please feel free to ask your questions in this thread.


    Some basic rules
    • Don’t bundle multiple unrelated questions in one reply. One question/topic per reply.


    • Only questions related to the topics of the session are permitted.


    • All questions will be fielded by our Community Managers (@LeonhardP and @AskCarol)
      • Replies will have to be approved by the moderators to show up in the thread.


      • Once approved, the questions will be forwarded to the relevant experts.

    We really look forward to hearing from the community.

    Thank you!
     
    Last edited: Jun 4, 2020
  2. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,222
    How can I use Kinematica with a small set of keyframed clips?
    Example of small set: walk and run forward/backward/strafe right/strafe left, vertical jump up, fall down, land
     
    Alverik likes this.
  3. merpheus

    merpheus

    Joined:
    Mar 5, 2013
    Posts:
    202
    Should we motion match with Kinematica for Upper body animations such as aiming and shooting a pistol or swinging a sword etc or use an animator mask with a new animator layer that driven by traditional animations/transitions?

    Second question, if we want to blend Kinematica with IK or traditional animation, how hard it is to make?
     
  4. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    I'm interested. I'd like this system to work out the best anims / pose given a head and 2 hand transforms (a typical VR setup). This will allow a full body representation a lot easier, and work on anything from a Quest to a Console. Will you do that incredibly challenging and inspiring work for me? :)
     
    Alverik likes this.
  5. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    Edit: I misread your question. I'll leave the answer here because it is tangentially related, and answer your actual question at the end

    At the moment, we have limited support for clips that are shorter than what Kinematica calls the "Time Horizon". The default for this setting is 1 second. We want to add implicit support for short, looping clips, but this hasn't made the cut for the first preview. For any clips that are shorter than the Time Horizon, you will need to stitch clips together to have at least 1 second of continuous motion.

    For clips that are at least as long as the Time Horizon, you can provide "boundary clips", which will provide trajectory information outside of the range of your clip. This allows you to have valid frame and trajectory for the whole duration of your clip. Otherwise, your trajectory abruptly stops for last 1 second of your clip.

    For the moment, we support only a single boundary clip for pre and post trajectory extrapolation, so you should use your most likely transition, or the same clip if it's meant to loop. While testing and building the demo content, this hasn't been a problem, but we may adjust if this becomes problematic to trigger correct transitions in specific cases.


    Regarding your actual question: using Kinematica with a small set of clips:
    Yes, you can use it with restricted set of clips, either like a classic state machine with motion matched transitions: "Find me a good pose inside 'Locomotion' to transition to", or by using trajectory matching. How smooth the result will depend on how well connected your animations are.

    You can also use Kinematica as a "state machine if you want to. It's a bit overkill, but it's helpful if you want to force certain clips to play and you don't care about matching. There are samples in the package that showcase this.
    If your set of clips for locomotion is limited to, for example a strict 8-direction motion set, and you want to use path-based motion matching, then you will get better results if you limit your desired trajectory input to strict, 8-directions
     
    Last edited: May 7, 2020
    Alverik and laurentlavigne like this.
  6. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    For your information, the package is already publicly available on the Package Manager , so you can go try it out as you wait for the talk to start.

    Also worth mentioning, there is a Demo project mentioned in the video. Sadly, while the package is available, the demo is not yet available for public release, we need a bit more time to polish it.

    We will do our best to have it available as soon as possible, as well as give targeted samples that detail the more advanced use cases like parkour.
     
    Last edited: May 7, 2020
    Alverik and wetcircuit like this.
  7. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    Q1: It's up to you. Kinematica itself doesn't support partial animations at the moment, so doing this through motion matching would have a memory cost. But, Kinematica goes through the Animator and an PlayableGraph to write to GameObjects, so you can add other AnimationOutputs to overwrite parts of the hierarchy if you want to.

    Q2: It depends. The workflow we have focused on for the first preview is the "Full Kinematica" workflow, where you only use Kinematica, and go through the Kinematica Component. This should interact nicely with things that postprocess the Animation, like the Animation Rigging package or FinalIK.

    If you want to combine it with an AnimatorController, your own AnimationJobs, a Timeline, etc, it is definitely possible, but will require a bit of fiddling with PlayableGraphs. This is also an area where we have spent much less time, so feedback and bug reports are appreciated.
     
    Alverik and merpheus like this.
  8. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    Puppeteering is an interesting use case. I don't think Kinematica will perform well (or at all) in your use case. It is clearly optimized towards finding the next best pose given your current pose (which is usually known ahead of time). In the case of puppeteering, I imagine your possible poses are endless.

    Try it out, tell us if it works!
     
  9. Kaen_SG

    Kaen_SG

    Joined:
    Apr 7, 2017
    Posts:
    206
    I've watched all of Michael Buttner's videos from GDC, Unite, Siggraph etc and I've seen the talks about Dance cards and continuous mocap clips.

    My question is about support for "Cut-Clips". Do you support them and if so, can they be also be combined with dance cards?

    (Cut-Clips are small mocap clips that have been prepared to support the old state machine workflow, for those poor indie devs like me who don't have access to a mocap studio/suit. Currently all Mocap packs on UAS are sold as cut clips)
     
    Alverik likes this.
  10. Kaen_SG

    Kaen_SG

    Joined:
    Apr 7, 2017
    Posts:
    206
    If access to a mocap studio is somehow available to small indies like myself, what is the recommended option:

    1. Provide as much data as possible to kinematica to aid in matching. eg. Overlapping motions & multiple redundancy in motions.

    OR.

    2. Provide only the best, with no redundancy or overlaps (perhaps it confuses kinematica or consumes too much RAM?)
     
  11. Kaen_SG

    Kaen_SG

    Joined:
    Apr 7, 2017
    Posts:
    206
    Regarding the link in the video. " upload_2020-5-8_0-12-1.png

    Github says...

    upload_2020-5-8_0-12-37.png

    Us plebians need permissions. Thanks. :)
     
    Alverik and ftejada like this.
  12. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    This is partially answered here, and in the video. You can definitely use Kinematica without a full dance card. All the samples in the project use a limited move set, for size reasons, and to showcase how to use Kinematica without a mocap studio

    But Kinematica doesn't generate motion, so your results will be as smooth as your input motion. Kinematica will find the best match for you. It may happen that the best match is not a great match. You can still tell kinematica "go to my JumpForward animation, I don't care if it pops".


    The more data you give Kinematica, for a given Tag (e.g.: Locomotion), the harder you will have to work if you want to get this specific animation, and the more specific you will have to be on what joints you use to match these poses.
    This means more processing and more memory.

    But Kinematica lets you force a specific sequence of poses without requiring you to always use the pose matching, or the trajectory matching. If you tag a section of a clip or a clip as "JumpForward3M", then you can tell Kinematica to give you JumpForward3M, and it will happily do so.


    Regarding the demo content:
    This is also mentioned here. I'm sorry, the demo content is not ready for prime time yet.
    The video was produced a few months ago, and while we were hoping to have the content ready in time for today, a global pandemic happened, and our plans were a bit shuffled.

    We're working to make it available as soon as possible
     
  13. Kaen_SG

    Kaen_SG

    Joined:
    Apr 7, 2017
    Posts:
    206
    Thanks. Does kinematica have a preference performance-wise?
    ie. "1000 Cut-Clips" vs a "45min Single Mocap Clip"
     
  14. sqallpl

    sqallpl

    Joined:
    Oct 22, 2013
    Posts:
    384
    Hey,

    Is it possible for Kinematica to blend more than one 'clip' at the same time? I'm not asking about blending between poses/animations on start and end but general blending, layering and layering concept.

    For example it could be a melee locomotion for lower body blended with melee actions for upper body for compound actions like walk forward + weapon swing, step backward + weapon swing, stand still + weapon block, walk forward + hit stagger etc. Or maybe it could somehow use 'melee locomotion' for the base motion while playing upperbody actions at the same time? I'm just wondering here how it can work with things like that, based on some concepts from the presentation.

    In classic animation graph like Mecanim it's usually done with layering, masking and playing more than one clip. For example we play 'stepforward' clip on the legs and root layer, 'weapon swing' on the upper body layer that overrides the base layer and 'head stagger' on the additive layer for head and all of them are controlled independently by script and state machine logic.

    How it could be achieved with Kinematica? Would it require a dedicated 'fullbody' clips that contain fullbody compound motions? Or maybe it can mix separate clips so it could use sets of clips (upperbody, lower body, fullbody, hands only etc.) like one-handed locomotion, one-handed upperbody attacks, one-handed upperbody blocks, upperbody staggers etc. ?
     
    Last edited: May 7, 2020
  15. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    This is partially covered here. There are limitations on how short your clips can be before it becomes tricky. But besides that:

    Runtime-wise, there is no impact. Kinematica only knows about each discrete segment you have decided to tag. There is no functional difference between tagging all your "clips" out of a long mocap clip, or having all your clips separate and the whole clip is tagged.

    Workflow-wise, it's a balance. Longer clips are a bit easier to work with, but once your clips get long enough, it becomes harder to reason about them. It's easy to reason about "RunTurn45Deg: frame 45", but it becomes harder to figure out what animation is actually playing if the current frame is "RunTurnSpirals15To105Deg_mocap: frame 259".
     
  16. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    Kinematica currently only plays one segment at a time. On any transition between poses or segments, it uses a version of inertial blending (see this talk for the general idea). So you will get some amount of blending, but you will not get multiple clips running at the same time. If you want layering, you can setup Kinematica to run in a PlayableGraph, and there you can layer Kinematica with other Animation sources, with IK, or with another Kinematica Synthesizer if you want to.*

    This is partially also covered here

    *As mentioned in the comment above, we have validated that it's possible but this is not where we have invested the brunt of our efforts, so the user experience is not as straightforward, and will require more programming.
     
    CodeKiwi and Alverik like this.
  17. chusmaverde

    chusmaverde

    Joined:
    Jun 10, 2013
    Posts:
    29
    Hello, the Kinematica feature looks amazing, I am eager to try it out but for some reason I am unable to see the preview package in the package manager. I tried to find it in versions 2019.2.6f1, 2019.3.13f1 and 2020.2.0a10.

    upload_2020-5-8_0-45-28.png
    Current screenshot from 2019.2.6f1

    1) I would appreciate some hint on what I am doing wrong.

    2) What is the expected date to release the full demo from GitHub (taking into account the current situation)?

    Thanks
     
  18. brunocoimbra

    brunocoimbra

    Joined:
    Sep 2, 2015
    Posts:
    679
    Taken from 2019.3.9f1
    upload_2020-5-7_20-23-5.png
     
    Alverik likes this.
  19. Kaen_SG

    Kaen_SG

    Joined:
    Apr 7, 2017
    Posts:
    206
    upload_2020-5-8_7-30-3.png

    Shows up for me in 2019.3.12f1 as well.

    If you somehow do not see it in your package manager. You can add it yourself by editing the file "manifest.json" in your project folder /packages/

    and add this line

    "com.unity.kinematica": "0.5.0-preview",
     
    chusmaverde likes this.
  20. chusmaverde

    chusmaverde

    Joined:
    Jun 10, 2013
    Posts:
    29
    Thanks, I do not know what happened, but now I can see the package. :confused:
     
  21. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,222
    Nice! The paper talks about vector inertialization, which I guess means it uses IK, maybe for foot planting. Still no ik in mecanim for generics, so ... no multipede kinematica?

    Guys, if you get this
    upload_2020-5-8_1-9-2.png
    double click the kinematica assets and press build... and go grab a cofee because this can take a while.

    At first I thought the 180 turn in Navigation sample was wonky then I noticed the guy is looking at the floor, he smeared a dookie, so I checked out the clips and saw all that in "Fancy". It's really cool. So I searched the code for some random selector and saw nothing, tags are all "locomotion". How does the system decide?

    BTW: Kinda cool that 200 dudes running even works, how far along is K optimized?
    upload_2020-5-8_1-28-16.png
     
  22. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    The layman explanation for vector inertialization is: "you blend speeds, not positions". It doesn't really necessitate IK per se.

    As for IK for the Animator, AnimationJobs lets you define your own animation post processing, and it's compatible with Kinematica. You can use these to drive transforms procedurally. If you want a bit less low-level, you can use the Animation Rigging package to setup an IK rig. This should work fairly well with Kinematica too. If there are serious friction points that make it really hard to use either of these solutions with Kinematica, we could have simple IK support directly in the Kinematica package, but I would like to avoid overlapping feature sets with existing packages if possible.

    As I mentioned higher up, we haven't spent much time mixing Kinematica with Animation Rigging or Animator controllers yet, but we have a bit of time before 1.0, we can adjust if there are issues.

    Multipede Kinematica?
    Good question. We know it works for bipeds and quadrupeds. I don't see a specific reason why it wouldn't work with multipedes. If I was doing a millipede for example, I would use Kinematica to drive the body shape based on trajectory, then have a post-process IK controller that orchestrates the legs to do something sensible.

    Optimizing the build time is next on our list of priorities. We know that we can significantly decrease the build time by using jobs and burst.

    Better error messaging is also up there.


    The biped sample matches on direction and foot position.
    The bones to match on are defined in metrics (in the Kinematica Asset inspector), so it is all configurable.
    Then there's some hysteresis (defined in TrajectoryHeuristicTask) to avoid constantly switching if the next frame in the "clip" is good enough (so that the head is not popping up and down for no reason).


    Runtime is decently optimized. There are always opportunities, but it's already jobified and bursted. I think major gains could be achieved by integrating more DOTS tech, as a large part of the cost for Kinematica is to go through the Animator to write to Transforms. We want Kinematica to be 1.0 sooner rather than later, though, so we are focusing on GameObject-centric solutions for now.

    Memory size is still mostly unoptimized. We have good ideas, but it's not the top priority at the moment. Usability, reliability and Samples are.
    As mentioned higher up in this comment, build times are a big part of usability that we want to improve next.
     
    Last edited: May 8, 2020
  23. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,222
    Could you add a quadruped sample next time you push a drop?

    I'm very curious to see what weird things you guys manage to animate in Kinematica. And yes I realize you're trying to push 1.0 soon so no rush but for example can I use Kinematica as sole animation engine for steam engines, doors, robots, space ships, robotic-arm-that-move-fruits-from-crates? From a cursory look at how it works, it's just a matter of plopping clips in the Kinematica builder, tag them and then call the tags with code.

    (ideally replacing entirely Mecanim so we only have to use one animation system)

    Ok so 70 humanoids is the cap on a modern machine unless going ECS, humanoids have about 20 bones, I could push 200 4 bones-creatures back on a 2012 mac so ... K is a bit slower than legacy animation (?).
    Is there a quality mechanism in place, sorta like LOD for animation where whatever takes time gets ticked more rarely or with less precision?

    Ok so it found whatever clip matched that configuration and that happened to have these interesting moments captured - very cool. I'll look into this more when the other things settle, like enjoying the sunny days.
     
    Malbers likes this.
  24. chusmaverde

    chusmaverde

    Joined:
    Jun 10, 2013
    Posts:
    29
    Hi there,

    Now that I have been able to install it and play with it a bit. There is a lot of information to be processed and probably I have missed some information. Anyway I have a few of questions:
    1. How do we control contact points? I guess once the full demo at github is available we will be able to see, but in the meantime, how would you control a hand/foot has to touch certain point in a surface?
    2. Is it possible to mix traditional mecanim animator with kinematica? This might be relevant for cases mentioned before when layers are involved (running, running + attacking, etc).
    3. How do we need to set up the animations? with root motion or is there a specific configuration needed?
    4. I see the asset store has a "Huge FBX Mocap Library part 1" (2 and 3 as well). Could those be good enough for Kinematica, or is there going to be a library available from Unity Technologies/ asset store for this?

    I was implementing a climbing controller and I was able to specify the next hand position accurately, the problem is to move the player from point A to point B without sliding along the edge
     
  25. Kaen_SG

    Kaen_SG

    Joined:
    Apr 7, 2017
    Posts:
    206
    1) You use something like FinalIK or unity IK system.
    2) Yes you can still use mechanim with Kinematica if you wish. I think there's some kind of DOTS equivalent in the works too.
    3) Root Motion.
    4) Yes you can use that with Kinematica... it will work but it's not perfect. (But it's free)
     
    chusmaverde likes this.
  26. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    The last minute improvements we made before shipping the public version have unstabilized the behaviour of the demo content, so we need to assess how many changes we need to make. But we have planned our next 3-week sprint to focus on this first. We expect this to be in a decent enough state to show by the end of these three weeks.
     
    chusmaverde likes this.
  27. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,419
    So in terms of believable motion, I think mocap data works pretty decent on its own -- but it is broken when brought into a game's more procedural requirements for animation. For example, there are times where the motion is a bit "stiff" -- For example, when "hopping" over a fence, the motion is very fluid, but even though the hand positioning is correct, the sense of inertia is still wrong. The "magic" in animation happens when motions have a sense of "weight" -- This is the secret recipe for an animation to feel believable. And "vector intertialization" currently kills any sense of weight. The character feels like he is sliding all around the world (which is actually what's happening) -- and as such, it's simple for the observer to pick up on this. Our brains are wired for it.


    Some background in Animation Theory:

    In animation, "weight" is usually conveyed in animation via principles like "anticipation" and "recoil" (or another term for this energy absorption or transfer or dissipation is "follow-through"). Right now, I see no way to handle "weight" in Kinematica (whether through anticipation or recoil/follow-through), meaning actions are just that much less believable.

    You need these frames (marked in red):

    upload_2020-5-12_10-27-18.png

    Kinematica has these frames (marked in gray):

    upload_2020-5-12_10-38-46.png

    As far as I can tell -- ALL of this has to do with with the "vector inertialization" blending being based ENTIRELY around POSE data, which butchers VERY critical things like animation curves for timing movements to imply weight and energy, etc., which clearly can't be (reliably) avoided with Kinematica's approach to pose-picking as it currently exists due to the fundamental problems with weight. "Game Logic" is no place to handle "animation weight" so please don't give me that "figure it out on your own" BS -- since Kinematica clearly fails as a realistic animation solution without a proper solution to represent weight too. Anyone using this for believable procedural animation in games would likely agree.

    But no worries -- I've got my Unity peeps covered!

    I believe a "weight transition" or "energy transfer/dissipation" should be easily be added back in procedurally. This could probably be handled via the physics rig (or perhaps the "animation rig" -- if I remember the terminology you guys posed), but I think a more performant option would be to simply use "vector inertialization" PLUS an extra bit of data derived from the overall length/offset of a chain of bones relative to the (average) bone chain. This would be based on whether that weight is front-heavy, back-heavy, or centered-evenly across a given chain. For example, a giant fist and a small shoulder's weight offset versus a giant shoulder but small hand's weight offset could be simulated with this tiny bit of data that leads to a real sense of "heaviness" from front-to-back. This could be quickly defined for half the rig by just selecting a start bone and end bone for a chain, applying how front-heavy or back-heavy that chain is, and then mirror it to the other side automatically (assuming respective left/right bones are found). This method would add in a "believable" weight to the inertial-vectorization as it occurs, and it could be mapped at authoring time (rather than just runtime). I realize this is an extra step, but it is a critical one to getting believable animation out of this lovely Kinematica system. If you guys need more insight or details on how this could work -- just PM me. I can give a lot more details.

    Please say yes. Please say yes.




    A note regarding usability for Kinematica purposes:

    Regarding usability, I notice that manual animation poses are tedious to achieve in Kinematica.
    No real tooling available yet to author Kinematica-compatible animations (without a lengthy import/export process).
    I animate things manually, and as such I prefer to generate my motions from hand-authored data. So I need a tool that is capable of that to be compatible with Kinematica (and preferably in the Unity Editor).
    That said, have you guys considered that users could use Animation Rigging and Timeline together to generate our poses for Kinematica?

    As far as usability is concerned, without this kind of tool, Kinematica is still a bit early since it is kind of a walled garden.
    In my case, I would need to be able to quickly generate a standard animation clips of poses from various clips blended and tweaked together in something like Timeline + Animation Rigging that Kinematica can use to populate its hours of mocap data (created by hand), so I would need a tool to make this as quick and painless as possible. I think Timeline/Animation-Rigging is perfect, so exporting data for Kinematica from that should be possible.
    This would contribute greatly to how useful Kinematica is for users who do not have hours of mocap data (like myself), or to users who would need to tweak that data anyway (because the run poses look too generic!), and as such, a tool like this would be useful to more than just a "hands-on" guy like me. If I need a shoot left (upper-body twist) and a shoot right (upper-body twist), using a simple blend system like AnimationRigging would be ideal since I could do my blending right in the Editor and output an Animation Clip to pass over to Kinematica that has both motions. A quick layer system (i.e. Photoshop-style Layers) with a blend weight to turn on/off blending of certain animations and certain constraints (from a list view) such as which direction the legs are running in (so I can animate my strafes) would be quite useful too.

    Is something like this in the cards? D:


    Please say yes. Please say yes.
     
  28. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    It's coming in the demo content. We expect to have it available in three weeks. Next minor version of Kinematica should be available in 6 weeks.

    You can do that. If you are using tags, but you aren't using Markers, Trajectory or Pose matching, then it's a bit overkill, because it will essentially be the same as doing Animation.Crossfade or Animator.Crossfade. If you use pose matching, then you probably need your queries to be a bit more precise, so that Kinematica doesn't give you a pose in the middle of your clip just because it's a good match.


    That's an apple to oranges comparison. The biped sample is doing a search through 35000 frames of animation to find the best next pose, and can handle plant-and-turns without doing anything more than dropping the clips in.

    The SnappyLocomotion example is a bit closer to what Legacy does. I took some time to run the profiler on it, and the performance is reasonable. The pure Kinematica/Animation part runs 225 characters in the about same time it takes for OnUpdate to run on the 225 characters to set parameters on Kinematica. Of course, your mileage may vary based on the number of threads you have. Where it hurts the most is the debugger, which has no allocation pooling at the moment, and the code which runs on the main thread to tweak the root motion, which could probably use a bit of optimization.

    At the end of the day, we are a bit constrained because we still need to go through the Animator and Playables to write efficiently to Transforms.

    You can query the synthesizer as often or as little as you want. When you aren't asking anything, it will continue playing frames until the end of the segment. If you don't trigger a segment change, it will stick on the last frame of the segment until you trigger something else. If you don't query the synthesizer, it takes a lot less processing.
     
    florianhanke and laurentlavigne like this.
  29. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    You need to use markers. Markers are a bit like AnimationEvents: they have a time, and an arbitrary payload of data.
    In the builder, you go add the appropriate markers at given frames to mark important events (right hand contact, or left foot plant, etc..).

    The way Michael has it setup in the demo, contact markers have a position and orientation in their payload. These represent the offset from the Root position of the contact, as well as the expected orientation of the surface.

    What he does at Runtime is he processes the environment with raycasts and shapecasts, then he looks for segments with the correct tag based on what he found in the environment, then in these segments, he looks for markers in the future that have the right offset from the current position, and a decent normal. If he finds one, then he returns that pose.

    While that is certainly useful code to have, it is really implementation-specific, which is why it's not included in the base package. We could have included it in the samples, but the amount of content necessary to get parkour going is fairly large, and it would make the package too big.

    You can use IK like Final IK or Animation Rigging to place the hand more finely.

    You can. Kinematica uses the Playables API to interact with GameObjects through the Animator. You can layer multiple AnimationOutputs together, in which you can use an AnimatorController, Animation Rigging, or your own Playables. It is a bit harder than using just one AnimatorController or just Kinematica though.

    With Root motion.

    For Locomotion, the Biped Sample uses a reasonably limited set of animations, and could probably run with way less, we have really just dropped in the files and tagged everything as Locomotion. I believe a Mocap Library named "Huge" should be fine to use with Kinematica.

    As for a Unity one, we haven't decided yet.
     
    florianhanke and laurentlavigne like this.
  30. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    That's a rather long post, and I can't address every single point, but you seem to have a few misconceptions:

    Kinematica does have all the frames in the animation, at the given sample rate. It doesn't use inertial blending in-between poses, just between segments (essentially sub-clips). The default sampling rate of 30 frames per second may be a bit aggressive and may cause some minor loss of movement, but you can increase it if you think it lacks accuracy.

    Also, in normal operations for things like a jump, it doesn't jump from pose to pose every frame, only at junction points. If your velocities match at the junction points, then there will be very little blending.

    A lot of the floatiness of the samples comes from the fact that the camera is naively attached to the hips, which never looks good.


    Most of the tools necessary to do this exist in Unity, or in Packages.
    • The GameObjectRecorder lets you record transforms and save it to an AnimationClip. It has both a GUI and an API. It also supports keyframe reduction. You can also invoke it frame by frame at your favourite framerate.
    • Playables let you blend clips with any weight you want, and you can adjust speeds to match.
    • If Playables are too low-level, you can also use Animator Controller blend trees to do fancy blend trees, and you can drive them easily through parameters.
    • You can also animate Animator parameters, so you don't even need to script your RunTurn_Spirals or your Walk_To_Run
    The biggest thing we're missing is full-blown Character Animation Authoring, which we are interested in doing (Animation Rigging is one of the first steps in this direction), but we don't have any definite timeline on it. Animators are particular, and if you make an Animation tool that's halfway decent, no-one will really use it. It needs to be perfect.

    At the end of the day, yes, having a fully supported tool that lets you generate your clips would be a nice addition to Kinematica, but it's in the territory of feature creep for 1.0. Kinematica already has 3 different windows (Task Graph, Debugger and Builder), as well as the Runtime. That's plenty enough to maintain for now.;)
     
    NotaNaN, jannsen0007 and florianhanke like this.
  31. chusmaverde

    chusmaverde

    Joined:
    Jun 10, 2013
    Posts:
    29
    Thank you very much for your answers @DavidGeoffroy. As I want to learn how to implement this I was trying to use the standard asset player controller work as in Kinematica.

    Goal: I want to remove all the animator blend trees and transitions to only use Kinematica and see if we can get the same behavior.

    When I create my own Kinematica Asset and add the animations in the Kinematica Asset Builder I run into a problem with my metrics saying it is too short:

    upload_2020-5-13_0-0-51.png

    As I do not understand well the metrics I am just using the same as in the Short Animation Clips demo.

    upload_2020-5-13_0-4-10.png

    The tags are set to the ones I have created (crouch and standing). Is there a way to get the metric fit the animations?
    I can see that most people would only have access to this kind of short animations. You did mention something about small clips:

    and some of the clips are below 1 second, while other are just 1.02s long. All have the same issue. For example:

    upload_2020-5-13_0-12-45.png

    Are there plans to have a proper tutorial in Unity Learning (or somewhere else) to learn how to create a Kinematica setup from scratch (of course given some animations)?

    Additionally, it would be nice to have some documentation on the Query API with examples on how to use certain functions.
     
  32. Onat-H

    Onat-H

    Joined:
    Mar 11, 2015
    Posts:
    195
    Do the Characters have to be set up in a different way? I have noticed that the example character is set up as a generic instead of humanoid character, with the hips being the root node. My humanoid character doesn't move along the animation trajectory in the preview. ()
     
    laurentlavigne likes this.
  33. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    upload_2020-5-14_12-18-24.png

    The first and last "Time Horizon" frames cannot be used by Kinematica, because Kinematica needs "Time Horizon" seconds of past and future motion to calculate matching. This is indicated on the metric bar by the darker areas.

    This is not normally an issue with mocap, which always has unused frames before and after, but it can be an issue if you have short clips.

    If you need your whole clip to be used, you can use the Boundary Clip Selectors to provide additional frame information outside of the clip range.
    upload_2020-5-14_12-23-49.png

    You can toggle it using this control:
    upload_2020-5-14_12-25-11.png

    Assuming your clip is loopable, you can just put the same clip before and after.
    But all of this will only work if your clip is at least "Time Horizon" long. If your clip is under 1 second, then you will need to stitch it together with another clip so that we have enough frames to work with. There are multiple ways of doing that, but an easy one is to copy/paste the keys from two clips to create a 3rd one.

    For the moment, we are very much focused on improving the tool itself. We are taking your questions and feedback, and using this to make things easier or more evident. As Marketing and Content get involved, we will probably have better learning content available.

    Do you mean on top of the Manual Page, the Scripting API Page and the samples?
    The Scripting API is not fully fleshed out yet, but if you have suggestions of things you'd like to see explained, we can definitely improve the docs.
     
    chusmaverde likes this.
  34. SenseEater

    SenseEater

    Joined:
    Nov 28, 2014
    Posts:
    84
    You can definitely do much better job of highlighting the first & last frame reservation thing in editor that you have explained. I was also stumped by the same issue.

    And also why is boundary clip option hidden in such obscure way? I managed to find it out on my own when tinkering with the editor earlier but only after 2-3 hrs. And that too when there is not much going in 3 editor views that are shipped with package.
     
  35. SenseEater

    SenseEater

    Joined:
    Nov 28, 2014
    Posts:
    84
    @DavidGeoffroy

    Please throw some light on this as well. i am also facing this issue.
     
  36. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    Kinematica cannot ingest Humanoid clips. It requires access to the curves of all the transforms, which Humanoid doesn't provide. For the same reasons, it also doesn't support animating a Humanoid character, because Humanoids cannot be animated at Runtime through Transform values. We should add some error messaging around this when you try to add a Humanoid clip.

    That said, since we are aware that not everyone has access to their own mocap, and that users will need to get animations from multiple sources, in Unity 2020.1, we've added to the Animator the necessary APIs so that we can support using clips from different skeletons in Kinematica.

    By using Humanoid at build time, we can retarget clips Generic from multiple sources to play to a single Generic character, as long as you can provide us valid Humanoid avatars for the source rig of the clip, and the rig you want to play on.

    That feature still needs a bit more love though, since we've been focusing on features that we can showcase in 2019.3.
     
  37. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    When you look at the same UI 8 hours a day for a year, some things that you think should be hidden by default might not be evident to a newcomer. It's a complex tool, and managing the amount of information to show is tricky. This is why we do previews and don't go directly to release.
     
    NotaNaN, mutp, laurentlavigne and 2 others like this.
  38. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Hey @DavidGeoffroy,
    I'm interested in checking out the Kinematica preview and I have a question regarding the best approach for joining animations. I have many different movement animations (starts, loops, stops) for different speeds and directions. Does it make any difference if I join them all into a single cycle (idle -> start moving forward -> moving forward loop -> stop moving forward -> idle) or multiple clips (idle -> start moving forward -> moving forward, moving forward -> moving forward, moving forward-> stop moving forward -> idle)? What will give me the best result in the end?
    Thanks,
    Colton
     
  39. chusmaverde

    chusmaverde

    Joined:
    Jun 10, 2013
    Posts:
    29
    Yes I meant exactly those (specifically the last one) :oops: I might have missed the link before. It will be useful to see in a more clear way what options we have (rather than check source code directly). For the future, it would be nice to have examples of each function (some do, but not all) more like the official Unity documentation.

    From what I understand Kinematica will select from all animation subsets until it picks 1 animation to be used (or none). I was wondering, what if you need a mix between animations? Does Kinematica offer a similar functionality as blend trees? This would make animations much more flexible. If not, is it possible to create such blend tree in mecanim and access it using Kinematica?

    Thanks for all the answers :) and the mention of the boundary clips. Good reference I did not spot.
     
  40. awesomedata

    awesomedata

    Joined:
    Oct 8, 2014
    Posts:
    1,419
    @DavidGeoffroy:

    First off, let me say that Kinematica is a great concept and technology so far!
    I really appreciate what you guys are trying to do for us animators!

    That said, I've just got to point out that I'm not the only one with "misconceptions" about Kinematica.
    So please, bear with me as I attempt to explain my issue.

    (Btw, thanks for addressing my concerns before -- I didn't think the GameObjectRecorder was still supported!)



    This isn't the problem -- The problem is the overall motion has no mass/weight, and is therefore not believable.

    Weight is carried in a fluid-like way. Kinematica, by default, works in a rigid way (under the hood) despite blending.
    By default, this cannot work to produce lifelike animation.


    To further expand on my point:

    Yes, movement is mostly-preserved (I've said as much), but "mostly-preserved" isn't enough to produce quality animation where weight moves fluidly because there is no "weight" carryover from one action to the next (which is what the breakdown of "anticipation / recoil / energy-transference" was meant to detail for you in my previous post).


    I get that you guys are programmers and not animators, but please take it from a professional animator that this is a BIG problem that makes Kinematica produce amateurish-looking animations.

    Why wait until people publish their games to gain insight that Kinematica animations suck?

    I don't like being a naysayer, but apparently nobody else wants to be the guy who says it.
    I'm pointing this out right now that, no matter how well-done your animation is, you're going to produce sub-par animation with Kinematica because of the simple fact that the poses carry no "weight" from one pose to the next.
    Inertial blending really can help with this, and since you're already doing it to some extent, why not go all the way?

    Reread my post to see a good (and fairly easy-to-implement) solution to this problem (regarding a cheaply "weighted" skeleton). As long as this solution is implemented by someone who already understands both the system and code in-depth (i.e. Unity), Kinematica can easily produce well-done (realistic) animations that carry weight procedurally.
     
    NotaNaN and florianhanke like this.
  41. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    The approach you take depends a lot on what you want to optimize for. Do you want to:
    • spend less time tagging?
    • have the best possible visual result?
    • have the smallest possible memory footprint?
    • get exactly the frames you want?
    • get the best performance?
    It will of course be a combination of these things, in an order that makes the most sense to you.

    Assuming you have infinite patience for tagging, and you want to keep memory and performance under control, then here is how I would set this up:
    1. Join your clips in a way that mimics how a human would showcase the specific motion. For example: Idle->IdleToWalk->Walk->-Walk->WalkToIdle->Idle.
    2. Tag only the relevant frames: don't tag the Idle sections in this context; you probably don't want 40 copies of Idle with different in/out trajectories in your binary. But have them there so that Kinematica can use them to infer past and future trajectory for the motion you are showcasing.
    3. For any animation that should loop, you should have at least two loops, so that there exists a set of frames where the past is a transition and the future is looping, as well as a set of frames where the past is looping and the future is a transition.
    4. Avoid tagging the same AtoB animation multiple times if you can avoid it (for size purposes).
    5. If you generate from blend trees, start with a limited set of angles. Kinematica can alternate from segment to segment with blending to achieve the trajectory you want. If you don't like the results, add more angles.
    If you want to spend less time tagging, you can use a short "Stand" animation instead of Idle, and tag the whole generated clip. The idea here is to avoid tagging a long Idle clip multiple times in different contexts..
     
    florianhanke likes this.
  42. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    For things like blend trees for locomotion, Kinematica can jump back and forth between "clips" dynamically to support a blendtree-like behaviour and give you the trajectory you want. Of course, it requires enough data if you want, for example, smooth motion in all directions. You should try it with what you have, and see if you get the results you want.

    As for ingesting blend trees directly: this is not aligned with the philosophy of Kinematica. The goal for Kinematica is to get the most out of your artist driven animations by specifying intent instead of wrestling with state machines. Automated blend tree import is pretty much the opposite of artist driven. It would also be a nightmare of UX to tag properly. We believe it's better overall to keep the Kinematica Builder solely focused on tagging AnimationClips.

    That said, there is a definite need for better examples on how to stitch together Animation Clips that don't come from mocap, as well as a better set of tools to edit clips yourself, not only for Kinematica, but in general. But for now, as much as I am itching to go and hack together something that you can use today, we still need to focus on improving what we have (Builder, Task Graph and Debugger) before we start another set of tools, so that we don't end up with something like this:
     
    NotaNaN and chusmaverde like this.
  43. Roycon

    Roycon

    Joined:
    Jul 10, 2012
    Posts:
    50
    I'm making a DOTS RTS game and im wondering if its possiable to use Kinematica in a pure DOTS environment.
    I've seen the MotionSynthesizer which looks like a great start, and I guess i can feed the Poses from that into the new ECS animation?
    Any clue how preformant just the MotionSynthesizer would be?
    By the sounds of it I can call the MotionSynthesizer on my own schedule which should help.

    Either way it looks amazing and I hope I can use it
     
  44. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    It's been implemented in a way where when we want to transition to DOTS, it should be easy. But we haven't really tried it in this context yet.

    There's no real reason why it shouldn't work, but you'll probably need to build a bit of scaffolding to transfer the poses to either ECS Transform system or to the DOTS Animation system.

    I also think you may hit some issues with the debugging and the task graph, which rely in part on the Kinematica component at the moment.

    Please tell us how it goes
     
  45. Roycon

    Roycon

    Joined:
    Jul 10, 2012
    Posts:
    50
    Awesome thanks for the quick reply, yea figured I would need to do somethings manually like copying the poses around, can't wait to try :)

    I expect I'm a month or 2 away from starting this work but I'll reply or make a post when I have something to show
     
  46. jannsen0007

    jannsen0007

    Joined:
    May 23, 2020
    Posts:
    1
    How to use Unity created Animation Clip (.anim) in Kinematica?
     
  47. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    216
    Is there a way to tweak responsiveness vs realism? Or is this decided for me by the ML?
     
  48. ColtonKadlecik_VitruviusVR

    ColtonKadlecik_VitruviusVR

    Joined:
    Nov 27, 2015
    Posts:
    197
    Thanks for the reply @DavidGeoffroy. I just found the time to start our Kinematica implementation this morning. I have joined my first set of clips and added it to the Kinematica Asset. Using the preview, our trajectory looks like this:

    WalkFwdTrajectory.PNG

    This is our walking forward cycle, should I attempt to clean it up so the trajectory is more aligned with the forward direction? (right now it strafes left slightly) Or will this not make a difference?

    Cheers,
    Colton
     
  49. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    There's nothing specific to be done about the clip, but you still need an avatar for your character.
     
  50. DavidGeoffroy

    DavidGeoffroy

    Unity Technologies

    Joined:
    Sep 9, 2014
    Posts:
    542
    You will get better results if you have trajectories that match the trajectories that you provide to Kinematica in your queries.

    If you have no clip or segment that closely matches the trajectory you ask for, Kinematica may jump around a lot between clips, and unless the poses are really similar across the whole character, it will probably show a bit.
    You'll always get the best result if you can play sequential frames from a single clip for continuous movement (like pressing straight forward).