Unite Now - Kinematica – Democratizing Motion Matching for Character Animation

Is it possible to blend from kinematics to the animator? Let's say the gameplay uses Kinematica and cutscenes use the Animator to do more complex animations such as interacting with other Animators (fight scene). This could even be during gameplay if the player needs to grab another character and push them or something similar, or is this something Kinematica could/will handle as well?

 

Thanks for addressing this -- That's a great answer!
I definitely would like to see how I can use Kinematica for my own purposes, as I do a lot of fast/near-instant animations, so I would definitely like to see how I could make Kinematica's graph automation work for fighter/action style animations. When this functionality is added, perhaps an example/tutorial could be provided to demonstrate its flexibility? -- In my own use-case, I was a bit skeptical, since Kinematica really seemed to be built with a lot of assumptions on how the end-user will use it. This would be fine -- except users like me would be left out in the cold.

A bit of both, to be honest -- For example, if Kinematica relies on a straight 1hour clip of animation data, and I already have my clips separated into smaller clips (of which most are loops or one-shot animations intended to be used on humanoids), rather than making an entirely new resource (i.e. an _actual_ 1hour animation clip) from all the separate, nicely categorized, animations in my project, I'd rather be able to grab and append the clips I want together (concatenate) without making it a separate resource (and possibly duplicating an hour's worth of data to add to the size of my binary).

To be fair, I haven't looked into the concatenation process yet due to time crunches, but the posts here make it seem like a long string of animation in a single, continuous, file is essentially required. And if this is the case, the 'duplication' of animation data (in terms of having to generate a large clip file from many smaller files that would still need to remain intact for humanoids to continue using them as one-shots) would cause a seemingly unnecessary spike in the filesize for my project, as those individual clips would still be necessary for other, less-complex, (humanoid) characters in the game, who each have one-shot or looping animations that use the same assets as the Kinematica clip would be needing. This means either file data duplication (which is a big no-no in game development), or some kind of json-like metadata that will let Kinematica "combine" the clips via processing, without actually duplicating the file data (or in memory.)

I'm not trying to give you guys any grief, as I know you are aiming for a 1.0, but I am curious how this situation is being handled right now (and how it is likely to be handled in the future), as my projects nearly always have this kind of data structure for their animation, and I don't want to have to deal with yet _more_ 'loose' assets floating around my project that are only there for a technical workaround. Hopefully that makes sense?

 

Sweeet! All good news here. Is this with the upcoming update or it's possible with the current version?

Also, how would you handle the other characters' animation with Kinematica? How would it know to transition into being grabbed?

 

Thanks a lot for taking the time of answering all my questions. Sounds very interesting. I'll check the samples.

 

Hey guys, great stuff.

I've been fooling around, and while all the samples work fine, I've tried making a custom character with animations from mixamo, and it doesn't seem to work well.

I'm using a generic avatar for the character, with root assigned as the hip bone. The clips seem to work perfectly fine with standard animator use - the transform of the game character is changed as you would expect and it uses root motion properly.

However, when I import the clip into the kinematica asset builder I'm using on the character, and take a look at the preview, it's moving all weird, wiggling around, and the path doesn't even stay horizontal (it goes up into air a bit, doesn't stay on XZ plane).

After some experimenting, when I use "Root motion node: <Root transform>" in the clip import settings, everything looks fine when I play the preview of the clip inside the kinematica asset builder, but there is not actual green path displayed, and it's only the hip bone that's really moving in space - the game object stays still. Root motion node none gives the faulty behavior I've explained above. Root motion node Hips works ok, but the path is displayed as coming from the hips.

I'm not very experienced with this and I'm new to root motion altogether, and I'm sure I might be missing something.

 

Hello, is there any interface to set playback speed?

 

Thanks for the reply, honestly, I'm so confused at the moment that I don't even know. And I'm not completely sure how root motion in unity works... but here's my thoughts... it will be quite lengthy and some little questions will be interspaced in it, but I'd still appreciate if someone helped me out with it.

//

My game object "Character" has an animator etc. It has two children - "Mesh" which contains the skinned mesh renderer, and the other is just a transform "Hips" and is the root bone of the armature. In the generic avatar settings for the character, I've chosen the "root" as these "Hips" - I assume that this means that this hip bone is used as the root for root motion (i.e. it's movement in the animation space is used for calculating root motion). The Game object "character" lies on the "floor", and the hips are displaced up into the Y axis a little bit - around ~1.05.

When just playing this animation on a normal animator with "apply root motion", everything works fine. So I'd assume that my setup of the avatar is alright.

But when this same animation is imported into the kinematica asset builder and I preview it, the path is noticeably going down into the ground a bit, then up again, etc, and the character wobbles all around left and round. The green preview path is relatively on the ground though, which I think is desireable.

1, First weird thing / side question - what does the "root motion node - <None>" setting actually mean? I'd assume the result would be no root motion... but when using this setting on a generic avatar animation, root motion works fine. So what does this setting do? Does "<None>" in this context maybe more mean like "don't use custom root transform" ?

 

Yep -- this is confusing.

This is why I keep saying UI should not be "phoned in" -- it greatly affects the UX.

Time definitely needs to be devoted to figuring out how to properly convey these ideas in the UI itself.
Unity is notorious for a lack of documentation, so why not just visually indicate (in some fashion) the distinction we need to understand? -- i.e. by automatically setting "hips" to the proper value in the majority of cases, but display a little "caution" icon you can hover over that explains this distinction in-summary via a tooltip or popup? -- (i.e. "click or hover here if your animation looks wrong" should be visually implied.)

Just my two-cents.

 

Thank you for your indepth reply.

Yes, I thought that's what the motion node does - what's confusing to me is that the default setting is "none". It would be a bit clearer to me to say the default the default setting is "root node" or something like that. It's just semantics though I guess - it only matters whether you consider the motion node to be something that is optionally used, that allows you to override the default root node as far as root motion goes (in which case "none" makes sense) or whether you assume it's always used (in which case it would make sense to just set the motion node to a default of "use root node").

Also, I wasn't aware that the root node is implicitly projected in general, I thought that was only the case with humanoid avatars. Just to maybe get a bit better understanding of how the root motion actually works, I'd appreciate any corrections on the following:

You start with an animation. This animation has some sort of root transform inside the animation space, generally something like the hips, projected down or something. When you're now using root motion for a character, each frame unity checks the offset of this root transform inside the animation space, applies this transform offset to the actual game object, and then "removes" this offset from the animation space (otherwise we would end up applying it twice in your actual in-scene representation), and the other transform settings of the body/pose (children of the root node basically) are then applied.

Few things I'm not sure about:

1,how exactly is the "offset" in the animation space calculated and applied? It would make sense that it's done in a pretty "local" way - i.e. if in the animation space you have two keyframes of the root transform position and orientation, let's call them A and B, where B is same as A but displaced in A's forward direction, you would just modify the in-scene character game object by displacing it in it's forward direction. Similar with rotations - it again is done sort of "locally".

2, you mention that the root is generally projected onto the ground. If this is the case, how is any root motion in the y axis even possible? If the root transform stays at the same level through the animation, how could there be any movement up/down? Generally that's undesirable, but surely for jumping etc. it often makes sense - so is there some way to turn that off?

 

Wonderful, all makes sense. Thanks a ton Francois!

 

Quick question. How would one go about using kinematica with combat animations ? Can we lets say set a "attack" trigger and the kinematica will motion match to a attack anim ?

 

Pretty much exactly that.

 

I see Kinematic uses the Unity Licence, I wonder if Unity User can publish a game or create assets for the Asset Store with these Animations of the Demos from Kinematica included?

 

Hello, I found that in Binary.CodeBook.Encoding generated from an animation clip, some boundingBoxe.inverseDiagonal == infinity.
This caused Synthesizer.MatchPose return unexpected result. Is this a bug or what should I do to prevent this?

 

here is the clip:

https://www.dropbox.com/s/zfzu87vps6qxp6f/strafeidle.fbx?dl=0

I use avatar created from this fbx, drag this clip to Kinematica, create a metric, then select L_foot_skin and R_foot_skin joints. When building, I found numValidDimensions in Builder.BoundingBox.cs is zero, causing the inverseDiagonal = infinity

 

Sure, the Model, Clip and kinematica asset wtih their .meta files are included in the Sample.zip.

 

How would one go about making Kinematica multiplayer ready? Would you want to calculate everyone's animation state on each client, to alleviate network traffic? Or would you opt for transmitting animation states over the network, to alleviate cpu usage? What about some sort of hybrid approach that sent high level animation states over the network, but detail states (like running into walls, etc.) was calculated locally?

 

Thank you very much. What does 'a joint in the metric is not used at all' mean?

In my case, when a character finishes an attack, I want to use motion matching to find the closest idle pose from some idle clips. But if I set inverseDiagonal = 0 when there is an error (inverseDiagonal infinite), the result will not be what I expect.

 

Hello
I got this when I tried to start Hello World sample:

--------

A Native Collection has not been disposed, resulting in a memory leak. Allocated from:
Unity.Collections.NativeArray`1:.ctor(Int32, Allocator, NativeArrayOptions) (at /Users/builduser/buildslave/unity/build/Runtime/Export/NativeArray/NativeArray.cs:57)
Unity.Collections.NativeList`1:ToArray(Allocator) (at Library/PackageCache/com.unity.collections@0.9.0-preview.6/Unity.Collections/NativeList.cs:567)
Unity.Kinematica.PoseSet:Create(QueryResult, Allocator) (at Library/PackageCache/com.unity.kinematica@0.8.0-preview/Runtime/Data/PoseSet.cs:31)
Unity.Kinematica.PoseSetp_Implicit(QueryResult) (at Library/PackageCache/com.unity.kinematica@0.8.0-preview/Runtime/Data/PoseSet.cs:50)
Unity.Kinematica.QueryTraitExpressionp_Implicit(QueryTraitExpression) (at Library/PackageCache/com.unity.kinematica@0.8.0-preview/Runtime/QueryLanguage/QueryTraitExpression.cs:72)
HelloWorld.HelloWorld:Update() (at Assets/Samples/Kinematica/0.8.0-preview/Hello World/HelloWorld.cs:23)

--------

Unity ver. 2020.1.0f1
Kinematica 0.8.0
Jobs 0.2.10-preview.13

 

This is the last post I've seen mentioning using this new animation system with DOTS & ECS. Has there been any progress or samples of it working the new way since this reply from late May?

 

Hi, this looks great and I am just getting started.
Can you point me to a discussion of the steps and requirements of a new custom character and rig.
Also any discussion of getting this new rig to work with Kinematica.
I have gone through the documentation and I seem to be missing these points in there.
Perhaps there is another source I don't know about.
I have been trying a number of different things but so far no luck.
Thank you.

 

I have downloaded Kinematica_Demo_0.8.0-preview.zip but I find no Samples dir. The only Scene file I find is Biped. I checked README.md as suggested and that is just a word doc pointing to the git download.
How can I find the Hello World sample?

 

Found it I think ... Kinematica_Demo_0.8.0-preview\Library\PackageCache\com.unity.kinematica@0.7.0-preview\Samples~\Hello World

 

Moving forward a little.
As you can see in the attached image I have my character loaded successfully.
I found that I had to change the names of the bones to what they are in the Biped character to get the animations to play with it. But obviously the normal's and something else are all wacked out.
The export from 3DSMax was fine as you can see in the companion character.
But once Kinematica is associated with the character the bones get all deformed.

Any suggestions on this would be very very helpful.
Thank you.

 

Yes, I understand this.
I have tested Skin application in 3DS Max and Mixamo and there is clearly a problem.
I figured out the solution for easy testing just as you provided.
Can you PLEASE tell me how or at least in what tool you are skinning your characters?
Kinematica is the only environment in which I have ever had this problem.
Thank you.

 

Understood.
Yes, this is a huge problem for us in using Kinematica.
I have invested a great deal of time into this already.
Can you possibly provide that Unity Animator contact so I can get a description of the process
they are using, no matter how extensive, to retarget Kinematica animations to another character?
Or perhaps provide my contact info to them so they could contact me about it?
Thank you.

 

The best way I know right now is to use a DCC app and retarget your animations there. Basically, all of them have the ability to retarget animations (because of motion capture so it's the same process) then you export the animation as if it was made for that rig (same usual exporting).

There's also an app on the Unity asset store that seems to be able to retarget humanoid animations to a generic rig so maybe that would also work.

 

Hopefully you don't have a LOT of animations and a LOT of variations, as this will definitely increase your build size due to the massive number of mostly-duplicate animations. D: