Search Unity

Assets Motion Matching (MxM) Animation System for Unity (Released-beta)

Discussion in 'Works In Progress' started by CptKen, Jan 29, 2019.

  1. kenamis

    kenamis

    Joined:
    Feb 5, 2015
    Posts:
    328
    hey, maybe a silly question, but can your system do strafing and walk/run backward (plus turning and forward)? I think all I've seen in your videos is turning and forward movement (or did I miss one?). You're calculating the direction of movement AND the direction/facing of the character to determine what animation frame to play, right? I'm guessing you just don't have the animations for the strafing in the demos?
     
  2. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    It can do strafing by fixing the facing direction in the future trajectory to the direction you want to strafe. Some issues can happen if you don't have enough animation coverage and transitions though.

    Alternatively, you could also tag your strafing animations with a strafe tag in the provided MxM tagging editor and basically force it to strafe by telling MxM "I'm strafing"

    The animations I've shown are pretty bad for motion matching tbh. They are the Unity Raw Mocap set. I have tested it with strafing though I can't show it because it's for an unannounced project.

    Rest assured I'll have a section on strafing in the documentation.

    P.S MxM allows you to switch between Animator Controller or even your own custom animation system through playable at runtime so if the motion matching doesn't suite a particular situation or you don't have good animation coverage for some situations you can always fall back on that.
     
  3. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    103
  4. chrisk

    chrisk

    Joined:
    Jan 23, 2009
    Posts:
    440
    It brings an interesting point. I guess we need to be able to search the animation more using than just "move forward" vector. For example, we should be able specify some contraints, such as crouch, prawn and whatever the extra parameters(constraints) necessary. I was assuming it's possible, otherwise, it will be very limited.
    So, what kind of search parameters can you specify?

    Of course, I assume that we have animations that does crouch, and prawn already in the db.

    Thanks.
     
  5. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Unfortunately these huge mocap sets are usually not very usable. They tend to have a lot of random animations. I'm not too concerned though, I know the system works and I'll try put together better examples closer to release for both mocap and also cut clips. I'll also provide documentation on how to shoot specifically for motion matching for those going down that route.

    Currently there are 32 tag slots that you can manually set and use for whatever you want. MxM has an easy to use timeline with preview functionality where you can make tag tracks and tag up ranges within your animation. Once you have tagged up the animations like this you would simply call MxMAnimator.AddRequiredTag(ETags.Crouch) <- something like that. Without this functionality and events, Motion Matching is useless for anything but pure locomotion.

    upload_2019-4-3_14-7-55.png
     
  6. chrisk

    chrisk

    Joined:
    Jan 23, 2009
    Posts:
    440
    Wow, great! It makes sense. Adding more tag can fine-tune the search. I really can't wait to try it. Are we getting close to the beta testing? within weeks instead of months? Thanks.
     
  7. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Possibly. Though anything can happen. I certainly hope so.
     
  8. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    785
    Could this be combined with lots of classic animation clips as well to achieve better/automatic blending and eliminate the need for mecanim? Is there anything that makes it so mocap data is absolutely required?
    Or is there maybe a way to help the system out a bit in a situation like this by placing more tags/hints?

    I'm working on a little fun private project where I have tons of animation clips. My goal is actually not getting best quality, but reducing development overhead and simply saving time by not having to deal with mecanim (and if possible still getting "ok" / acceptable animation results)
     
  9. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    The answer to this is not so simple. MxM supports cut clips but that comes with some caveats. Cut clips will take more work to get working nicely than mocap data. Let me explain a few key concepts....

    Concept #1: Continuity
    The reason why mocap data is typically better for motion matching is because at each pose there is plenty of time in the future and in the past. This is important because the system needs to calculate a future and past trajectory for each pose so that it can match the desired input of the player. Typically we're talking up to 1 second in the past or future, though it is customizable.

    With cut clips you will find they are often far under 2 seconds long and even for ones that are 2 seconds, what do you do about poses near the end of the clip? It can't find a trajectory out of thin air. To remedy this, MxM can automatically string together looped clips to determine trajectory and for non-looped clips it allows you to either extrapolate the trajectory based on the motion delta of the last few frames or take the trajectory from another clip (user specified).

    So.... there is more work with cut clips because you will need to tell the system how you would like to determine future and past trajectories if they aren't looping. Either a check box for extrapolation or dragging in another animation to steal the trajectory from.

    Concept #2: Coverage
    Mocap is also good because you can have a lot of it and get good a good range of motions / coverage. With cut clips you tend to rely on a few animations with a few transitions and blend them to fill the gaps. This isn't easily done with motion matching for a number of reasons and I'll go more into detail in concept #3. An example is that you may have walking animations at 45degree intervals but you only have starts and stops in the NSWE directions (pretty common situation). The motion matching will struggle to bridge the gap between standing still and walking at 45 degrees because you don't have a walk_45_start animation.

    Why is this? Well this is the second part of motion matching. You are not just matching the player input (responsiveness), you are also matching the current pose of your character (fluidity). If you don't match pose you will get jumps in your animation and you won't have fluid movement. The whole thing is a balancing act between fluidity (pose)and responsiveness (trajectory).

    I have implemented something I call 'Discrepancy Compensation' which tries to compensate for these gaps by procedurally rotating the model to close the gap. However, what if you are strafing and don't want rotation?

    What tends to happen in situations like this is the system gets stuck in idle or chooses the walk forward and then turn in the direction you are going via the the Discrepancy Compensation.

    In the future (after initial release) I plan to implement a kind of blended animation asset that can be used with the system to cover these gaps. However, you have to consider that is still a bit more work, and you're losing some of the quality. Maybe not as much work as mechanim though.

    Concept #3: Lack of blended trees
    Blend trees and spaces are pretty bad as they approximate animations. As a result, you tend not to have full strafe sets if you want good starts stops and plants (at least you shouldn't). Instead to achieve this have to use some complex logic and restricts your blend spaces to limit poor approximations. Also to blend animations they have to be in sync and that cannot be guaranteed with motion matching as the system jumps through animation clips anywhere at any time.

    In short, motion matching only ever plays one animation at a time and the only blending occurring is to smoothly transition from animation to animation. Sometimes you can get about 10 animations fighting to be king. However, only one will be blending in and all the others will be blending out. It's used for smoothing rather than making animations that don't exist.

    Conclusion:
    With that all being said, I have been able to get some good results with cut clips, though it is more work than just throwing the animations in and smashing the 'Animate' button. Everything I've said above holds true for motion matching in general as far as I know, not just MxM. I know Unity talked up Kinematica like you can just throw in your cut clips. However, I've read the technical paper Michael Butner published at Siggraph on Kinematica and unless they change how it works, it will be even worse if you lack animation coverage. I even tried implementing it the Kinematica way and it was so brutal if you didn't have animation coverage that I reverted to a more traditional motion matching algorithm (there really wasn't much difference tbh).

    So with all that out of the way.... you have to understand that it really depends on the animations you have available and their coverage. Hopefully, based on the information above you can figure out if it will work for you. I'm going to try put out examples of both mocap and cut clips to show how it works in both situations.

    Oh and one last thing. If you have poor mocap, you're actually better to go with cut clips. I'll also include all this information and how to run mocap shoots for MxM in the release documentation for those who want to go down that path.
     
    Last edited: Apr 18, 2019
    dadude123, GuitarBro and hopeful like this.
  10. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    MxM final alpha build has been released to alpha testers. This means beta is just around the corner and it will be available on the asset store soon.

    Regarding beta, I truly believe that the motion matching core of MxM is now complete and optimized. There are many additional features beyond that core that are already implemented. As a result I believe the beta will be relatively short. However, this doesn't mean that MxM will contain every feature you could possibly want in an animation system at first release. Features will be added as I update MxM following initial release.

    Current High Level Features Include:
    - Motion Matching
    - Support for cut clips
    - Support for non-cut mocap
    - Event System (for attacks, jumps, vaults, actions etc.)
    - Dedicated Idle System
    - Tagging System (for different styles, e.g. crouch, combat stance, casual stance etc)
    - Heavy use of Unity Jobs & Burst for optimization
    - Support for animation layers
    - Support for switching between Mechanim and MxM at runtime
    - Support for integration of custom playable graphs
    - Editor Tools for manipulating data & visual debugging
    - Hooks for custom motion model creation
    - Animation warping on events (positional and time)
    - No complex state machine logic
    - PoseMasks (to cull poses that never get used)

    Future Roadmap:
    - Multi Contact Events (e.g. Vault event ideally has 2 contacts, 1. hand on wall and 2. feet contact ground)
    - Animation Speed Manipulation (make your animations more responsive)
    - Managed MxM Updates (Optimisation)
    - Automatic LOD system
    - Angular animation warping for events
    - Debug rewind

    Regarding IK:
    IK is actually a post animation procedural step which actually has nothing to do with MxM. MxM is an animation synthesis machine. It is the same as the mechanim state machine in this way that it simply outputs an animation. Any IK applied after MxM is finished creating animations is not of concern to MxM.

    I have tested MxM with both Unity's built in IK system and FInalIK and can confirm that they work out of the box.

    Optimization:
    MxM is already very fast and well optimized at this point. It runs on all threads thanks to Unity's Job system and and is very fast thanks to the lightning speed of the Burst Compiler. There will always be more optimisation to be had but I believe that it is already performant enough for production. I will to continue to provide optimisation improvements as MxM evolves following release.

    What Now:
    I am doing a final polish pass of MxM's current features and also developing documentation and tutorial videos to satisfy Unity Asset Store requirements. I highly recommend reading the documentation and video tutorials that I will make available before purchase just so you are fully aware of what Motion Matching is and what it's strengths and limitations are.

    I will then submit MxM to the asset store and it may take a few weeks for Unity to approve it. Looking forward to finally getting this out.
     
  11. GuitarBro

    GuitarBro

    Joined:
    Oct 9, 2014
    Posts:
    45
    Awesome! That was faster than expected!
     
    CptKen likes this.
  12. sarum

    sarum

    Joined:
    Mar 2, 2014
    Posts:
    207
    looking forward the release :)
    what will be the price of the asset ?
     
  13. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    103
    Great news! Beta discount would be good :) Cant wait to try.
     
  14. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Not sure on the final price at this point but there will be an early adopter / beta discount.
     
    Last edited: Apr 30, 2019
    razzraziel and HitsuSan like this.
  15. HitsuSan

    HitsuSan

    Joined:
    Sep 28, 2014
    Posts:
    121
    All awesome news cause i'm literally waiting for this to be completed for my project to start but keep also in mind that i wouldn't mind to buy it directly from you or have it linked on your website (or something like that) in order to have in between versions instead of having to wait the long approval process of the asset store, like many dev do nowdays ;) just a suggestion lol
     
    CptKen likes this.
  16. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    It's absolutely doable. Worth noting that following release, I will have a discord server where latest builds will be linked as they are finished rather than waiting for Unity approval. That will also be the best place to get support as I can verify invoices and have private channels for verified users. Verified users may also get access to additional code snippets as examples or for use with MxM through the Discord.

    On a completely unrelated note. I will be releasing the documentation for MxM prior to release (to be linked in this forum). I highly recommend that everyone who is interested read the documentation before purchase. There are so many misconceptions about motion matching that I want any potential customers to know what they are buying so we can avoid issues post sale.
     
    Last edited: May 3, 2019
  17. HitsuSan

    HitsuSan

    Joined:
    Sep 28, 2014
    Posts:
    121
    Nice trailer :D Does it mean it's rc1 ?
     
  18. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Very close :)

    Here's the trailer for those who haven't seen it:
     
    GuitarBro, chrisk, razzraziel and 2 others like this.
  19. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    hopeful likes this.
  20. chrisk

    chrisk

    Joined:
    Jan 23, 2009
    Posts:
    440
    Excellent! I can't wait until it's officially released. I really hope it takes off and steal the thunder from Kinematica. They promised a preview version last year but they don't even bother trying to explain why such a long delay. But we will have MxM shortly. ^^

    MxM may not fit for all of animations and hybrid approach with Mechanim will be inevitable initially and I hope you will show some examples how it can be done soon after the release, so that users can gradually move over to MxM.

    Cheers!
     
  21. dibdab

    dibdab

    Joined:
    Jul 5, 2011
    Posts:
    797
    nice work.

    The animations I've shown are pretty bad for motion matching tbh. They are the Unity Raw Mocap set

    don't know if you tried, maybe you can get better quality by changing the compression ratio in the animation tab.
     
    Last edited: May 7, 2019
  22. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    It's not so much the quality of the animations smoothness themselves. It's the context in which they are captured. The actor is incredibly slow (even compared with other mocap data) which results in un-responsive animations. Also mocap captured for motion matching should be done in a specific way to get best results. The UnityRawMocap is a bit hap hazard in that way. I've tried to massage it to work nicely enough to convey the system.

    I'm not too concerned though to be honest. I would rather show you how MxM performs with crappy animation data because it's not like everyone has a budget to get high quality animation for this system. It would kinda be like cheating to show you it working with amazing mocap when most people don't have the resources for RawMocap.

    I have one video that uses real mocap but it's a bit old and the system has drastically improved since then. Unfortunately it wasn't my mocap so I can't just try it again.

    My focus right now is getting it out there and then for a little bit fixing bugs. Once I'm comfortable with that I will start writing more examples and then additional tools and features. I'd love to ship with all of this but I have to draw a line somewhere or it will never be out.

    I don't think you need to hybrid with mechanim to be honest. Not that there is anything wrong with that, MxM actually supports switching between the two. The actual system is solid and works great. The only place I can see a need for hybrid is if you lack animations. I've tried to compensate for that as best I can and will continue to provide tools to compensate for it but there's not a whole lot motion matching can do to fix that completely.

    The trick to cut clips (or at least one of them) is not to rely on root motion at all. Have a character controller that moves through code and clamp the model root motion so it cannot stray farther than 0 - 0.3m from it. This is probably one of the first example codes I will do because I think it is important.

    BTW all examples will be provided on the discord to verified customers, so if you do jump on board make sure you join the discord after that. This in part is a way to reduce piracy.
     
    christoph_r and HitsuSan like this.
  23. HitsuSan

    HitsuSan

    Joined:
    Sep 28, 2014
    Posts:
    121
    Good cause i've got so many questions about it already lol, i'll keep them for myself until i can test things out though :)
     
  24. hopeful

    hopeful

    Joined:
    Nov 20, 2013
    Posts:
    4,832
    Sometimes developers find that once they release, pretty much all their time is eaten up by support, and it is hard to add new features.
     
  25. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    MxM is currently being used in an active game development project:) so its features will be developed regardless.

    I wholly expect many people to wonder why MxM doesn't make their game for them with one button press and I plan to completely ignore those people too :). I draw a hard line in the sand between support and slave labour. A hard line but I believe a fair line. Always happy to provide reasonable support to reasonable people.

    May as well ask them. I wouldn't mind clearing up any mis-conceptions for anyone reading this thread. Also it helps me add to my FAQ
     
    Last edited: May 7, 2019
    Mohamed-Anis and hopeful like this.
  26. razzraziel

    razzraziel

    Joined:
    Sep 13, 2018
    Posts:
    103
    My root motion is already handled by script (probably final ik or puppet master). Do you think is it really ok? Or need integration at some point?

    Also do you think this type of mocap works ok? (Of course with knowing limitations of Kinect captures)
    https://assetstore.unity.com/packages/tools/animation/cinema-mocap-2-markerless-motion-capture-56576

    I think it captures with lower frame rates and then smooths, but i dont know about the results and how it would effect the mxm. I guess it jitters sometimes (when one of your body part behind another, or bad lighting etc). But recording same animations over and over in same clip without edits(and with some little iterations) would help or they only produce garbage for system?

    You mentioned about some specifics of mocap to capture for motion matching. What are those?
     
  27. HitsuSan

    HitsuSan

    Joined:
    Sep 28, 2014
    Posts:
    121
    Nah, i'm sure many things will only be clear when i can get my hands on MxM. I don't want to waste your time asking questions that can be easily answered just by being able to navigate the interface a little bit or stuff like that.
     
    CptKen likes this.
  28. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    I haven't tested puppet master yet but FinalIK works straight out of the box... IK is just procedural animation that works over the top of animation selection. Of course I haven't tested every possible feature of FinalIK but the typical use cases work.

    For puppet master, I just have to test what happens when it triggers ragdoll. I might have to add something for this but it honestly isn't difficult to do these kinds of things.

    My experience with mocap in general is that it's never as good as you expect, never as good as they show in videos, especially these marker-less solutions. Even IMU mocap like Rokkoko has some severe issues. Heck I've even used a $170k Optitrack system and still was disappointing.

    But forget about quality for a moment. That's not really the concern. Does it give you enough space? When capturing mocap specifically for motion matching you need to have 1 second either side of each motion. For cycles this means at least 1 second either side of your cycle. This can be pretty tough to capture in a small space. If you can't achieve that then you are better off using cut clips even if that means cutting the clips from mocap.

    This is not a limitation only for my asset but all motion matching in general. It's deeply ingrained in the way that motion matching works on a fundamental level.

    Usually mocap is captured with the intention of breaking it up for clips... at least non-cinematic animations. For motion matching this isn't the goal ideally. The goal is to have relatively long clips containing a lot of movements. You will have multiple takes but each take should contain all the different angles required for that type of movement... For example walk plants - all angles. The more angles that you can do the better but there is a reasonable cutoff limit. I've done increments of 22.5 degrees which seems to be industry standard for motion matching but that's probably difficult to achieve for your average user.

    To achieve this you need to plan your shoot with dance cards to achieve this. I'll have a full guide on how to ideally mocap for motion matching in the best case.
     
    razzraziel likes this.
  29. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    The user manual is finally complete. This took much longer than expected.

    https://docs.google.com/document/d/1zBdEQh8nJyOkWnKS5fAwchaE7u0kEvl4fSxlR7S2Wfk/edit?usp=sharing

    It may be big but I've tried to be very comprehensive. It's not just a manual for using all the editors and inspectors... it contains a lot of detail on concepts you need to understand to make the most of motion matching. So don't ignore it.

    This was one of the biggest things standing in the way of submission. I've found a few bugs while writing it... so I'll have one last bug fix pass and then submit it this coming week.
     
    Last edited: May 12, 2019
    mons00n, Rtyper, razzraziel and 2 others like this.
  30. Rtyper

    Rtyper

    Joined:
    Aug 7, 2010
    Posts:
    381
    Can't wait to give this a try once it's out! One quick thing I noticed in your documentation though - in the section about events, you have the colours for the "action" and "follow-through" phases the wrong way around (at least if the screenshot above it is correct) :)

    Anyway, keep up the good work!
     
    CptKen likes this.
  31. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Thanks for the heads up. After writing such a big manual I'm sure there's a few errors Thanks for pointing that out :)
     
  32. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    I've made another trailer which is intended to be more 'informative' than the other one. Check it out

     
    mons00n likes this.
  33. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Hi all,

    Just letting you know that I submitted 'Motion Matching for Unity' to the asset store a few days ago and I am just waiting for approval and then it will be available. This could happen at any time, maybe tomorrow..... but more likely in a week or two.

    Just remember this is beta. There will be bugs. In fact, I've fixed several even since submitting. Be sure to join the discord for the best level of support, reporting bugs and exclusive tutorials / code snippets.

    Also you will get updates early if you are on the discord as Unity takes some time to approve things. Once you are on the discord, send me a PM with your invoice number for access to the verified users section.

    Not too long now :)
     
    Flurgle, GuitarBro, mons00n and 4 others like this.
  34. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    647
    Good day, could you suggest any references to read about the technique?
     
  35. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    There is very little in terms of written reference for motion matching. There is a paper from Siggraph 2018 but it's not free so I can't share it.

    There are, however, a few videos that go in depth on the technique. Here they are:




    Those videos are ordered from least technical to most technical. I think the second one (while poor quality) is the best one in terms of content.
     
    hopeful likes this.
  36. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    647
    I appreciate that.
    Thank you. Any plans to integrate your upcoming asset with the preview version of ECS?
     
  37. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Unfortunately the preview version of ECS is so far from what it needs to be to be usable for MxM that it's not worth trying anytime soon. Every-time I open up ECS preview its changed significantly so it's very hard to maintain code in an environment where the foundation is in constant flux. It's also missing several features that would be required for MxM.

    I will look into ECSifying MxM when ECS is more mature. However, MxM is already using a number of features that were developed for ECS such as Jobs, Burst and the SIMD mathematics library. So it's quite fast and I'm not shy of these new technologies.
     
    Last edited: May 26, 2019
    christoph_r and wobes like this.
  38. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    Feels like



    Since I submitted the asset. Still awaiting approval.

    It's not all bad news though. MxM still gets better every day with many bug fixes since I submitted. I've also changed the minimum required version of Unity to 2018.4 LTS because well it's LTS and much more stable than 2019.1 which crashes for me on a regular basis (still works with 2019.1)

    We might have multi-contact events and improved animation warping soon after release.

    Why would you want this. Well even as simple vaulting action has two contacts. 1 when the hand touches the object and one when the feet touch the ground on the other side. With multi-contact events and warping you can make sure they touch at both points regardless of your geometry (caveat: you still need to code the game-play to detect vaulting points).

    What else could you do with it... well pretty much anything where you want a specific animation action to have precision contact with the world. Wall running anyone?

     
    hopeful and christoph_r like this.
  39. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
    I'm happy to announce that 'Motion Matching for Unity' has just released on the asset store.
    https://www.assetstore.unity3d.com/#!/content/145624

    If you do decide to pick it up, make sure you join the discord server and PM me your invoice number so I can give you verified status. I will try to get back to you as when I can, please keep in mind though, I live in Australia so I may be sleeping. I will response :)

    What does the verified status give you? Well access to the latest builds before Unity approve them, exclusive code snippets, and possibly exclusive examples.
     
    michealbach, christoph_r and wobes like this.
  40. CptKen

    CptKen

    Joined:
    May 30, 2017
    Posts:
    91
  41. GuitarBro

    GuitarBro

    Joined:
    Oct 9, 2014
    Posts:
    45
    Congratz on the Unity feature! Snagged a copy to help support development (and maybe use in a real product someday ;)).