Skewed rotation when character looks up and down (he also moves side to side)

My character is twisting/rotating/skewing when he should only be looking directly up/down. Honestly I don't know how to explain my problem in words, which is probably why I'm clueless as to how to approach fixing it as well.

Here's an example of what I mean:
https://i.imgur.com/Axnzs9u.mp4

The camera transform (highlighted in the scene view) should not be moving sideways, but it is. Or that is, I don't want it to move sideways, I only want it to move straight up and down, which would mean the head should only move up and down as that's where the camera is positioned, but currently it is moving side to side. And I have no idea how to fix it.

My character setup is what you see in the hierarchy on the right there. I have a camera with a simple mouse look script, currently with the x-axis disabled. Every frame I set the camera position to the character's head position (it could be a child, but I just wanted it easily accessible in the hierarchy) and I set the camera's rotation to match my mouse look input. Then I position the aim target in front of the camera's forward vector, and the Animation Rigging stuff looks at the aim target. There's a head constraint which looks directly at the aim target and there's the spine setup where each subsequent link looks more and more at it, starting with the lowest at the bottom and getting higher nearer the top, to get a smooth bend. They are all using the Multi-Aim Constraint. Currently I have made no changes to the settings for the constraints, they look like this:
https://i.imgur.com/HLQuiqE.png

But I have experimented with all the axes, offsets and limits without having gotten any closer to what I want.

I feel like this shouldn't be so hard, but there are so many variables when it comes to working with 3d rigs and animation that it I find it so utterly frustrating and overwhelming. Like how far back is the root of this problem? Is the rig broken? Is the character idle animation broken? I've probably spent a hundreds hours on this if not more and I'm no closer to what I want and I hate thinking about all the wasted time. I could have spent that time having fun designing games or writing game logic, instead I'm killing myself with this, but I so dearly want a solid foundation to build upon. :/

And if it isn't clear by now I don't exactly have a firm grasp of 3d vectors, rotations etc. It's mostly trial and error until I get the result I'm after, but I want to learn about it and I felt achieving something like this should be manageable for me, but maybe it just isn't. I have tried doing this long before the Animation Rigging package was a thing using the animator IK system which involved quite a bit of scripting, mostly fiddling with the head/body look-at samples provided by Unity over the years, but I've never gotten anywhere. The saddest part is that this is the closest I've gotten so far, and this current attempt is only a few game objects and a few default components. But this is also the reason why the Animation Rigging package has reinvigorated me. I've done so little work with it and gotten so close to what I want.

But I guess the fundamental question I'm struggling with is: Am I supposed to be able to get what I want without scripting with this system? Or am I misinterpreting the entire thing then?

If I make each bone in my rig shoot a ray towards the direction they are facing then I can see that they are all intersecting the aim target so I guess the system is doing exactly what I've told it to do, and instead the problem is that I don't know what to tell the system to get what I want to achieve. But must I involve scripting to get there or should I be able to do it just in the inspector?

And if I need to involve some scripting, where do I even begin?

Also where does this rotation/twisting originate from? In my head if all the bones are facing the aim target then they should also bend directly towards it, but instead they bend out to the side as if they are following the rotations they had originally in the idle animation or being told by something else that they need to rotate like this.

Also, if I don't have an animation on my character and leave him in the t-pose or I have an animation where the hips, spine, head etc. all have no rotation on them then this works as I want it to, but surely that can't be a limitation here? That would be impossible to work with. There are so many games that do this sort of thing and I know Final IK even ships with a demo scene which does this as well, but I want to use free systems so that I can open source my projects, and ironically I want to learn the built-in systems rather than third party solutions because I feel that would be most future proof, but with Unity that hasn't really been the case lately. I'm just hoping that if I finally learn this and get this working that this is something that sticks around and isn't deprecated soon for something else...

I also just discovered this thing which has a really solid looking character with full body IK on him which looks interesting to me, but what is this made with?
https://github.com/Unity-Technologies/animation-jobs-samples

Is that made in a deprecated system? Is Animation Rigging the replacement? Or are they different things? Can they be used together? Or is Animation Rigging the GUI for the animation jobs thing?

And what is this?
https://docs.unity3d.com/Manual/Playables.html

Is this the foundation for both the above samples and the Animation Rigging Package? Can all these things be used in conjunction with each other?

If anyone is able to shed some light on this I would be ecstatic. This is by far the project I want to proceed with the most, but working with it at this point in time, being stuck with this stuff, is excruciating.

 

I'm still very new to this package, so I can't help you on most of those things. However, I did have the same problem with the character looking to the side when I moved the aim target up/down. I figured out that the problem was that the animation put my character at a 45 degree angle, so the multi-aim constraint worked fine horizontally, but it had that offset when aiming vertically.

I thought it would be too hacky to just put in a fixed offset to the multi-aim constraint, but after many days of trial and error I found a combination of constraints that worked for me. I have two copies of the spine bone hierarchy (spine->chest->upperchest->neck->head). The first copy has a multi-parent constraint on each transform that just lines it up with the corresponding bone on the skeleton. The second copy is below the first in the hierarchy and has the multi-aim constraints and another set of multi-parent constraints that align the bones' transforms to the first copy's.

For the multi-aim constraints, I turned off Maintain Rotation Offset and kept all axes enabled. You can adjust the weight on the sourceobjects to change how much that particular bone looks at the target. For the multi-parent constraints, I kept all the settings as default, except for the Maintain Offset, which I set to none.

I still think that this is a pretty hacky solution and I still get a little bit of horizontal rotation when looking up and down, but its good enough for what I want at the moment. In its current state, this package is very hard to get started with, not helped by the lack of any proper documentation or tutorials. Hopefully someone who knows a little more about the package can give us a pointer in the right direction. Good luck with your project, I hope you can get it all figured out.

 

Are you just aimin the head towards the target? or also the spine?
If so - are you using a constraint straight to the bone or are you using another transform with override or parent constraint? if the answer is yes your probably inheriting the rotation of each bone and that makes the weird aiming.

 

I actually found a better way to do this not long after my previous reply. What I did was basically the same as in the Ninja props interaction example, except I used multi-parent constraints instead of override transforms to apply the motion to the skeleton and the "ctrl" transforms had zero rotation instead of being aligned to the skeleton.

 

Holy S***, I hate working with this so much, lol.


I honestly have no idea what I'm doing and nothing about this is intuitive to me. I feel like I'm just trying things without having any idea what the end result will be only to see them fail horribly again and again because nothing does what I expect it to.



This is the closest result I've gotten with my attempt at recreating the "Ninja rig" using control objects and override transforms, but it's exactly the same as what I had before. The pink sphere is the target and the lines are the forward vector of all the bones, which should be pointing at the sphere, but they clearly aren't. The blue line is the camera which is irrelevant here. The pink line is the head, then there's red, yellow, grey, white for each spine link from top to bottom.

In my head if I have controls and then set the bones to follow those controls using the override transform then the bones should point where the controls are pointing, but the bones are still pointing in their original directions. I tried changing all the settings I could find to fix this, but that only gave me the result I showed at the start.

Are you able to share this or show me a screenshot of your hierarchy or something? I have the Ninja character in my project and I've tried replicating what I see on him, but the above screenshot is basically what I've achieved so far.

I'm sorry, but I have no idea what you're asking or suggesting. I have a Multi-Aim Constraint for each bone in my spine and my head and all are set to aim towards my target with default values as shown in the original post.

 

I'll have a go at going more in depth into how I did this - bear with me here.


You can see all the effectors and targets in this screenshot of my character.


This is the full hierarchy for the rig that controls the spine movement and aiming. It can be broken down into three parts.


The first part I call the "control" transforms. They have no components except for a bone renderer, where you can see them in the screenshot in pink. The blue squares in the screenshot are just effectors I put on these transforms to see them a little more clearly. I have approximately aligned them with their corresponding bones, except that I have set their rotation to zero for all axes.
The exact position of these transforms doesn't really matter, but they need to have zero rotation to counteract the looking to the side problem.


This next part has the constraints that actually do the aiming.
"CenterOfGravity" has a Rig Transform on it. I use it in case I need to shift the rotation of the entire character. It should be roughly in the center of the character near the hips, with zero rotation. You can see it in the screenshot as the green circle.

"Pelvis" is responsible for the rotation of the hips. You don't have to include this transform to get this working, I just have them in case I ever need to use them.
"Pelvis_constraint" has a multi-parent constraint constraining "Hips_control" to "Pelvis_target"
"Pelvis_target" represents the target rotation for the hips. I just leave it at zero position and rotation. It is the bottommost yellow cube in the screenshot.

"Torso" is responsible for the rotation of the spine, chest, and part of the upperchest. It has a TwistChain constraint with the following parameters. (You can find this constraint in the Siggraph workshop project, just make sure to use "Twist Chain Final")

"Spine_target" and "UpperChest_target" have multi-position constraints (Maintain offset turned off) that constrain themselves to the "Spine_control" and "UpperChest_control" transforms. They can be seen as the two middlemost yellow cubes.
Additionally, "UpperChest_target" has a multi-aim constraint on it:


"Head_lookat" is almost the same as "Torso". (Unity wouldn't let me call it "Head" because of some weird thing with humanoid.)
It has the same TwistChain constraint as "Torso" except the "Spine" transforms are replaced with the "UpperChest" transforms and the "UpperChest" transforms are replaced with the "Head" transforms.
"Head_target" is the same as "UpperChest_target" except the "UpperChest" transforms are replaced with the "Head" transforms, and it is aiming at "Head LookTarget"


The final section of the transforms are what actually apply the motion to the skeleton.
Each "constraint" transform has a multi-parent constraint: (This is for "Spine_constraint")

Each of these multi-parent constraints constraint the corresponding bones to "control" transforms, and I adjusted the weights of each according to how fast I think they should rotate.
The full weights are: 0.3 for spine, 0.35 for chest, 0.4 for upperchest, 0.8 for neck and 1 for head.
The exception to all of this is "Hips_constraint" where I used an override transform set to pivot space instead of a multi-parent constraint. I'm sure that it would work just as well if you choose to use a multi-parent constraint.

"LookTarget_zero" is like a default location for the look target transforms. I moved it so that it was a couple of metres in front of the character's face.
"LookTarget" has a Rig Transform so that I can move it and have the head and torso look targets update as well.


Well, that's everything. I'll try and fix up an example project, which I will most likely send here tomorrow.
If you have any more questions, I'll be happy to try and answer them, either here or in a private conversation (I think you can do that here?)

 

Thanks for the detailed explanation. Your setup sounds very complex, but using Multi-Parent Constraints instead of override transforms actually got me a lot closer to where I want to be and this finally started to make a tiny bit of sense.

I still have the same rig setup as I had originally (based on the Ninja rig):


But I swapped out all the override transform components with multi-parent constraint components instead and after some fiddling all my bones are finally pointing in the direction I want them to. This is how all my constraint objects are setup now:


The reason why this makes sense to me now is because now I have all the zeroed out controls and then by using the multi-parent constraints I'm telling all my bones to ignore their own rotations and take the rotations from the control objects and using the multi-aim constraints I'm telling all the control objects to point at my target. I feel it makes sense in theory and it seems to more or less work in practice. It does mean I'm currently completely overriding the animation on the spine completely and pointing it directly towards my aim target, but I'm sure various tweaking can remedy that. Only the head actually needs to go where I want it to, the rest of the spine can do what it wants as long as it looks somewhat natural.

It's still quite wonky and buggy though, but I'm hoping I can figure that out. A new problem that arose just now is that the rig is keeping my character from moving so something is constraining him in place, but maybe I've just messed up a constraint somewhere. He's also very snake like in how he bends his spine and he can get into some very unnatural rotations, but at least now I have something I feel I can work with.

So I haven't used the custom twist constraint from the Siggraph sample or anything more advanced yet. I'll see what I can get with what I have at the moment first.

But I would still love to see an example project if you want to share one. Just to get some more insight into how this all fits together. I'll share what I have once I actually have something that I feel works as well for anyone else in a similar situation.

PS: I also realized, as I initially feared, that the actual character setup matters quite a bit as well. My character's bones are pointing in all kinds of directions, some are -Y forward, some are Y forward, some are -Z forward, some are X forward etc. This meant I needed to rotate all my control objects to match the rotation of the bone they were constraining otherwise the character would explode. So ensuring you have a good foundation with all the bones pointing in the same direction etc. is probably very helpful as well. This after all is a character from an Unity 3.x example project so maybe not the best character to use for something like this.

Edit: Constraining his hips made him unable to move even if I set the weight of that constraint to 0 and even though I'm only constraining rotation and not position for any of my constraints, but just deleting the constraint fixed it and it didn't seem to affect the aiming at all. Everything still seems to point where I want it to.

 

I've just finished with the example project, you can find it on my GitHub here: https://github.com/SpiccyMayonnaise/animation-rigging-character-looking-around
It's a little rough around the edges, but it works.

I'll definitely be looking to un-complicate it, since I'm sure that there's some steps that I can cut out of the process. I'll try and keep the example project updated if I ever make any progress.

Regarding the wonky movement in the spine - I found that using the twist chain constraints instead of several multi-aim constraints helped with more natural movements, at least in my opinion anyways. (Plus using less multi-aim constraints results in better performance?) The twist chain is really not that complex, In my opinion Unity should just go ahead and include it as one of the built-in constraints. There's a copy of it in my example project but I suggest you go and look at the Siggraph workshop project (https://github.com/Unity-Technologies/animation-rigging-workshop-siggraph2019), the PDF in the project is pretty good at going through most of the stuff. I couldn't find a video recording of the workshop being done live, though.

Again, if there's any questions, I'll always be happy to help out.
The whole learning process has been somewhat torturous, but I guess that's the price we have to pay for being the first to use this package.

 

Awesome, thanks. You went above and beyond on this.

I also had to resort to the twist chain script from the Siggraph example to get rid of the snake like behavior of my spine. Using only multi-aim constraints work fine for getting a natural spine if they are used on their own, but in combination with multi-parent constraints it feels like the influence setting or whatever you call it on the multi-aim constraint (the slider that goes from 0 to 1 for the Source Objects) no longer sets how much the source object should aim at the target, but instead sets how slow or dampened the look at becomes. Even if set to 0.01 the source object still looks 100% at the target, it just takes much longer to happen. This feels like a bug or at least an interaction I don't understand.

As for using this package versus using the old IK system or writing your own system I feel this package has gotten me a lot closer to what I want a lot faster than what I've been able to before. And my earlier attempts required a lot of scripting (that I didn't really understand).

Your example looks very much like what I have now achieved myself. It's far from perfect for my purposes, but I feel that has a lot more to do with my first person controller than the rig setup. And that is what I must focus on now.

I forked your repository and added my first person controller to the character just to see how he would behave and so far all the same issues I'm struggling with in my project are still present:
https://github.com/oyvind-stromsvik/animation-rigging-character-looking-around

My main problem I think is actually a chick and egg situation. Every frame I'm rotating the camera based on the mouse input, positioning the camera at the character's head and positioning the look at target based on the the camera's forward vector. Then the IK rig looks at the look at target. The problem then is fairly obvious in my head. I have no control over the order in which all of this happens. When I move the mouse the camera rotates which causes the look at target to move which causes the rig to move which causes the position the camera should be in to move, but I don't know in which order this happens and each step relies on the other steps. But when not looking directly up or down it seems to work rather well, but it breaks very much at the more extreme angles and I feel I need to solve this in a better way.

Rotating the camera directly feels wrong from the onset, but at the moment I'm not sure how to approach this another way. If the character is looking directly down then the camera should never rotate around its own axis, but it should rather pivot around the character's pivot/root.

I'll continue investigating this using your project as a base as it's a lot more focused than my mess of a project that I've had since Unity 3.x, and also the character seems to be cleaner with all the bones pointing in the same direction etc.

Edit: And just to make this clear. This no longer has anything to do with the skewed rotations I initially reported, they are fixed now or at least I know how to fix them. This is now just about me trying to achieve a proper full body first person character controller using this system.

 

Hmm...

Well I can tell you where you've gone wrong in regards to changing the weight of the multi-aim constraints. Each frame, the rotation of the "target" transforms does not get reset, so if the weight on the constraint is less than 1, it will just keep rotating until it is aiming at the LookTarget. This is why you are seeing the "smoothed", "dampened" rotation when you move the target around.
What you should be doing instead is changing the weights on the multi-parent constraints on the "constraint" objects. Because the rotation of the skeleton bones gets reset each frame, a smaller weight on the constraint will just apply less of the rotation instead of applying it more slowly, if that makes sense.

I totally agree that this new package has helped me get a working result much faster than I would have gotten otherwise, I just wish I didn't have to do all of this figuring out on my own before I got there.

I don't have much experience with FPS cameras, so take my thoughts with a grain of salt, but I would try and not have the camera as a child of the head because that results in the feedback loop that you are seeing. Maybe find some way of seperating the camera from the head, and then use animation constraints to align them at runtime.
Let me know if you get anywhere with this.

Have you ever used Mixamo's auto rigger? You can feed it almost any model and it will give you back one that is rigged like the model that I am using. Perhaps if you tried it on your model, that would make the skeleton transforms a little easier to work with?

I am glad to hear that my efforts to help were not in vain! Here's to hoping that this package gets sorted out soon.