Question How to handle advanced sprite swapping with a 2D skeletal rig?

Hi all,

I'm looking to create a skeletal rig for my game that can dynamically swap out various parts based on its rotation/context. For reference, here are some things that I want my rig to be able to do the following:

• Swap out hands (open hand, fist, back of hand, pointing)
• Use a different head depending on what direction the player is looking
• Change mouths for lip-syncing/expressions

From what I understand, this can likely be accomplished using Sprite Libraries. However, all of the use cases I've seen for sprite swapping have been to change the appearance of limbs while keeping the same visual more or less (eg: people using it to show damage or for changing outfits). This means the different parts are generally on different sprites. For my case, the parts I want to swap out would most likely be part of the same sprite sheet, as they're all part of the same state.

So far, this process seems to be the closest to what I'm looking for, but the process seems very clunky and I can't skin the whole sprite at once.

I imagine no matter what route I take I'll need to do some scripting, but I want to make the process as simple and maintainable as possible. I just don't know if this is the least painful approach.

 

Hello @ibbitzzz, maybe what you are looking for is this ?
https://docs.unity3d.com/Packages/com.unity.2d.animation@3.2/manual/SpriteSwapIntro.html

 

Hello all, thanks for your replies so far!

Yep, this is what I'm trying to accomplish. The video I posted above also uses Sprite Library Assets for this. I'd like to be able to use this to make a detailed rig, but am getting hung up on the details for how to implement such a level of detail.

To help explain what I need my rigs to be capable of I made a small MSPaint diagram.


As you could guess there's a different number of torsos, arms, hands, mouths, etc. So the way I see it it's not possible to just make a rig for the front view, side view, etc. Additionally, there's plenty of animations that require swapping between these variants, so the only solution as I see it is to make all parts swappable within the same rig.

Taking a look at your provided resources, I did have followup questions:

• When using the PSD Importer, is it essentially doing the same thing that the video is doing to create a rig, just using the layer info to position? Or are there other things that happen behind the scenes?
• Is PSD Importer really required to import these as a rig/generate the prefab? It seems like a weird design choice to give tool-agnostic projects a worse experience and prevent them from being able to define default poses.
• How would sprite swapping even work for foreshortened limbs
• I may need to do some squash & stretch for certain animations to help with their fluidity, are there already known ways of accomplishing this? The simplest approach is probably to change the part's scale, but can I do something like enlongating an arm?

Would you be able to point me in the right direction for this? Not really sure where to start with updating bones, rebinding poses (assuming that's what is needed to accomplish this kind of rig)

 

Thanks for your detailed writeup Ted, this answers a lot of confusion points for me!

Totally fair point. As a fellow software engineer, I understand it's probably a product decision & it gave the simplest workflow for 99% of use cases. Hope to see more as the project is fleshed out!

As far as design goes, my initial thoughts would be to define a few scripts to manage rotations/connections:

• AnimationRigPart
  • Manages rotation along the axis of the part's root bone
  • Maps each SpriteLibrary sprite to a rotation value
  • Traverses Transform hierarchy to determine actual rotation
    • eg: If torso is rotated, limbs should also rotate to match
  • On Update/OnValidate change sprite to match rotation
• TorsoAnimationRig
  • Subclass of AnimationRigPart
  • Maps limb locations to a SpriteSkin bone & an offset for each sprite in library
    • Unsure of best way to do this; Quaternion against bone vector or lock it to a point on the mesh?
  • Uses rotation to update the render order/depth of each limb's sprite
• LimbAnimationRig
  • Subclass of AnimationRigPart
  • Handles foreshortening using trig functions to scale bones (?)
  • Blends IK constraints of the limb based on rotation
  • Uses rotation to update the depth of each child sprite (eg: if forearm is in front of bicep)

With something like this, I think I would be able to just animate the rotation values of each AnimationRigPart and the scripts could dynamically adjust the sprites as needed. Other things like hands or mouths and such I can probably just change the sprite manually. For now, squash and stretch can probably just be accomplished via scaling & if I need to bring in FFD then I can probably explore it more then.

I guess just 2 other questions stem from this:

• For positioning limbs on a deformed torso sprite, is there an ideal approach? I'm guessing that since the geometry weights affect deformation it's not the most straightforward answer.
• I can see that bones are accessible through the SpriteSkin, but is there a way to access/modify the mesh procedurally?

 

My thought was that by using the mesh data that I could effectively lean on the existing SpriteSkin logic to handle limb repositioning. Given a starting position for the limb, I could find the triangle that the point is in (or at least closest to), and adjust the position as the mesh (and consequently the triangle) deforms.
One other idea I had that was more of a stretch would be to pull the verticies closer/farther from a bone for more control over squash/stretch, but given that vertices are not writeable its not critical.

 

One thing that could help developers tackle this problem is clarifying how the Sprite Resolver works with the Sprite Skin, bones, weights, meshes, etc.

I want to swap hands, feet, etc, much like the OP was hoping to do. But after getting the sprite swapping up and running it's unclear how the bones from the different sprite skins work together.

 

Thank you @Ted_Wikman for the quick reply. This definitely helps me get my head around things better.

As I experiment, it seems having the exact same skeleton is the limiting factor I wasn't expecting. And within that having the same bone list. Some sort of skeleton fragment system might be nice. I might have 20 different hands but just 3 different heads and 100 different bottoms. It might also be nice to have some sort of X, Y flipping feature available in the resolver.

It sounds like for sprites that get swapped often you don't want Auto Rebind turned on if you can avoid it anyway.

I am noticing that my bone bindings break in multiple ways in the editor if I have a different bone list and swap the sprites using the Sprite Resolver.





You can see it doesn't map the bones as anticipated. It looks like the list of bones needs to be ordered the same. Is that right?

The Entry_0 sprite ends up not being influenced by any of the bone positions, which I am guessing is because there are "unassigned references".

I started my experimentation with an extreme example here so I could figure this out by using a bikini bottom versus a long dress. I was anticipating something would break. Recommendations on how to handle this would be great.

Would it be better to just make the bone list match even if a particular sprite isn't weighted by that bone?

I had copied and pasted the bone rig. Next, I am going to try to use the "Main Skeleton" field to directly reference the skeleton that way.

 

Managed to fix this issue by making the bone list the same and in the same order. So even though the bikini bottom doesn't use the tibia bone at all, we keep it in the list of influences anyway.

Is there any issue with this?

 

I think the easiest and most intuitive way to handle this would be allowing us to map multiple sprites to a single bone in the Sprite Editor - or just make sure the SpriteRenderer component is attached to each bone.

If a sprite is to have a mesh - that's good - but meshes only go so far in animation and the ability to swap sprites on a skeleton for different body part perspectives easily is a huge must-have WITHOUT jumping through hoops setting up complete alternative rigs, using library components, etc - it's not intuitive and has a LOT of drawbacks.

You can work around this by simply slicing up your sprite sheet, adding your bones, then dragging the individual sprites from the pack as objects, adding the Sprite Skin and assembling the object yourself which will then have a SpriteRenderer component attached.

In other applications like Spine2D, sprites on bones are treated as attachments so you can swap sprites as needed as you go through for different hand gestures, perspectives, etc. As it stands it's incredibly cumbersome to do this one little thing in Unity while I feel the overall bone system and mesh is faster and cleaner inside of Unity. I can rig characters faster but this is where everything tends to fall apart and is very much so needed for serious artists.

It seems like the current swap system was more for swapping entire characters to use the same skeleton that all have similar shapes and implementing the Libraries for single items rather than just seeing a SpriteRenderer attached to each bone is a bit weird.

We are able to reposition bones and tweak skeletons during animations so for simple humanoid stuff I don't even see a real use-case for entire copy/paste swaps with libraries as it is far easier to just swap all the sprites on each bone (if the renderer component is attached) than it is to re-rig an entire skeleton with a different skin and use the cumbersome Library feature.

Making a SpriteRenderer component accessible on each bone/limb would be an easier (and faster) approach.

Just my .02

 

Just wanted to express my agreement on all points with jsip.

Replacing individual sprites on a rig during runtime has been a pain, especially as my project has multiple rigs with shared parts and animations. It would be a godsend if there was a way to just slap on an individual sprite replacement without having to make sure it conforms to the bone structure of the rig it's going onto!

Otherwise, the 2d animation tools have been fantastic, but this feature certainly needs work. Please let me know if anything addressing this is in the works; I would prefer to delay working on this aspect of my project until there's an easier way to swap out individual sprites.

 

Appreciate the response!

In my case, the character is a robot so there is no deformation of the sprite, but it is a part of the robot and has a single bone linked to it. To be more specific about my use case; I'm looking to implement a number of shared sprites for hand/arm components that I can swap out based on the specific situation (holding a weapon, leaning against wall, balled up in a fist, relaxed, etc).

 

Given that the thread's title is "How to handle advanced sprite swapping with a 2D skeletal rig?", I am surprised that no one has mentioned pivot points yet. Manually setting each sprite's pivot point in the editor isn't scalable for me.

And I can't find a way to set the sprite pivot via script using SpriteAlignment: https://docs.unity3d.com/2021.2/Documentation/ScriptReference/SpriteAlignment.html

This would make rigging with sprite swapping in mind much easier in my opinion. Writing the code to set the pivot using a Vector2 relative to the original texture seems like code I'd rather not have to write myself given that Unity must have that code somewhere. But that's the only way I can figure it out. That or creating a
TextureImporterSettings
https://docs.unity3d.com/ScriptReference/TextureImporterSettings.html

But I am worried that will muck with the PSD Importer.

 

In my case, I want to use different SpriteAlignments and even custom ones. It just depends on what I am processing. For instance, an all "ear" sprites might be center right but all "hair" sprites could be top center. All "hand" sprites might be custom x=30, y=5