Search Unity

  1. We are migrating the Unity Forums to Unity Discussions. On July 12, the Unity Forums will become read-only. On July 15, Unity Discussions will become read-only until July 18, when the new design and the migrated forum contents will go live. Read our full announcement for more information and let us know if you have any questions.

Question How to create a 2D sprite with a 3d skeleton?

Discussion in '2D' started by pertholdth, Jun 7, 2024.

  1. pertholdth

    pertholdth

    Joined:
    Feb 28, 2024
    Posts:
    227
    I have been watching the dev of this game and they have mapped billboarded sprites on a 3d skeleton

    here is the link to their system

    I was wondering if this could be done in unity

    obviously, this is no small feat and there are many moving parts, so I figured I will try to explore it one step at a time

    For a start, my question is:

    How can I create bones to form a skeleton in unity?

    From what I gather, .fbx files are usually modeled, rigged and animated in 3d software like blender, and only after that they are imported into unity, where you can only blend the animations

    I was wondering if there was an out of the box feature that would let me start creating bones and mapping textures to the bones, and I would tentavely start from there

    If thats not the way you would tackle this problem, your alternative advice would be much appreciated

    I understand this is the kind of "if you have to ask about it, you arent equipped to do it" - kind of question, but I would still like to give it a try just to set myself at ease before discarding the idea, I wanted to at least give it a try
     
  2. Kurt-Dekker

    Kurt-Dekker

    Joined:
    Mar 16, 2013
    Posts:
    39,571
    You can actually do it by hand right in Unity, just by creating a hierarchy of invisible GameObjects in whatever armature / skeleton shape you want.

    Then you target your animations to those bones and after that, you can "hang" any kinds of geometry you want on the different bones, including sprites if you want. Those things move because the underlying bone moves, not because the animator drives the newly-added thing.

    Then you can set those sprites to billboard and change out depending on direction to camera, like the head in the above linked Youtube.

    Oh amen for sure. Start small... think like a pair of chopsticks with a rubber band at the top walking along... that's doable in 15 minutes or so once you're familiar with things... then play with knees, body, and gradually get more comfortable at a full skeleton, etc.

    There is an IK package out there too which can make doing walk cycles a lot simpler.

    EDIT: just to remind myself, I quick-cheesed a little chopstick leg dude... see attached project, made 100% in Unity except for the lowres textures I photobashed off the google image search.

    Take your favorite head sprite, remove the ball / eyes in my project, and hang your head sprite there. Then swap it out depending on heading to camera, like front head, side head, quarter-side head, back of head, etc.

    Screenshot-KurtGame-133622071432980080.png
     

    Attached Files:

    Last edited: Jun 7, 2024
    pertholdth likes this.
  3. seanrich92

    seanrich92

    Joined:
    Oct 13, 2020
    Posts:
    10
    It is possible to create this kind of system but it would take a lot of trial and error on your part. While I wouldn't say I created the same exact system that they have, I made a version that suits my needs based on what they have publicly shown on their tweets. You just need to tweak whatever you need for your own game. Vector math, shader programming and custom editor experience are things that you might need here.

    You can see what my system can do which was based on on Project Terra's ideas
    https://seanrich92.github.io/tools/bone-sprite-animation-system
     
    Last edited: Jun 7, 2024
    pertholdth likes this.
  4. pertholdth

    pertholdth

    Joined:
    Feb 28, 2024
    Posts:
    227
    in your estimation, lets say you wanted to repurpose your system to animate a horse

    would it be a whole new thing since the horse has a completely different anatomy

    or do you think that all the work you laid would make it easier?

    overall how much time would it take you to repurpose it to a horse, since the main system took you 5 months?

    thanks for the advice!
     
  5. seanrich92

    seanrich92

    Joined:
    Oct 13, 2020
    Posts:
    10
    Probably just setup the bones through my editor, slap in the sprites through another editor tool then adjust the direction and pivot there. Then theoretically it should work already. The creation of the sprites would probably take the most time. It is important to augment your workflow with a lot of custom editors.
     
  6. pertholdth

    pertholdth

    Joined:
    Feb 28, 2024
    Posts:
    227
    and what does your output sprite look like in 3d space?

    I assume that your game is 2d so you "flatten" out the final result? And it would be as if it was a normally drawn sprite (flat as in, it has no z value in an orthographic view)

    Or does the final result have some 3 dimensionality?
     
  7. seanrich92

    seanrich92

    Joined:
    Oct 13, 2020
    Posts:
    10
    Sprites are always "flat" but you can add some dimensionality through shaders by rotating them in a dimension. You might want to do this only on certain body parts that should look skewed depending on the angle towards the camera.
    I do this by changing an enum on my scriptable object that contains the sprite data which adds some value in the shader to do this. Some sprite data do not skew with respect to the camera while some do. It is really up to you which makes sense and which doesn't. You can see an example of this from the sword gif in my blog.
     
    pertholdth likes this.
  8. pertholdth

    pertholdth

    Joined:
    Feb 28, 2024
    Posts:
    227
    thanks

    I was referring to the whole composition on the skeleton, for example, there is a torso sized distance inbetween the right arm and the left arm, if you change the perspective to a side view, the right arm is now supposed to be closer to the camera and the left arm further away

    this is what I mean by sprites existing in a 3D space

    correct me if im wrong, I assume that you finish by displaying the whole combination of small sprites as 1 single object and like the rest of your game, it doesnt use the z axis

    I ask because in project terra they are using 3d terrain and Im not sure if they also allow their characters body parts to exist in 3D to some extent, here is one example of what I mean

    on this timestamp

    if you see when the char falls on the water, the foam that displays on the intersection between the water plane and the feet is not being display at the same height on the sprite

    1.png
    and finally here, the whole sword on a frame that is on the lower section of the sprite, should be fully underwater if a water plane was intersecting normally with a fully flat sprite
    2.png
    however only the feet are shown underwater and the sword is displayed above the water

    this could be done with just masking, its true, but it could also just be that there is actually some tridimensionality and the sword does not intersect with the water plane due to being on a greater height value

    any way it seems there is some segmenting of body parts and how they are treated differently when intersecting with the water, a character slashing a sword southward would result in the sword showing fully submerged since there would be no way to distinguish relative height between the sword arm and the feet

    3.png

    as you see, if nothing is done, the sword will display underwater

    I hope you understand what I mean

    please let me know your thoughts, appreciate the advice once again
     
    Last edited: Jun 7, 2024
  9. seanrich92

    seanrich92

    Joined:
    Oct 13, 2020
    Posts:
    10
    How sprites interact with your terrain/water mesh is an implementation that you have to think about as I didn't need to do anything like that. The problem is that with unity you have less control over components like sprites over a bespoke engine as sprites, meshes etc are part of unity internal engine code. You might need to just use meshes instead of sprites to have them work better and use shaders to get the behavior that you want.
     
    Last edited: Jun 7, 2024
    pertholdth likes this.
  10. pertholdth

    pertholdth

    Joined:
    Feb 28, 2024
    Posts:
    227
    could you tell me what is the proper way to create a rig? From what I gather there are joints and bones? bones are like represented by diamonds and joints connect the diamonds. You can stretch and rotate ( the joints or the diamonds)

    I could spitball and do something that somehow works but I feel like its important to get this behaviour right, so that im not reinventing the wheel

    (is there a specific reason bones are represented as diamonds)