In-depth look at Sprite Switching

Hi all,

I want to start with a few notes: This will be a pretty lengthy post, I apologize for my writing if it becomes lacking, I'll try my best to stay to the point.

Also, I have seen the unite videos about sprite swapping, rest assured. The purpose of this post is to highlight the current issues I'm having with frame-based animations and sprite swapping, and possible solutions to this problem.

So, with that, I want to begin with the current possible ways to handle this, to my knowledge. Feel free to correct me if I'm wrong on any of these points.

1. Set up animations through mecanim, with an AnimationController. If it is desired to swap sprites (skins, outfits, etc) then the sprite can be changed in LateUpdate. Managing sprites would entail ensuring at the least that a list of sprites is kept in some capacity in relation to the animation's sprites, often by string name. Layering sprites requires a new SpriteRenderer to be managed per layer.
2. Set up animations like above, but also animations per layer and per animation. This by far the most labor intensive option. These animations will then be setup in an Animation Override Controller.
3. Creating an editor to register spritesheets in a Texture2DArray, and a shader to swap sprite layer via index of the Texture2DArray. Will not use mecanim though, and will need to be animated via UV through the shader.
4. ??

​

Options 1 & 2:
These options are similar enough to me to almost place them together. Its unrealistic with frame-based animations for any significant amount of sprites to go through the effort of creating individual animation files for every combination of sprites, or to do the same per layer of an animation (hair, clothing, weapons).

Option 3:
This is what this post is going to focus on. This method would mean all characters that share these sprites/layers would be drawn with a single draw call. The only issue/roadblock I've run into is that the filesize of this method grows extremely large very quickly. So at the end of the post, I will ask questions on how this issue could be avoided.

Texture2DArrays

https://docs.unity3d.com/ScriptReference/Texture2DArray.html

If you are unfamiliar, Texture2DArrays are arrays of textures created through code that can be sampled within a shader as if it were a single texture, where the index points to the depth in the stack of textures.

The limits are that there cannot be more than 2048 layers in the array, and that each layer (texture) be the exact same size as the other textures.

Since layers of a frame-based animation all should stack and be used at the same UV for animation, this works perfectly.

Within ShaderGraph even, its easy to make a shader that could sample and combine texture layers into a single composite image on a quad for the final sprite.

Example


Above is a very simplified example to illustrate what I am trying to accomplish. Obviously, the top row of sprites is the base "body", which is drawn over with clothing, hair, etc to generate mix & match outfits.

In order to make this possible, each sprite for an article of clothing or item, will be drawn over the sprite and exported as a sprite sheet separately, so it can be applied to the Texture2DArray at a specific index.

As you can imagine, if I were to draw the minimum for 8 directions of movement, I am at the least required to draw 5 different animations (North, Northeast, East, Southeast, and South) and mirror the Northeast, East, and Southeast animations to become the remaining 3 animations. I am drawing these animations at 8 frames each. Each frame is currently at 32x64, the smallest I can go while leaving room for attack animations.

For the sake of this argument however, I'm going to continue with an assumed 32x64 sprite size, 8 directions, and 8 frames each animation. Lowering to 5 like I could may help, but given we need square textures at the end anyway, It seems it wont help much.

With that in mind, we can assume that a single walk animation would produce a spritesheet 512 pixels tall, and 256 pixels wide. (32 * 8 x 64 * 8)

To be a square sheet (GPU optimization), we could fit two animations on a 512x512 sheet cleanly. Realistically though, I would need more animations than this. Even at 5 directions and 3 repeated directions, I could only fit a 3rd animation leaving one string of sprites blank.

If we jump up to the next power of 2, to 1024, we could fit a nice 4 animations wide, by 2 animations tall,
for 8 total animations on the sheet. It's far better, and I might could settle for this, though more animations would be better.

Here lies the issue though

Already at 512x512, if I only had the sheet attached above:

• Hair
• Shirt
• Pants
• Shoes
• Gloves
• Body
• Weapon
• Accessory

We have (8) 512px textures. If I need more animations, this grows higher. And If I only have 10 variants of each category? Already almost 100 textures in the array, which is growing to be a pretty huge size. This memory while all within the shader would be taking up that full amount of VRAM, correct? It just doesn't seem feasible to work.

So I started thinking about how Unity currently handles this:

Sprites are tightly packed into spritesheets, which the engine handles behind the scenes. If sprites are within the same sheet, they share a single drawcall. With the current number of sprites I have, undoubtedly I would spill over into several sheets. This would be fine however. The number of sheets in disc memory would be the same, but the load on the GPU would be far lighter.

At the same time however, each of those layers would have to be its own object with its own renderer, and required to be managed (sorted within a group).

So I guess my questions are as follows:

• Is anyone else handling a similar case as me, differently than the methods I outlined?
• Is it possible to pass a sprite reference to a shader, or I guess realistically grab the packed spritesheet and UV somehow from unity and only sample that portion of the sheet? If so, I suppose I could let Unity pack all the sprites together and store a reference to their position through a custom editor or similar for layering them in a shader?
• Is there somehow a way to reduce the number of images I need to store for a frame based animation? IE only the pixels that changed?
• Any further thoughts?
• Is Unity planning on offering some sort of updated solution for frame-based animation to remove the pain points? At this point frame-based animation is in my opinion the most lacking part of the 2D workflow in Unity.

Thank you.

 

Thanks for taking a look at my post! I realize my use case might be beyond the scope of traditional sprite swapping (rendering multiple overlayed sprites) but some sort of CompositeRenderer or something where you pass a component an array of same size sprites and it creates a texture2DArray behind the scenes and draws them all in order, or some way to include this in animations would be really cool.

At the moment though I've continued working on this for myself and think I've reached a solution using the method I described above and a shader to sample my different indices of the array to draw the characters 10 layers all at once.

It means leaving mechanim behind for animating the character however, and rather animating the textures through the shader using the quads UVs.

I would assume what you are targeting is more along the lines of creating a animation as we do now, and providing a way to mark a texture as a variant of another texture and easily swap which texture the animation references?