Feedback 2D System Optimizations

I downloaded the Lost Crypt 2D demo and it looks really cool. Thank you! I'm glad that you're working on an integrated spine-esque animation system. Also that you've implemented a more robust toolchain for SpriteShape, Tilesets, and 2D lighting. All of these things are looking great, and will no doubt increase the visual fidelity of 2D projects. However, there is an aspects of the 2D system that is seemingly being ignored with these new features.

PERFORMANCE

The primary being fundamentally performance. 2D games are typically fill rate limited and the Lost Crypt demo project is full of problems in this regard. You need to recognize that fill rate is not free. Most consoles and mobile devices are fill-rate limited in the 2D space, yet there have been very few optimizations to help with this fact.

One of the tools that people have used in the past SpriteSharp (https://assetstore.unity.com/packages/tools/sprite-management/spritesharp-mesh-optimizer-37599). The problem is that it appears that SpriteSharp's dev has disappeared (as is the problem with most 3rd-party assets) and Unity doesn't have anything comparable to the feature set that we could utilize.

Some things I'd like to see focused on ASAP in the 2D toolchain:

1. Tight mesh optimization. This has been a problem since the 2D stuff was introduced in Unity 4.2 and I don't think it's been improved since its release. This is a great tool to cut down on fill rate by increasing the vert count on sprites. The problem is that the mesh generation that happens automatically is extremely inefficient. Sometimes missing the most ideal mesh by hundreds of verts/polys. Sure throwing a few hundred verts extra into the tight mesh here or there may not sound like a big deal, but it adds up quickly. Please optimize your tight mesh algorithm to be more like the SpriteSharp's. There is no reason why I don't see why this couldn't be done.
2. Opaque rendering. As you know, everything that is transparent is blended and it hurts fill rate tremendously having huge overlapping sprites that have multiple foreground and backgrounds of overlapping sprites. There is a solution to this, and it is using the depth buffer to quickly clip pixels that have a pixel in a foreground layer. I recognize with 2D lighting this is tricky, but we need to be able to do an opaque pass on all of the rendering layers in 2D so that we can spend GPU time on the things that actually need transparent blending.
   1. Sorry to keep harping on this, but SpriteSharp again has tools to assist with this. The biggest one is to allow us to split the sprite into opaque and transparent sprite meshes. With this being built-into Unity's Sprites that would be great, but even if we had to use 2 sprite renderers that is fine too.
   2. Add opaque support to the SpriteLit master node in shader graph in addition to doing an opaque pass on the renderer to depth test pixels.
   3. Allow sprite shapes and tilemaps to take advantage of opaque rendering to improve performance on the fill sprite.
      
   4. Add support for the renderer to render opaque sprites first to cut down on the transparent blending.
3. Improved SkinnedSprite performance. This has already been brought up in the 2D animation forum, but I want to stress it here that the current performance is horrible. Even in the lost crypt demo with Burst installed, I'm seeing the sprite shape consume a huge amount of CPU time per frame by a factor 50% of the CPU frame time with only the character being the only sprite skin!
4. GPU Instancing for Tilemaps. With tight meshes being used, obviously this creates a problem with GPU instancing as an option, but that doesn't mean that tilemaps can't benefit greatly from the feature and yet I'm not seeing support for this?

Anyways, I do really appreciate everyone's hard work on the 2D system. I think the feature set is great, but I struggle to really be willing to use the features without some massive considerations to performance of the entire 2D toolchain. I want to stress again that Unity is trying desperately to get performance by default, but the 2D toolchain doesn't seem to reflect that.

Are these performance improvements being considered? Is there time on the roadmap to implement performance improvements? Is there anything that the community can do to try to bump up the priority of this work?

 

It is great to hear that the team can focus on improvements now that many of the core feature set has shipped. I think there are a ton of opportunities to improve the workflows for 2D including manual mesh generation (when the auto-gen isn't quite optimal), fast/efficient runtime atlasing, and the other optimizations mentioned above.

Thanks!

 

Very disappointed!

For features that are needed but currently missing, the reply is always:
”The most important thing right now is to improve performance.“

However, more than a year has passed, there is no obvious improvement in performance, no road map, no guarantee of development progress, and it is only under consideration forever.

 

The priority is completely wrong.
Performance is the most important thing, not something like The Transparent Sorting Mode.
For performance, the most important thing is to optimize the fill rate, otherwise there is no way to use it on mobile platforms.

1. Tight mesh optimization.
2. Opaque rendering.
3. 2d Lighting batch.

 

From LWRP's promises of being light weight, to now, there's been a lot of consideration of the needs of mobile and light weight gaming.

Yet this has rarely, if ever, turned into any meaningful point of difference for UnityNEW™ in mobile game rendering.

Does anyone else see a pattern of promises and claims of focused considerations not bearing fruit within Unity?

Would it be possible for Unity to take a quick last look at what can be done to optimise Builtin for mobile before it becomes frozen in time?

 

Since this thread was resurrected I’m happy there was a response on improving performance of the 2D renderer for some of these issues.

While this has some focus wanted to request better tight atlas efficiency. There are many instances where atlases will not pack efficiently. I have submitted bugs for this and it was supposedly “by design” as it was supposed to reduce enter play mode delays with packing the atlas. However, with the V2 atlas being an asset generated in the Library I feel this should not be a an issue and it should more efficiently pack atlases. Spine has a much better atlas efficiency/logic than the default Unity atlas. This should not be the case, atlases should be pre-computed and pack as efficiently as possible to reduce texture atlas RAM size (another huge perf problem for 2D games).

 

Behold the holy grail. I'm not buying that since I can't do it at runtime (a req for me) but if Unity can get to 1/2 that, that would be amazing.

 

Hi @kennyy_ case I submitted was case 1254179 and was done under the V1 atlas. I just tested the behavior under the V2 atlas in 2021.1.15 and the problem still exists. If you add/remove the incense sprite to the atlas you will see that the atlas will pop from 2048x1024 to 2048x2048 even though there is still space on the atlas next to the chair.

Spine's meshed atlasing is much better about finding space to fit the sprite and packs much more efficiently. Spine atlas also supports divisible by 4 atlases (not targeting mobile) allowing us to export more tight atlases at a smaller size also reduce memory/load times. Also spine's atlas system seems much better about rotating meshes to find the best rotation while Unity's seems to support 90 degree rotations and even then it seems very spotty at best about its packing efficiency.

My understanding is that V2 atlas is still only V1 atlas logic but generates an asset database artifact in the library so that it can be used with the V2 asset database. However, if we're doing that work I fully expect the atlas generated to be as efficient as possible and not taking shortcuts in order to generate an atlas faster. If the atlas is pre-computed it should be as efficient as possible since it should only need to be done when changes are made to the sprite list. The hope is that you can work on getting a better atlasing system for 2D games as it helps everyone by not having to be so stringent with cropping/positioning while allowing us to cut out the alpha out of the source images for meshed sprites.

 

Why don't Unity buy Spine?

It's somewhat unique, very good, well loved, and would save Unity the embarrassment of spending 3 years x 3 or 4 programmers + management and public relations, documentation and integration energies.

back of mental napkin calculations... they save more a million - could offer this to Spine owner/creator - and get the goodwill of supporting a product that works on other platforms.

Then they can use Spine's meshed atlasing for all their 2D and 3D texture packing needs, too, and we'd get the clever optimisation that is meshed sprites the way Spine does it.

 

Spine in the beginning, at the start, only with the KickStarter gathered more.
At the cost of $300 per one copy (continues to be sold successfully), and the "de facto" - "industry standard"
- who will sell it?
Also so cheap that consider for nothing ...

(google translate, sorry)

 

You think it's worth more than a million USD, well after the peak of 2D mobile gaming's drive of its desirability?

 

Yes i do. Absolutely.
"Fashion" comes and leaves. But 2DGames were born not yesterday and already survived so much that this "past peak" - they will not notice

And about the Spine - offer Adobe to sell their Photoshop or Illustrator for a million - I am wildly laughing...
I will explain - these programs are about the same "standards of industry", as well as Spine now.

Despite the fact that there is (yes!) there are other programs and in something similar, in something better...
But - these are "Standards"

 

I have no idea about its adoption level and current user base, nor what market saturation looks like for this kind of (to me) quite esoteric and niche product.

So I'll take your word for it.

I just don't think it's anywhere near the 30 million registered users that Photoshop has on a subscription model

 

Have someone from the UV world of texture packing for 3D consider what you're doing with sprite packing.

These are solved problems, from a very long time ago.

 

Hi @kennyy_ see my responses below.

OK maybe for Pixel games you might run into some problems interpolating the UV I've never worked on one. Also, I'm not sure what you mean by "Quality of Sprites" I've never heard that term. Regardless, I'm not making a pixel perfect game, I'm using texture filtering on my game (and I'd argue a lot of games using your engine including your 2D demos) and yet I'm burdened by this decision? This is does not produce optimal atlases and should be an option that is default OFF unless you're using point filtering on the atlas texture. For non-pixel games an atlas is a texture that is packed tightly using the mesh shapes and has accompanying meshes that have UV coordinates assigned to that texture for its vertices. You're burdening every project with sub-optimal packing in order to support a subsection of your userbase (who can turn off rotation on the atlas if needed).

No, that's not correct. The atlas system requires that a master atlas be a power of 2. Then you can uniformly scale down the atlas from that with a variant. What I'm asking for is to generate a texture that is divisible by four so that it can still be compressed by DXT5/BC7 compression for console/desktops without wasting the space of a power of 2 texture would may add a lot of extra alpha to fill. We group our atlases by props by room and by usage which doesn't always align well with power of 2 sizes. However, a divisible by four packer will make the most efficient atlas possible (which is what spine does). See below (blacked out to protect the innocent) this is an atlas that doesn't mip so we don't need power of two. We can compress it with a divisible by four size and the packer picks the appropriate size as needed but still divisible by four so that it can be compressed by GPU texture formats.

 

Thank you @Venkify, I appreciate you looking into improving the atlas efficiency/algorithm.

Another thing I forgot to mention is that I'm not sure why I can't mix tight mesh sprites with rect sprites in the same atlas. Why is this an all or nothing checkbox on the atlas (tight packing)? I should be able to pack the tight sprites and rect sprites on the same atlas. So that I can still have some sprites used for slicing. Obviously, the rect sprites wouldn't tightly pack within its rect bounds, but that to me is expected behavior. Thanks!

 

Ahh yes you are right, I mistakenly attributed this to the atlas packer but it was indeed the import settings for the sprite. Thanks for the correction.

 

I have already mentioned this somewhere, but in order to produce better atlas packing, you have to have the best tight mesh possible in the first place if you are using tight mesh packing.

So when you talk about tight mesh generation here is my feedback.

1. Like it has been mentioned here, we need best packing algorithm improvement.

2. We need to improve sprite editor so that custom tight mesh generation can happen across all the sprites in the sprite sheet. Currently, you have to manually select each slice and the change option to 1 or so, and then apply. Then, again and again. Which is not fun and not very practical. We need button to apply all to the all slices at once. And we need api opened up so we can make better use for it in the editor script.

3. This is not sprite atlas related, but since I have mentioned sprite editor here is another one. We need to be able to slice using pixel unit for the pivot while doing slicing! We can do this for individual sprite, so make no sense to not have the same option available while slicing.

4. Make so that the all of the above can be applied to all of the sprites selected in the project. If not, expose script api so that we can create our own sprite batching process tool. Asset post processor is a good idea, but I feel like it kinda forces you to have no options per individual sprite needs, so it would be best if we can just select some assets in the project and the use sprite editor api to just do some batch processing.