Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  3. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Feedback 2D System Optimizations

Discussion in '2D Experimental Preview' started by Ferazel, Jan 8, 2020.

  1. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    I downloaded the Lost Crypt 2D demo and it looks really cool. Thank you! I'm glad that you're working on an integrated spine-esque animation system. Also that you've implemented a more robust toolchain for SpriteShape, Tilesets, and 2D lighting. All of these things are looking great, and will no doubt increase the visual fidelity of 2D projects. However, there is an aspects of the 2D system that is seemingly being ignored with these new features.

    PERFORMANCE

    The primary being fundamentally performance. 2D games are typically fill rate limited and the Lost Crypt demo project is full of problems in this regard. You need to recognize that fill rate is not free. Most consoles and mobile devices are fill-rate limited in the 2D space, yet there have been very few optimizations to help with this fact.

    One of the tools that people have used in the past SpriteSharp (https://assetstore.unity.com/packages/tools/sprite-management/spritesharp-mesh-optimizer-37599). The problem is that it appears that SpriteSharp's dev has disappeared (as is the problem with most 3rd-party assets) and Unity doesn't have anything comparable to the feature set that we could utilize.

    Some things I'd like to see focused on ASAP in the 2D toolchain:
    1. Tight mesh optimization. This has been a problem since the 2D stuff was introduced in Unity 4.2 and I don't think it's been improved since its release. This is a great tool to cut down on fill rate by increasing the vert count on sprites. The problem is that the mesh generation that happens automatically is extremely inefficient. Sometimes missing the most ideal mesh by hundreds of verts/polys. Sure throwing a few hundred verts extra into the tight mesh here or there may not sound like a big deal, but it adds up quickly. Please optimize your tight mesh algorithm to be more like the SpriteSharp's. There is no reason why I don't see why this couldn't be done.
    2. Opaque rendering. As you know, everything that is transparent is blended and it hurts fill rate tremendously having huge overlapping sprites that have multiple foreground and backgrounds of overlapping sprites. There is a solution to this, and it is using the depth buffer to quickly clip pixels that have a pixel in a foreground layer. I recognize with 2D lighting this is tricky, but we need to be able to do an opaque pass on all of the rendering layers in 2D so that we can spend GPU time on the things that actually need transparent blending.
      1. Sorry to keep harping on this, but SpriteSharp again has tools to assist with this. The biggest one is to allow us to split the sprite into opaque and transparent sprite meshes. With this being built-into Unity's Sprites that would be great, but even if we had to use 2 sprite renderers that is fine too.
      2. Add opaque support to the SpriteLit master node in shader graph in addition to doing an opaque pass on the renderer to depth test pixels.
      3. Allow sprite shapes and tilemaps to take advantage of opaque rendering to improve performance on the fill sprite.
      4. Add support for the renderer to render opaque sprites first to cut down on the transparent blending.
    3. Improved SkinnedSprite performance. This has already been brought up in the 2D animation forum, but I want to stress it here that the current performance is horrible. Even in the lost crypt demo with Burst installed, I'm seeing the sprite shape consume a huge amount of CPU time per frame by a factor 50% of the CPU frame time with only the character being the only sprite skin!
    4. GPU Instancing for Tilemaps. With tight meshes being used, obviously this creates a problem with GPU instancing as an option, but that doesn't mean that tilemaps can't benefit greatly from the feature and yet I'm not seeing support for this?
    Anyways, I do really appreciate everyone's hard work on the 2D system. I think the feature set is great, but I struggle to really be willing to use the features without some massive considerations to performance of the entire 2D toolchain. I want to stress again that Unity is trying desperately to get performance by default, but the 2D toolchain doesn't seem to reflect that.

    Are these performance improvements being considered? Is there time on the roadmap to implement performance improvements? Is there anything that the community can do to try to bump up the priority of this work?
     
    NotaNaN, Aratow, oakus and 5 others like this.
  2. rustum

    rustum

    Unity Technologies

    Joined:
    Feb 14, 2015
    Posts:
    190
    Hi @Ferazel! Thank you for your detailed feedback. Allow me to reassure you that all of these are being considered, specifically items 1, 2 and 3. Item 4 isn't something we're actively working on, but we'll put it on our list of ideas with potential. The suite of 2D tools in Unity has been focused on functionality previously and we are now turning our attention to improved performance and better workflows!
     
    oakus and Ferazel like this.
  3. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    It is great to hear that the team can focus on improvements now that many of the core feature set has shipped. I think there are a ton of opportunities to improve the workflows for 2D including manual mesh generation (when the auto-gen isn't quite optimal), fast/efficient runtime atlasing, and the other optimizations mentioned above.

    Thanks!
     
  4. xiao-xxl

    xiao-xxl

    Joined:
    Nov 16, 2018
    Posts:
    48
    Very disappointed!

    For features that are needed but currently missing, the reply is always:
    ”The most important thing right now is to improve performance.“

    However, more than a year has passed, there is no obvious improvement in performance, no road map, no guarantee of development progress, and it is only under consideration forever.
     
    EvOne likes this.
  5. xiao-xxl

    xiao-xxl

    Joined:
    Nov 16, 2018
    Posts:
    48
    The priority is completely wrong.
    Performance is the most important thing, not something like The Transparent Sorting Mode.
    For performance, the most important thing is to optimize the fill rate, otherwise there is no way to use it on mobile platforms.

    1. Tight mesh optimization.
    2. Opaque rendering.
    3. 2d Lighting batch.
     
    april_4_short likes this.
  6. kennyy_

    kennyy_

    Unity Technologies

    Joined:
    Apr 7, 2021
    Posts:
    96
    Hi @xiao-xxl

    Items 1 and 2 are at the top of our optimization priority. Item 3 is already available in Unity 2021.1 or later, where layers containing the same set of lights are batched together. While we cannot guarantee fixed dates, we can assure you the aforementioned requests are not being overlooked. We are focusing our efforts on usability/extensibility and performance improvements.
     
    Lo-renzo likes this.
  7. april_4_short

    april_4_short

    Joined:
    Jul 19, 2021
    Posts:
    489
    From LWRP's promises of being light weight, to now, there's been a lot of consideration of the needs of mobile and light weight gaming.

    Yet this has rarely, if ever, turned into any meaningful point of difference for UnityNEW™ in mobile game rendering.

    Does anyone else see a pattern of promises and claims of focused considerations not bearing fruit within Unity?

    Would it be possible for Unity to take a quick last look at what can be done to optimise Builtin for mobile before it becomes frozen in time?
     
  8. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    Since this thread was resurrected I’m happy there was a response on improving performance of the 2D renderer for some of these issues.

    While this has some focus wanted to request better tight atlas efficiency. There are many instances where atlases will not pack efficiently. I have submitted bugs for this and it was supposedly “by design” as it was supposed to reduce enter play mode delays with packing the atlas. However, with the V2 atlas being an asset generated in the Library I feel this should not be a an issue and it should more efficiently pack atlases. Spine has a much better atlas efficiency/logic than the default Unity atlas. This should not be the case, atlases should be pre-computed and pack as efficiently as possible to reduce texture atlas RAM size (another huge perf problem for 2D games).
     
    NotaNaN and april_4_short like this.
  9. Lo-renzo

    Lo-renzo

    Joined:
    Apr 8, 2018
    Posts:
    1,520
    Behold the holy grail. I'm not buying that since I can't do it at runtime (a req for me) but if Unity can get to 1/2 that, that would be amazing.
     
    EvOne likes this.
  10. kennyy_

    kennyy_

    Unity Technologies

    Joined:
    Apr 7, 2021
    Posts:
    96
    Hi @Ferazel,

    Thanks for providing more insight. Do you have links to the bugs you mentioned, so we can have a clearer picture of what you're discussing. Are the bugs reported part of V1 or V2 atlas? You mentioned Spine as a better alternative because of its efficiency. Can you clarify where the performance gains are in terms of your workflow? If you have samples or use cases to demonstrate this, it would be really helpful to us.
     
  11. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    Hi @kennyy_ case I submitted was case 1254179 and was done under the V1 atlas. I just tested the behavior under the V2 atlas in 2021.1.15 and the problem still exists. If you add/remove the incense sprite to the atlas you will see that the atlas will pop from 2048x1024 to 2048x2048 even though there is still space on the atlas next to the chair.

    Spine's meshed atlasing is much better about finding space to fit the sprite and packs much more efficiently. Spine atlas also supports divisible by 4 atlases (not targeting mobile) allowing us to export more tight atlases at a smaller size also reduce memory/load times. Also spine's atlas system seems much better about rotating meshes to find the best rotation while Unity's seems to support 90 degree rotations and even then it seems very spotty at best about its packing efficiency.

    My understanding is that V2 atlas is still only V1 atlas logic but generates an asset database artifact in the library so that it can be used with the V2 asset database. However, if we're doing that work I fully expect the atlas generated to be as efficient as possible and not taking shortcuts in order to generate an atlas faster. If the atlas is pre-computed it should be as efficient as possible since it should only need to be done when changes are made to the sprite list. The hope is that you can work on getting a better atlasing system for 2D games as it helps everyone by not having to be so stringent with cropping/positioning while allowing us to cut out the alpha out of the source images for meshed sprites.
     
    Last edited: Aug 4, 2021
  12. april_4_short

    april_4_short

    Joined:
    Jul 19, 2021
    Posts:
    489
    Why don't Unity buy Spine?

    It's somewhat unique, very good, well loved, and would save Unity the embarrassment of spending 3 years x 3 or 4 programmers + management and public relations, documentation and integration energies.

    back of mental napkin calculations... they save more a million - could offer this to Spine owner/creator - and get the goodwill of supporting a product that works on other platforms.

    Then they can use Spine's meshed atlasing for all their 2D and 3D texture packing needs, too, and we'd get the clever optimisation that is meshed sprites the way Spine does it.
     
  13. EvOne

    EvOne

    Joined:
    Jan 29, 2016
    Posts:
    173
    Spine in the beginning, at the start, only with the KickStarter gathered more.
    At the cost of $300 per one copy (continues to be sold successfully), and the "de facto" - "industry standard"
    - who will sell it?
    Also so cheap that consider for nothing ...

    (google translate, sorry)
     
  14. april_4_short

    april_4_short

    Joined:
    Jul 19, 2021
    Posts:
    489
    You think it's worth more than a million USD, well after the peak of 2D mobile gaming's drive of its desirability?
     
  15. EvOne

    EvOne

    Joined:
    Jan 29, 2016
    Posts:
    173
    Yes i do. Absolutely.
    "Fashion" comes and leaves. But 2DGames were born not yesterday and already survived so much that this "past peak" - they will not notice :D

    And about the Spine - offer Adobe to sell their Photoshop or Illustrator for a million - I am wildly laughing... :p
    I will explain - these programs are about the same "standards of industry", as well as Spine now.

    Despite the fact that there is (yes!) there are other programs and in something similar, in something better...
    But - these are "Standards"
     
    Last edited: Aug 5, 2021
    NotaNaN and april_4_short like this.
  16. april_4_short

    april_4_short

    Joined:
    Jul 19, 2021
    Posts:
    489
    I have no idea about its adoption level and current user base, nor what market saturation looks like for this kind of (to me) quite esoteric and niche product.

    So I'll take your word for it.

    I just don't think it's anywhere near the 30 million registered users that Photoshop has on a subscription model ;)
     
    EvOne likes this.
  17. kennyy_

    kennyy_

    Unity Technologies

    Joined:
    Apr 7, 2021
    Posts:
    96
    @Ferazel Thanks for the great feedback. I spoke with the team and here's the response.

    I am assuming that it only packs Tight-meshes generated exclusively within Spine and not general purpose.
    And as I guessed it uses a variant of LSCM method. For more info Google : LSCM method uv packing

    This is basically reducing Size which we can already do using Variants.

    Rotating anything other than 90 or multiples would cause visual artifacts. Since Spine always generates Tight-mesh and uses for Animation its hardly noticeable. But for things like Pixel-Perfect or Quality of Sprites this just wouldn't work.

    Having said that, we still need to improve packing efficiency compared to SpriteUV. Do you have an example of a Spine atlas using arbitrary rotation(non-90) for packing in your project?
     
  18. april_4_short

    april_4_short

    Joined:
    Jul 19, 2021
    Posts:
    489
    Have someone from the UV world of texture packing for 3D consider what you're doing with sprite packing.

    These are solved problems, from a very long time ago.
     
  19. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    Hi @kennyy_ see my responses below.

    OK maybe for Pixel games you might run into some problems interpolating the UV I've never worked on one. Also, I'm not sure what you mean by "Quality of Sprites" I've never heard that term. Regardless, I'm not making a pixel perfect game, I'm using texture filtering on my game (and I'd argue a lot of games using your engine including your 2D demos) and yet I'm burdened by this decision? This is does not produce optimal atlases and should be an option that is default OFF unless you're using point filtering on the atlas texture. For non-pixel games an atlas is a texture that is packed tightly using the mesh shapes and has accompanying meshes that have UV coordinates assigned to that texture for its vertices. You're burdening every project with sub-optimal packing in order to support a subsection of your userbase (who can turn off rotation on the atlas if needed).

    No, that's not correct. The atlas system requires that a master atlas be a power of 2. Then you can uniformly scale down the atlas from that with a variant. What I'm asking for is to generate a texture that is divisible by four so that it can still be compressed by DXT5/BC7 compression for console/desktops without wasting the space of a power of 2 texture would may add a lot of extra alpha to fill. We group our atlases by props by room and by usage which doesn't always align well with power of 2 sizes. However, a divisible by four packer will make the most efficient atlas possible (which is what spine does). See below (blacked out to protect the innocent) this is an atlas that doesn't mip so we don't need power of two. We can compress it with a divisible by four size and the packer picks the appropriate size as needed but still divisible by four so that it can be compressed by GPU texture formats.
    upload_2021-8-5_23-18-18.png
     
    MousePods, NotaNaN and april_4_short like this.
  20. Venkify

    Venkify

    Unity Technologies

    Joined:
    Apr 7, 2015
    Posts:
    648
    Thanks for the feedback.

    SpriteAtlas is designed to scale and work well for all use-cases (styles, sizes etc..) and hence general purpose. While its true that packing Tight-mesh sprites with all degree of rotation does work for some use-cases, it may result in artifacts in Pixel-Perfect games, deforming high-def sprites etc.. Besides for very large projects, build-time is critical too.

    Sprites are generally categorized either based on Types (Characters/Props/Backgrounds/FX etc..) or based on (Scenes/Levels) or specific use-cases such as UI/Gameplay etc.. and then added to Atlas. What we are looking above is for a very specific set of Sprites that can be packed optimally similar to the usage of 2D PSD Importer.

    This is correct. We start with the minimum POT texture and increase the height or width by next power of 2 when there is not enough space. This is to ensure consistency of packing across different texture formats (as the requirements of compressed formats could range from Multiples of N to Power of 2 etc..).

    We will explore improving 1) current packer for tight-packing and 2) investigate NPOT support. Will keep this thread posted. Thanks again for the feedback.
     
    Last edited: Aug 11, 2021
    NotaNaN, Ferazel and MousePods like this.
  21. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    Thank you @Venkify, I appreciate you looking into improving the atlas efficiency/algorithm.

    Another thing I forgot to mention is that I'm not sure why I can't mix tight mesh sprites with rect sprites in the same atlas. Why is this an all or nothing checkbox on the atlas (tight packing)? I should be able to pack the tight sprites and rect sprites on the same atlas. So that I can still have some sprites used for slicing. Obviously, the rect sprites wouldn't tightly pack within its rect bounds, but that to me is expected behavior. Thanks!
     
  22. Venkify

    Venkify

    Unity Technologies

    Joined:
    Apr 7, 2015
    Posts:
    648
    You can add FullRect and Tight Sprites in the same Atlas.
    If Tight-Packing is enabled, it uses Tight-mesh for applicable Sprites and uses rect for others. Please let us know if this is what you are referring to. A quick demo below :
     

    Attached Files:

    Ferazel likes this.
  23. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    518
    Ahh yes you are right, I mistakenly attributed this to the atlas packer but it was indeed the import settings for the sprite. Thanks for the correction.
     
  24. castor76

    castor76

    Joined:
    Dec 5, 2011
    Posts:
    2,517
    I have already mentioned this somewhere, but in order to produce better atlas packing, you have to have the best tight mesh possible in the first place if you are using tight mesh packing.

    So when you talk about tight mesh generation here is my feedback.

    1. Like it has been mentioned here, we need best packing algorithm improvement.

    2. We need to improve sprite editor so that custom tight mesh generation can happen across all the sprites in the sprite sheet. Currently, you have to manually select each slice and the change option to 1 or so, and then apply. Then, again and again. Which is not fun and not very practical. We need button to apply all to the all slices at once. And we need api opened up so we can make better use for it in the editor script.

    3. This is not sprite atlas related, but since I have mentioned sprite editor here is another one. We need to be able to slice using pixel unit for the pivot while doing slicing! We can do this for individual sprite, so make no sense to not have the same option available while slicing.

    4. Make so that the all of the above can be applied to all of the sprites selected in the project. If not, expose script api so that we can create our own sprite batching process tool. Asset post processor is a good idea, but I feel like it kinda forces you to have no options per individual sprite needs, so it would be best if we can just select some assets in the project and the use sprite editor api to just do some batch processing.
     
    Ferazel likes this.