Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug Unity's static batching: why does it shoot itself in the foot?

Discussion in 'General Graphics' started by Gondophares, Feb 4, 2021.

  1. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    This post was borne out several days of poring over the Frame Debugger and running tests. Consider it both a bug report and a request on Unity's behalf for more insight into what seems a bizarre and counterproductive decision. A minimal example project has been attached for your convenience.

    I work on a complex project that involves relatively large levels with realtime shadows in a forward rendering setup. There's much to be said about how that's not an ideal setup, and we can give just as many explanations, but that's besides the point. Also, this project runs on the built-in render pipeline and while it's true that SRP batching deals with this better, it's still only dealing better with what remains sub-optimal input. The problem described here is unrelated to any rendering API or other technical decisions.

    The observations

    Using the Frame Debugger in two of our levels (one fully realtime, one lightmapped with a shadowmask), we were seeing an inexplicably high batch count, caused largely by "the objects either have different "Receive Shadows" settings, or some objects are within the shadow distance, while some other objects are not." But so, so many. How come? Let's investigate.



    That number, I think you'll agree, is considerably higher than we would expect.



    This batching is bizarre. You'd expect to see some sort of logic to it. There's more, though.





    That's right, the final sorting step is sorting by... instance ID? This value is just some sort of hash/identifier; the value is not meaningful and, worse, it's not even preserved between devices or even after a scene load. You could reload the scene in the editor and the value driving static batching in your scene is scrambled. Behold what happens when we enforce sequential IDs for a second...



    Using static batching now, our batch count is suddenly massive improved:



    What gives?
    As shown in the examples above, the sorting by the renderer's instance ID is particularly devastating to the efficiency of the static batching. In fact it seems to do a pretty good job at minimizing it by scrambling batch division and rendering order. I've debated this with several people and simply could not come up with any scenario in which this would be desirable behavior. I cannot imagine why you'd ever want an optimization feature driven by a functionally random number.

    So the question to Unity is... why? Is this simply a case where the implementer long ago operated under the mistaken understanding that these IDs were somehow sequential (and it's just never been noticed)? Any chance you can address this?

    In case someone might be led to believe this is some hard-to-fix low-level rendering problem; it's not. In fact, you can see it happening right here on line 284 in the C# reference. More so, the virtual method and the fact the sorter is internally passed as an object suggests the original implementer foresaw this problem and the rendering ID is just some placeholder that was designed to be overridden.
     
    ebaender, mh114, Noisecrime and 21 others like this.
  2. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Because I forgot to attach the project on the first post.
     

    Attached Files:

  3. hatless

    hatless

    Joined:
    Dec 15, 2010
    Posts:
    48
    This is a real horror story.
     
  4. joshuacwilde

    joshuacwilde

    Joined:
    Feb 4, 2018
    Posts:
    725
    For us, we just decided to write our own custom batching solution... as many have. It's too bad that unity's static batching is so close to being extremely useful, and yet in many cases it fails to be. With our own static batching we save 1000s of draw calls that would otherwise not be static batched correctly. Also important to keep in mind that it sorts batches by distance to camera. This is more or less worthless (from what I understand) on mobile due to their TBDR nature. So you will probably get better results on mobile in your above demo. This can be tested in editor by turning off opaque sorting on the camera. Of course it helps more in Immediate Mode GPUs, like many desktop GPUs are.

    Also it will sometimes help to make sure that different materials have different RenderQueues, so they can be drawn sequentially, and thus have higher likelihood of getting batched.
     
    Gondophares likes this.
  5. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    These are both good observations. Yes, we too have our own batching solutions up to a point and both of the tricks you mention here are part of that. To not complicate matters further, they weren't included in the minimal example.

    In fact, a lot of the reason we went so deep into this was to investigate why our solution still underperformed. It turns out that was largely due to the fact that under the hood it relied on Unity's batching utility simply for merging the meshes.

    Opaque sorting
    Correct, we are talking about desktop rendering here. We investigated the effect of distance-based vs. no distance-based sorting, but it seems it has relatively little bearing on this issue. That makes sense, of course, because even if the rendering of a single material is broken into a small number of depth buckets vs just a single one, the far greater source of batch breaking is still the inefficient ordering of submeshes. Either because the geometry is split across a bunch of batches with no spatial contiguity or because the submeshes within each batch are ordered without spatial contiguity.

    If we remove distance-based sorting, we can maybe merge some draw calls at the "border" of the different depth buckets, but that's still vastly outweighed by the effects of the skipping across the submeshes wantonly.

    Render queue
    Yup, though in this case we prefer to work with sortingOrder, which accomplishes much the same thing, but without creating a new material instance (or adjusting the source asset, which may be undesirable if it's reused in other contexts). This indeed allows you to group all instances of a single material together in the rendering process, but the gains are similar to the sorting trick above. You basically turn several groups of many inefficient batches into one big group of only slightly fewer inefficient batches.

    The most meaningful bottleneck remains the lack of spatial contiguity in the submeshes, which is pretty much entirely caused by the sorting by instance ID and seems like it should be pretty easy to fix.
     
  6. Neto_Kokku

    Neto_Kokku

    Joined:
    Feb 15, 2018
    Posts:
    1,751
    The answer for all bizarre Unity quirks like this is always the same: because Unity doesn't make games.
     
  7. jamespaterson

    jamespaterson

    Joined:
    Jun 19, 2018
    Posts:
    395
    Thanks for a very detailed post. I'm afraid i can't offer any help or further insight but it is interesting to understand more about this aspect of unitys rendering.
     
  8. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Well, yes, my main motivation in pointing this out is that (in certain scenarios) this seemingly trivial fix might have a bigger impact on improving performance than other much more complicated or low-level optimizations.

    If anyone from Unity is reading this, I imagine it might already help a lot of the sorting could be changed to follow this kind of logic:

    Code (CSharp):
    1.    
    2. public static GameObject[] SortGameObjectsForStaticbatching(GameObject[] gos, StaticBatcherGOSorter sorter)
    3.         {
    4.             gos = gos.OrderBy(x =>
    5.             {
    6.                 Renderer aRenderer = StaticBatcherGOSorter.GetRenderer(x as GameObject);
    7.                 return sorter.GetMaterialId(aRenderer);
    8.             }).ThenBy(y =>
    9.             {
    10.                 Renderer aRenderer = StaticBatcherGOSorter.GetRenderer(y as GameObject);
    11.                 return sorter.GetLightmapIndex(aRenderer);
    12.             }).ThenBy(go =>
    13.             {
    14.                 Renderer aRenderer = StaticBatcherGOSorter.GetRenderer(go as GameObject);
    15.                 return aRenderer.transform.position.x;
    16.             }).ThenBy(go =>
    17.             {
    18.                 Renderer aRenderer = StaticBatcherGOSorter.GetRenderer(go as GameObject);
    19.                 return aRenderer.transform.position.z;
    20.             })
    21.             .ThenBy(go =>
    22.             {
    23.                 Renderer aRenderer = StaticBatcherGOSorter.GetRenderer(go as GameObject);
    24.                 return aRenderer.transform.position.y;
    25.             }) .ToArray();
    26.             return gos;
    27.         }
    28.  
    Note: I'm not saying this is pretty code (it's pretty awful, actually). In fact, that whole utility class feels like it has some weird choices and nomenclature. But it gets the point across.
     
  9. hatless

    hatless

    Joined:
    Dec 15, 2010
    Posts:
    48
  10. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Ugh, wish I had discovered that post before. Did a lot of googling (and headscratching) but didn't come across it.

    Anywho: the analysis made in that thread by lc-gd is correct IMO. Unity's approach to static batching, with separate index buffers (separate submeshes) has theoretical advantages. It's just that the current batching algorithm, with its bizarre reliance on instance ID, completely removes those advantages and the result is considerably less optimal than the more "naive" approach of simply merging everything into a single submesh.
     
  11. hatless

    hatless

    Joined:
    Dec 15, 2010
    Posts:
    48
    It would be nice if unity has a response beyond "It's not broken, it's advanced."
     
    Prodigga and Gondophares like this.
  12. xgonzal2

    xgonzal2

    Joined:
    Jul 3, 2012
    Posts:
    62
    It's a fairly old system so I'm not even sure anybody at Unity has detailed knowledge on it. Would be good to get their take on this though.
     
  13. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    I've wondered if perhaps instance ID in those ancient days was less of a random value. Would explain how this "got broken under the radar". That said, I've used Unity for over a decade and have never known instance ID to be anything else.
     
  14. xgonzal2

    xgonzal2

    Joined:
    Jul 3, 2012
    Posts:
    62
    I'd be surprised if its behavior has changed since original implementation. The ID itself comes from the Unity Object class so as far as I know it doesn't have any way to smartly generate the ID. I think we would need a spatially coherent way to generate it which Object doesn't look like would be able to do.

    On top of that the function you pointed out is marked as virtual so I can definitely believe it was meant to be overridden with something more useful but that's just me speculating.
     
    Gondophares likes this.
  15. joelv

    joelv

    Unity Technologies

    Joined:
    Mar 20, 2015
    Posts:
    203
    Yeah, this is an unfortunate side effect of the current (and yes indeed it is very old) static batching implementation. We are landing some minor improvements to static batching (support for negative scales on static normal mapped geometry for example), but nothing more is road mapped at the moment.

    I agree that adding spatial sorting somehow would be a great thing though.
     
    PutridEx, xgonzal2 and Gondophares like this.
  16. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Thanks for the input!

    Good call - that static normal mapped issue has affected us in the past as well, but nowhere near as much as the issue described here, which negatively affects pretty much all of our levels to some degree.

    If some improvements for the static batching are slated anyway, I'd very strongly advocate at least considering this one too. My arguments:

    • The cause of this issue is very contained. It doesn't require overhauls of complex systems. Its most naive fix requires a change to a single line of C# code.
    • The potential fix is easy to test. Let alone any actual smart spatial mapping; what does it even do if the function simply returns 0 instead? (In my estimation that would not be worse and potentially already better.) After all, the issue is not even so much the lack of spatial logic, but the existence of a randomizing factor. From there, you could implement any kind of spatial sorting - I don't think it even needs to be particularly complex because, again, anything but this would be a massive step forward.
    • The effects of this change are easy to demonstrate. I would imagine the performance effects to range from very small to significant, depending on the scene setup. I can't really imagine any compatibility issues (forwards or backwards) - would any project knowingly rely on the current weird sorting?
    • The current system is just plain wrong. I would say it's only because it's hidden in obscurity that this wasn't considered a bug years ago. There are no arguments in favor of it and many against. It knowingly creates a near-worst case scenario in every single Unity project. That's, frankly, a little embarrassing, as if Unity doesn't know what it's doing.

    Around the time of making this thread I also created a bug report with the same example attached (case
    1311996.) Again, please consider the big ramifications of this issue compared to the workload required to address it. Feel free to get in touch for any feedback if that could help move things along.
     
  17. joelv

    joelv

    Unity Technologies

    Joined:
    Mar 20, 2015
    Posts:
    203

    Thanks for the bug report. Yes I agree this should be changed/improved somehow and we will see what we can do.
     
  18. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Hey. Any news on this?

    Here's a picture just from today to illustrate the issue again. Note how the various paint buckets are broken into a bunch of different draw calls, even though many of them fit inside the same combined mesh. The various submeshes are processed in forced random order, causing the drawing loop to constantly switch contexts (lighting, reflection probes, etc.)

    upload_2021-4-30_14-42-28.png

    I reckon a lot of developers don't encounter this issue, likely because they use external tools (as do we, simply to avoid this). What continues to boggle my mind is that the default implementation, rather than being not great, could hardly be less optimal and could improve with the smallest of changes.

    I keep trying to think of some hidden gotcha where this issue would be more complex than it seems, but I fail to come up with anything.
     
  19. apkdev

    apkdev

    Joined:
    Dec 12, 2015
    Posts:
    277
    So I came across this thread yesterday and thought this would be a fun problem to play around with.

    Here's the proof-of-concept repo: https://github.com/apkd/UnityStaticBatchingSortingPatch

    Basically, this replaces the
    SortGameObjectsForStaticBatching
    method with a modified implementation that uses Hilbert curves to sort the objects in order that's a bit more spatially aware.

    It kinda probably works, maybe. If anyone has a workload that this could be benchmarked on, I'd love to find out if this actually improves performance in any real-life scenarios.

    Note that out of the box, this post-processes your scenes in play mode and during build, but won't work at runtime in builds (eg. if you manually use
    UnityEngine.StaticBatchingUtility
    ). You could probably make it work in Mono builds if you #if-out the editor types, but this "method function pointer patching" technique definitely won't work with IL2CPP.

    To install, just stick the script somewhere in your editor scripts. Keep in mind this requires Burst, Mathematics and Collections packages, but rather unnecessarily so - the sorting seems to run just as fast on Mono.

    Not sure if this is of help to anyone. Maybe a starting point for a better implementation?

    Before:


    After:
     
    funkyCoty, mgear, mitaywalle and 5 others like this.
  20. Sky77

    Sky77

    Joined:
    Jan 30, 2014
    Posts:
    171
    Awesome!
    We have a couple of complex project we can benchmark it on, both in 2019.4 and in 2020.3.
    We’ll try this week and let you know!
     
    PutridEx and apkdev like this.
  21. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Sweet - nice work. (I mean... the way the "patch" works is right out of Satan's playbook, but it's good enough for diagnosing the issue. :p)

    I will be applying it to the above "paint bucket" scenario and report back later.
     
    Prodigga and apkdev like this.
  22. Blepius

    Blepius

    Joined:
    Mar 9, 2021
    Posts:
    68
    Hi all,

    Is this still relevant for the forward renderer in URP 2021.2+?

    I tried to reproduce this, but things seem to be working fine?

    In the two attached images, there are around ~1600 chairs. The non-static version is rendered in ~5500 batches. When set static, they render in 150 batches.

    (Note: don't mind the FPS counts in the images, I was alt-tabbing around.)
     

    Attached Files:

  23. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    I have no reason to suspect it isn't still relevant, but I imagine (and vaguely know) that the SRP Batcher might counteract the effect to some degree. Do you have a minimal example/Unity package? I might be able to compare.

    Even so... I would estimate 150 batches still seems quite high. Your statistics window indicates about 2.3M vertices, which leads me to assume that each chair is a little over 1400 vertices? If a static batch can contain (off the top of my head) 64.000 verts, the theoretical batch count should be about 36. Note how the SetPass calls (often the heavier and more relevant metric) remains at 58. I imagine the SRP Batcher is responsible for keeping it that low.

    Also note that (from the look of it) all of your chairs are well within the shadow distance. What happens when the shadow distance is somewhere before the end of the rows of chairs?

    What does the Frame Debugger look like? Is there a lot of 'jumping back and forth' there?
     
  24. Blepius

    Blepius

    Joined:
    Mar 9, 2021
    Posts:
    68
    Hmm, I've turned off SRP Batching in the URP Asset (which is now a hidden property not mentioned in documentation), and it still seems to work. GPU instancing is also off in the chair material.

    The chairs are casting shadows here, which I think is responsible for a few of those extra batches over the theoretical 36.

    The frame debugger is looking pretty stable.
     

    Attached Files:

  25. mitaywalle

    mitaywalle

    Joined:
    Jul 1, 2013
    Posts:
    247
    Hi, this is great work.
    Earlier I've made similar sorting, but it was not in form of patch, we had our own bake-step, that sorts renderers buy coordinates, then store it in separated lists, and call StaticBatchUtility.Combine() on this lists one by one. It add's a lot of time at runtime, but work same way
     
  26. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    I feel compelled to add, though, that under the current circumstances, rendering order inside your 'separated lists' is still maximally inefficient due to the problem described in this thread. Of course, the problem is mitigated quite a bit since the 'jumping around' is contained with a more local area. In other words, if your manually batched sectors were sufficiently large, it wouldn't make a meaningful difference.

    That's because the issue is inside StaticBatchUtility.Combine(), which is used by both the default static batching and any API with which to make homebrew solutions. (Short of manually merging meshes at a lower level, still retaining submeshes for culling, and keeping track of all possible batch breaking conditions.)



    Having worked with URP some, I can confirm that the issue is not (or less obviously) present there, but I haven't done systematic research into it so far. In other words, I'm not sure if that is due to SRP batching or some inherent property of the SRP rendering process (as Catsploration suggests above).

    Even so, until the built-in render pipeline is actually deprecated, it doesn't feel right to let this weirdo issue stand. I, and plenty others, work on projects that cannot easily make the switch to SRP (or want to). Not long ago, another dev I know asked for input on a bizarrely high drawcall count ("It's all statically batched right?"). Top-down cityscape with lots of very low-poly houses. After staring at the frame debugger for a while, I started with "This might surprise you..."
     
  27. richardkettlewell

    richardkettlewell

    Unity Technologies

    Joined:
    Sep 9, 2015
    Posts:
    2,281
    Hi all.

    (I had to quote that second post too because it amused me)

    Anyway, I have some news to share about this issue. Simply put - we've fixed it in Unity 2023. The first version with the fix is 2023.1.0a5. Credit to @alextyrer for the fix!

    We've gotten rid of the Instance ID sorting, because, well, it was a real horror story :) We've replaced it with a spatial sort, which uses the positions of the MeshFilters instead as the input.

    In one example, we saw a render of 271 batches reduce to 55 batches.

    Thanks for the feedback on this, and the detailed explanation!
     
    lclemens, saskenergy, mh114 and 21 others like this.
  28. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Well... Let me just say... Thank you.

    While, we're at it... Any more info about the spatial sort? Is the size at all configurable? Not that it matters that much, but it might just be the cherry on top.

    For context: I am on vacation today and my colleague still pinged me to instantly alert me about this. :p
     
    Last edited: Sep 10, 2022
    richardkettlewell likes this.
  29. SuperPoopsy

    SuperPoopsy

    Joined:
    Jan 20, 2020
    Posts:
    17
    @richardkettlewell That's good news indeed!
    Any chance we might see the fix backported to more production-safe releases?
    (2020/2021LTS/2022.1)
     
  30. Zapan15

    Zapan15

    Joined:
    Apr 11, 2011
    Posts:
    186
    Please backport this for all LTS versions if possible.
     
    KYL3R, my_little_kafka, KAJed and 5 others like this.
  31. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    719
    hey, great idea. just wanted to chime in and say the 'burst' version isn't faster probably because it's not actually running a burst compiled method. Putting the BurstCompile attribute on a static method isn't enough, you have to put that attribute on a Job, or reference the method from within a Job, to get the benefits of using Burst. Nonetheless, thanks for posting this.
     
  32. Ricquert

    Ricquert

    Joined:
    Aug 7, 2017
    Posts:
    2
    I was literally typing up a thread about an issue similar to this when I somehow wandered off into another tab and into this thread. Thanks Gondophares and apkdev for the research and I hope this can be backported to LTS releases.

    Two short questions:
    Does this work for non-static batched objects too? I often see draw calls of a single GPU instanced object -> the same object with the SHADOWS_SCREEN keyword enabled -> single GPU instanced object -> ... -> ad nauseum. Same with static batching obviously. It's really rough on CPU-bound machines like for example a certain popular console.

    Is this defined per camera and if so can we override the sorting system with our own?
     
    Last edited: Oct 4, 2022
  33. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    I've never looked into how instanced rendering is grouped together, but I'd be surprised if it somehow passes through this same API. (Unless there's another independent implementation of this same behavior. :p)

    Just shooting some ideas... Do the objects maybe have LOD levels, one with shadows and another without, and the cutoff distance happens to land somewhere in a big group? I feel like I've maybe seen this pattern before, but I can't recall ever finding something unexplainable.

    Doesn't help that the frame debugger specifies why a new draw call was started (if sometimes very obscurely), but can't say how the order was established in the first place. What I sometimes find helpful is offsetting a material's render queue one from default. This tends to "group" the calls more (often reducing a few), making it easier to diagnose if there's other weirdness going on.
     
  34. Ricquert

    Ricquert

    Joined:
    Aug 7, 2017
    Posts:
    2
    For me it was a combination of the original issue (batcher being trashed by static objects with dynamic shadows) and instanced rendering having a similar issue. It didn't end up being LODs. All of my LODs have been properly marked with the correct Receive shadows setting. Statically batched and GPU instanced objects near the border of the shadow distance just seem to interrupt each other's draw calls.

    I decided to "work around" this and rewrote most of my shaders to use the deferred pipeline. I ended up halving my SetPass calls (from ~300 to ~150).
     
    FM-Productions likes this.
  35. Sky77

    Sky77

    Joined:
    Jan 30, 2014
    Posts:
    171
    It would be really great if the fix will be backported to at least 2021 & 2022 LTS.
    If you're in production right now it's pretty much impossible to use 2023 unless you're targeting a 2025 release, and this fix would greatly benefits a lots of projects.
     
    koirat and Gondophares like this.
  36. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,051
    Well this is a fascinating thread. I’d done some work with static batching in the past, but never had a large scene that would have exhibited these issues. I was more concerned with wanting to tailor or control the index buffers, or simply draw the batches myself, which appears impossible as they are never exposed via api.

    Anyway I’m glad to see the original problem has been addressed and like others I’d love to see this back ported to earlier versions.

    if not I wonder if the improved code is native or part of Unity C#? If the latter I wonder if apkdev’s method above to overwrite the sorting method could be used with the new C# code from 2023 version? It would be cool as then you’d get the same batching in a project using an older version of unity.

    if the improved code is native, I wonder if Unity might be willing to release it or perhaps provide a C# port so we could use it in older projects. Again it’s just about getting consistency between unity versions assuming Unity are un able to back port it.
     
    Last edited: Oct 9, 2022
    Gondophares likes this.
  37. richardkettlewell

    richardkettlewell

    Unity Technologies

    Joined:
    Sep 9, 2015
    Posts:
    2,281
    Thanks for all the feedback about backporting. I've had a chat with my colleagues, but we aren't going to do it.

    The reason being that we also made some large code changes for other optimizations, and backporting this fix would need all the other code taking back with it too, due to the order in which the code changes were made. This is too much code and adds too much risk of regressions. We also discussed just recreating this change in the LTS's, but diverging our release lines like this also adds additional risk + maintenance costs if there are undiscovered bugs lurking. Plus it's a change in behaviour, which, even though we think 100% of use cases would be happier with the new way, we can't know that for sure.

    So there's all our excuses...! :)
     
  38. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,051
    Any thoughts to my request to provide example code or detailed breakdown of the new static batching code so we could implement it ourselves? Of course this is assuming that's possible using the technique highlighted above. I mean the main issue seems to stem from a build time process of data and so wouldn't require any runtime changes making the hooking via reflection less risky than for other situations.

    The main reason I'd like this is not simply that we might gain some performance ( though that would be nice ), but so that as a project upgrades naturally through Unity versions the static batching build process remains consistent.

    For example the majority of my current projects are still on 2019.4, a few have branched into 2021 or 2022 and I was pushing to start new projects with 2022, but then recently several patch versions broke UI rendering multiple times and so I've had to put that push on pause for now. So essentially I'm usually lagging 2-3 years behind Unity versions, but some projects will need to be upgraded over time, for which having improved static batching would be useful.
     
    FM-Productions likes this.
  39. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    719
    Could you at least backport the "one line change" of removing the random factor from the sorting? Surely that single change can't be too tied up in all the changes around the engine.
     
    FM-Productions and Noisecrime like this.
  40. Sky77

    Sky77

    Joined:
    Jan 30, 2014
    Posts:
    171
    Don't get me wrong, this is totally understandable, but I think it exposes a major problem with the current Unity update cycles: some important fixes like this one takes years before a game dev studio can actually benefit from them.

    We're currently working on a fairly big console / PC production that pushes the engine hard, with a target Q2 2024 release date. This means that we'll probably freeze the engine version at 2022LTS, because jumping to Unity 2023 would pose a huge risk, expecially considering that URP tends to break hard between major releases, so we need to be extremely careful if we plan to update the engine version...

    Maybe for smaller indie production this is not a problem, but on a bigger / more complex game with longer production cycle you're pretty much forced to stay on 1 - 1.5 years older versions of Unity.
    We may consider 2023LTS for a 2025 release game, not 2024...

    Again, don't get me wrong, I really appreciate all the effort and we're huge Unity fans, but it's frustrating when we can't benefit from major fixes because they're not backported. :)
     
    Gondophares likes this.
  41. richardkettlewell

    richardkettlewell

    Unity Technologies

    Joined:
    Sep 9, 2015
    Posts:
    2,281
    This is a key reason why we are _not_ doing it :p changing the static batching build process in a patch version is more disruptive than in a major version.

    We just sort the batches along each axis, preferring the XZ plane:

    Code (CSharp):
    1.                // simple "spatial" sort on axes (prefer x/z plane)
    2.                auto lPos = lhs.localToWorld.GetPosition();
    3.                auto rPos = rhs.localToWorld.GetPosition();
    4.                if (lPos.x != rPos.x)
    5.                    return lPos.x < rPos.x;
    6.                if (lPos.z != rPos.z)
    7.                    return lPos.z < rPos.z;
    8.                if (lPos.y != rPos.y)
    9.                    return lPos.y < rPos.y;
    "but then recently, someone changed all the static batching code in a 2020 LTS patch and there were a couple of new bugs in it, and now i can't ship my app!" ;-)

    While it's true in that changes can take years to trickle out as users migrate to newer versions, a reason we aren't backporting it is because we don't see it as an important fix. An important fix fixes a bug, and it's even more important if it's a regression bug. (something that used to work). We see this as an optimization. It's not unheard of for us to backport an optimization, but the risk factor needs to be much lower, and the benefit much higher.

    Until we get better at making this more seamless for you, if we start backporting stuff like this, then updating patch versions will start to become harder too.

    I hope this explains our position. I totally get that it's frustrating when you just want the thing in the version you are using.
     
  42. Flow-Fire-Games

    Flow-Fire-Games

    Joined:
    Jun 11, 2015
    Posts:
    305
    We thought so too, until for example you figure out that for xbox DX12 is now required and only non-LTS versions currently have roughly the same rendering performance.
    Just a warning that the freezing strategy is very likely not going to work out for you.
    Currently also LTS has similar amounts of bugs than tech stream in our project, actually mostly the same bugs due to bugs being released in packages during the LTS cycle.
     
  43. Sky77

    Sky77

    Joined:
    Jan 30, 2014
    Posts:
    171
    Don't get me wrong, this is totally understandable and it makes a lot of sense. :)

    I just wanted to point out that IMHO on longer / bigger productions, the current yearly Unity update cycle can be quite difficult to work with or plan for. There's a lot of fragmentation, expecially if you're using SRPs or new packages. Then throw in a couple of vital Asset Store things that maybe are not updated on time + the various console SDKs and upgrading between versions can be a total nightmare.
    I'm sure that if your production cycle is short you won't feel any difference, but on bigger projects... it stings. :)

    To be fair, it's not better in other engines: things tends to break badly between minor releases of certain another engine too. ;)
     
    richardkettlewell likes this.
  44. Sky77

    Sky77

    Joined:
    Jan 30, 2014
    Posts:
    171
    I think this is very tricky depending on the project.
    We tend to stay on the latest version possible and we've been pretty update happy in the past, taking some considerable risks. But now if we update too soon we risk to hinder production with a very unstable version of the engine (Unity Alpha + untested packages + Asset Store stuff not updated + console SDKs and so on...), if we switch too late we risk an upgrade nightmare.

    Again: it probably makes sense to be super flexible on a small production, but not for a bigger scope one, expecially if you tend to push the engine and use a lot of fancy stuff.
     
  45. Flow-Fire-Games

    Flow-Fire-Games

    Joined:
    Jun 11, 2015
    Posts:
    305
    For Unity standards we are a pretty big production and we are 2+ years in.
    We have learned over the last two years that freezing a version is just not viable.
    I totally agree with the sentiment of wanting to lock in a viable version, but over and over we are seeing that we have to update.
    Unity is basically in a constant stream of changing things with the result that no release is actually suffeciently "complete" for us that we can actually leave it alone and it sounds like you are going to sit in the same boat in 1-2years.
    From our perspective it is great to hear when a dev is not backporting potentially breaking changes into the current stream.
    We have enough dependencies into Unity that in 21 LTS no release in our project really works. 2020 is not nearly as performance efficient as we need it and the console cert requirements are moving away from us.
    HDRP introduced bugs, UI backported bugs, currently Input system and Polybrush have to be downgraded to not wreck the project...
     
    Blepius likes this.
  46. Sky77

    Sky77

    Joined:
    Jan 30, 2014
    Posts:
    171
    I totally agree that not backporting stuff that can potentially wreak havoc in previous version of the engine is the way to go.
    On the ‘no freeze policy’: it really depends. If it works for you: good!
    We try to stay pretty nimble and -maybe- we’ll try to be on the latest version of the engine possible, but for now for the sake of the production we’re aiming to ship on 2022.

    We’re on a 40 people project right now, that must sim ship on all consoles and PC (Switch included), which means that there are a lot of variables involved.
    Just jumping to 2023 4 months before going gold is not a decision to be taken lightly… ;)
     
  47. Flow-Fire-Games

    Flow-Fire-Games

    Joined:
    Jun 11, 2015
    Posts:
    305
    It doesn't really work for anybody, it's just the least bad option.
    But because of all the backporting most LTS updates tend to add more issues meaning when you want to go for your release you won't have any version that is sufficiently issues free.

    Your original point of the Unity release model not working for productions is obviously correct.
    The solution is to not have LTS releases with tons of backporting for years but to deliver proper releases when it actually makes sense instead and exclusively fix issues in these releases afterwards.
     
    Sky77 likes this.
  48. tatoforever

    tatoforever

    Joined:
    Apr 16, 2009
    Posts:
    4,364
    Currently, no static batching is way faster than static batching in our project. It give us 10% faster CPU time on Nintendo Switch hardware. I guess working to batch those meshes is heavier than just bruteforcing all drawcalls...
     
  49. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,631
    Yeah, static batching hasn't worked right since the 4.x days. At some point in early 5.x / 2017 they changed something without giving a F*** and it became unreliable, but I guess changing it to making sense has higher standards now.

    You would think with how often we hear the excuse of "we won't backport this because it might break things" that LTS releases would be really stable, and yet...
     
    funkyCoty and tatoforever like this.
  50. Gondophares

    Gondophares

    Joined:
    Mar 9, 2013
    Posts:
    28
    Note, with static batching there's no real "batching" work going on at runtime. All batched meshes are generated at build time. So apart from the inefficient grouping of batched meshes described in this thread (and all its consequences), static batching shouldn't have any real overhead.

    Even so, I agree that the effect is minimal unless the amount of draw calls is significantly reduced. But that depends almost entirely on level design. If your game uses a thousand unique materials, it won't make a difference, if everything has the same material, it can do an awful lot.
     
    richardkettlewell likes this.