Feedback Core product interoperability and stability

From the Unity blog post "The road to 2021":

https://blogs.unity3d.com/2020/08/13/the-road-to-2021/


I'm monitoring the public issue tracker since 2018. In your "The road to 2021" post you wrote key focus of Unity 2021 is to drive down bugs. However, according to the public issue tracker, bugs are not going down since November 2020.

What's the reason for this? Below you can find a screenshot of my statistics, which is showing the number of active bugs.


https://docs.google.com/spreadsheets/d/1jOju0jhGrWa21ZtOKgZPy0DZo7vVTfiACvIvE1AUEfQ/edit#gid=0

You can see bugs didn't go down since November 2020. Bugs did go down between August 2020 to November 2020, which I guess was for the Unity 2020.2 release, but not the Unity 2021 release which as focus on fixing bugs.

 

Our graph god is back at it again keeping unity true and honest, praise be

 

Thank you both @cirocontinisio and @QA-for-life for the insights. It's really interesting.

Do you think you could make a blog post recapping the process and showing some more statistics? Maybe when the 2021 cycle gets to LTS, as a sort of conclusion of the post Peter mentioned at the beginning

 

The quality of Unity has definitely gone up for us. We still see an uptick of bugs when a new major feature comes out (New Prefab System, Asset Pipeline 2, etc.), but it goes down pretty fast. Also you would see way fewer issues if you stay on LTS, which we're using for now. I think a lot of people use the tech or even beta versions and forget that these are supposed to be less stable; I'm definitely guilty of that.

 

As you say, its very hard to judge, and a total count is not a good indication of individual experiences.

We still shy away from any new version for the first several patches, because they are riddled with issues. Luckily QA is really great, when you report issues they respond promptly and are incredibly knowledgable.
Its worryisome that several very core elements break, and what Unity finds acceptable to ship or call 'production ready' differs from what 'the community' seems to think. Scriptable render pipelines come to mind, but also for example when the new prefab workflow was released it absolutely was in an unworkable state for the first few releases.
This makes me very anxious about any new feature, and especially if there doesnt seems to be a way back. I want stability, not a different UI, it irks me that i read in the blogposts "we're listening to the users" marketingspeak but i, in these specific cases, dont see that reflected. Yes the developer working on it is incredibly responsive, and thats really great, but the existing workflows and stability (seem to) be brushed aside too hastily in my opinion.

Of course there is also the issue that Unity is a platform on which we build our own software. When we or a plugin author do it wrong, Unity is blamed. In our own project i suspect this is the main reason why we find assetdatabase V2 to be overwhelmingly worse than what was before, with random imports happening incredibly frequently, and assets simply not having reflecting what is on disk. The random nature of it makes it impossible to report a bug, let alone fix it, but we're not alone.

That being said, it has been less painful to upgrade every release. The amount of absolutely blocking issues has been reduced, eventhough the feature set we use has been increased. We now have to wait for 10 patches in order to use a new version, instead of 15.

 

"Not reproducible", "by design" and "duplicate" are distinct states that aren't counted towards the "active" ones which @Peter77 is tracking.

What are you talking about?

 

Based on browsing the forums daily:
overall feeling is increased frustration with all kind of issues & bugs in pretty much every aspect of Unity.. (especially anything after unity 2019, latest versions just keeps getting slower/sluggish even on highend computers and latest GPU's)

personally still would prefer 2017/2018 over any other version (due to fast iteration/enter play mode and UX).

 

Been using SRP and the new input system, and trying out all of the new stuff that Unity provides and no bugs or issues from my side (guess I dont look at it funny to cause it eh). My only complaint is the long domain reload times, the compilation speed is great and all but the domain reload between each save is a bit annoying, which is why I've been following the .NET thread in the other forum category, it's not necessarily a QA thing though since theres nothing much to bug fix, its just the nature of how Unity needs to handle compiling while playing.

I've had a few silent crashes, which were mostly my fault since they're related to multithreading with jobs and I had some edge cases unhandled. I wish I could get logs for these and something to go off of instead of guessing but its not always possible with silent crashes. Otherwise, I don't really feel frustrated with Unity, everything works as expected including all of the new work. I honestly feel like I'm in a parallel universe with how buggy Unity is apparently supposed to be according to the forums and other people (groupthink?).

So has it improved? No not really, I never had issues prior to 2019, or 2018 or whatever the magic Unity version is to begin with, didn't have any in 2020 and not having any in the 2021 releases either. My only reference point is the few release cycles within the alpha channel that have a bug due to human error and then get fixed 2 cycles later (which is expected, its alpha anyways, not too concerned about this).

 

Hey Thomas, thank you for the reply!

That's what I see in my version of the graph too. It's the drop between August 2020 to November 2020.

I don't understand why I'm not seeing the drop from November 2020 to today. 18% is significant enough to be visible in the graph or it should be. While it's possible only non-public issues were fixed during this time, I think it's unlikely. Do you have any idea why it's not reflected in my graph?

If you're asking for Unity 2020+ specifically, I don't know, I'm still using Unity 2019.

If you're asking in general, then I can absolutely say my experience with Unity has improved. When I compare the problems (report here) from Unity 4 & 5 with Unity 2018/2019, it's a completely new level of stability. I think it's related to Unity stabilizing and me working on smaller and less ambitious projects now, that don't try so hard on Unity.

I know that you know, but I'm not trying to put everyone's work at Unity Technologies in a bad spot. That's not my intend with this post. In fact, I think many teams doing a better job than a few years ago. At this point I'm trying to make sense of the numbers I'm seeing in the issue tracker and the statement that was given in the blog, because they don't seem to match.

 

This matches our experience in 2020.3 too. The biggest problems right now are Editor performance issues and crashes -- which is way better problems to have than Player performance problems and crashes -- but it's still not optimal.

 

It's almost a year when I last posted the stats. Below you can find an updated version:


https://docs.google.com/spreadsheets/d/1jOju0jhGrWa21ZtOKgZPy0DZo7vVTfiACvIvE1AUEfQ/edit#gid=0

The spike in November 2021 is correct, I checked it multiple times. What's the reason for it?

 

Thank you for the reply. In this case I'll stop tracking the public issue tracker. One thing less to care about