Unity number of bugs constantly growing

For the past 35 weeks, I've tracked the "page count" of Unity's public issue tracker. This gives an estimate how many known issues exist, how many they add and fix over time, etc.

• Vertical axis shows the issue tracker page count for that specific "state". Multiply by 10 that get the number of bugs. (because 10 issues are shown per page)
  
• Horizontal axis shows the week. I captured the numbers every Monday evening.

Fixed Issues (link)

1863-1650 = 213 pages = about 2130 bugs fixed in 35 weeks.


Active Issues (link)

In March and May they moved a lot of issues from active to "won't fix".


All Issues (link)


Here are the remaining states


Just looking at the active issue count, we have today more issues than 35 weeks ago. Even though they constantly fix a ton of bugs, according to the fixed graph:

Are they ever able to catch up with the bugs?

 

That seems to be about what I would expect. So much new stuff going in recently, plus just the general approach they have now of release early and often.

If they ever catch up with all the bugs, it means we will have a really stable engine, that's years behind the tech a lot of competitive games are using. Most competitive 3D games live on the bleeding edge in at least some areas. In that case it's better for your engine to be working in the same paradigm more or less. You don't need perfect, what you need far more is to not have to do major engineering that your engine should be doing. But because you are a year or more ahead of them on the technology curve, you have to do yourself.

Which is where we were for a good number of genre's 2-3 years back.

 

Yeah, but it's fluctuating up and down. In the context of the whole dataset, taking two random point samples and comparing them to one another tells you... nothing.

Looking at the active issues graph I can't confidently say which direction a trend line would be pointing in. You must already have this in a spreadsheet, so it'd take you all of a couple of minutes to do some more significant analysis on that.

That aside, the number of features in Unity is also increasing, so I'd expect the number of bugs to increase alongside that.

 

This, pretty much

 

Why Unity get more and more bugs? It's really frustrating.

 

Because Unity has and is trying to move beyond being just good enough for mobile games. For that matter they're trying to move beyond being just a game engine. Quite successfully if I'm not mistaken.

Yes, I can understand why it would be frustrating for a beginner, but as someone who has been at it for years now the only frustrating part for me is that it took them this long to get to this point (though I am occasionally annoyed when they make a massive change but don't announce it well or don't have a good replacement ready like with the removal of Enlighten).

Unity's bugs haven't been anything more than a mild annoyance for me for a long time. Easily fixed and forgotten about.

 

Read the thread.

I'd wager a great majority don't affect you.

 

I suggest Unity make a different name for the product beyond a game engine.
HD pipeline can't be compatible with most assets, so give it an another name, independent with Unity.

 

You can choose not to use it, you know?
https://blogs.unity3d.com/wp-content/uploads/2018/09/Unity_Hub_GIF.gif

 

I've been talking about Unitys regression problem for a long time now. Glad more devs see it

 

It's probably fairer to only compare the bugs affecting year.4 releases rather than total bugs at a point in time. The .1 to .3 introduce new features (and new bugs), and .4 is the final stable release for that years version and is where I'd expect the least amount of active bugs.

I do have some concerns about whether bug fixes get backported to previous versions as often as they should. I know Linux isn't official on 2018, but there is not one single release of 2018 that runs on AMD Phenom II in Linux unless you delete the resonance audio so file.

5.x and 2017.x all at least start successfully, so that's a pretty major regression where a known fix was never applied to 2018.4 (the fix in this case was known before release).

I hope 2019.4 will mean the new asset store stuff and things like Tiny Mode get 2019/Linux versions. With 2019.3 being the only full official non beta Linux release, it's disappointing that so much is still officially unusable. I hope Unity start treating lack of Linux support (working on 2019.3+) as a release blocking bug for their assets, or it's still premature to consider Linux a fully supported platform despite it's (mostly) steady improvement.

 

Why? What would that achieve?

 

No. Unity does not seem to have a serious path for polishing new features enough to get them ready for production. For example, DirectX12 and Vulkan support are both pretty broken. They are permanently experimental. If they cannot even get DX12 and Vulkan production ready, then there is no reason to think they will suddenly get all of the other issues polished.

What they need to do is create an internal company policy that a few core features will get polished to production ready status each month no matter what. They need to start with DX12 and Vulkan.

 

True, but that is part of the problem. Unity assumes everybody can simply use old features or work around major problems, so many of the new features are permanently experimental. The new features never become production ready. DX12, Vulkan, 64bit scene files, Gfx Jobs, etc.

When a dev complains about these incomplete features, Unity staff will ask for a bug report. But when they receive a bug report, the problems remain in the software. DirectX12 is widely used throughout the gaming industry, but it is still to this day considered experimental (not production ready) in Unity. Same with Vulkan.

 

And gfx jobs :/ I understand they need to add new features to stay relevant, but if they worked a bit more agile and test driven both new features and a stable code base could be achieved

Edit: saw that you mentioned gfx jobs.

 

Thanks, I will take a look at this next weekend. In the meantime, I added another graph to the first post. I think it makes it a bit easier to see trends, having the other states in the graph as well. Maybe not necessarily for "Active", but for "Won't fix" etc.

 

btw, gfx bugs even have wont fix labels on them, thats discouraging

https://issuetracker.unity3d.com/is...g-vulkan-and-openvr-and-graphics-jobs-enabled

 

Yep, totally true! Still interesting imo.

Cool, thanks to much for that insight!

 

Age of bugs is another good one to track. I like to increase priority on bugs with age so none get stuck under new bugs for too long.

 

"Catching up" with bugs can't happen because feature development won't stop, and feature development is a significant source of bugs. Many get caught and fixed before the release ships, some get caught early but get postponed until a later release for a variety of reasons, some are unfortunately caught by customers. It is just a fact of software development.

The Unity LTS versions specifically don't include new feature development for this reason, which makes them the closest to "catching up" as you're going to see. If this is a concern for you, you shouldn't be using any non-LTS release.

 

I wouldn't put documentation issues in the same category as typos or cosmetic stuff. They can be just as bad as functionality issues despite often being quite easy to fix... the for the right person.

Honestly, they should be treated as high value targets rather than lumped in with cosmetic issues.

Who's ever spent hours trying to figure out what's going on somewhere due to unclear or inaccurate documentation?

 

Sooo... after three years of collecting data, I have a bit more to show.

They had a good run reducing bugs while working on Unity 2020.2. But now it's stagnating again.

Document can be found at https://docs.google.com/spreadsheets/d/1jOju0jhGrWa21ZtOKgZPy0DZo7vVTfiACvIvE1AUEfQ/