Feedback The process for submitting bugs is too cumbersome for us to justify submitting many bugs we find.

The current process that requires an example project in order to submit bugs is causing many bugs to go unreported and this leads to multiple community members needing to spend time identifying and working around the same bugs.

Unity Goals
1) I would assume your goal is to get feedback from the community on how to make the Unity better
2) I'd assume you do not want to spend engineering time on non-issues and user errors.

My Goals
1) solve issues that effect my teams products or workflows
2) to evaluate the feasibility of using new pre-release features for my company
3) be a good community member that contributes back to the community.

The problem
Creating test projects / or uploading ours is time consuming and problematic. We have tons of proprietary work flows and even stripping a project out of our custom dev environment is hugely problematic

Our Reaction
We only submit the issue if it is a showstopper or we have extra non-product critical time to contribute to the community. If a bug isn't a show stopper for us we often don't submit it as it will be closed no-repro without a test project even if we provide code examples that demonstrate how to make it happen. Even if we can tell you how to make it happen by copying our code into one of your own demos.

Effect on the community
The community members will spend time finding, identifying, and working around the same issues we found previously.

Example
Case 1419941 is a great example of an issue that is in pre-release. The issue was created in an attempt to be a "good community member". The code sample and explanation I provided should be enough to evaluate the issue. This is an issue that multiple users have reported:
https://forum.unity.com/threads/you...-the-onunloadeventcompleted-callback.1267208/

NOTE: in this case the multiplayer team has a separate process to evaluate things on "git" and this was resolved in that way. However this would not have been resolved through the normal process and as it is not effecting one of our products and we can not justify spending the time required to create an example project to help you repro the issue. In fact I didn't know this would be evaluated through the git process and have failed to get this looked at with unity's official process without attaching an example project.

Suggestion
Evaluate the likelihood of an issues and have a team who has the technical chops to evaluate the problems without always putting it on your community to do the work of creating an example project for you. I understand you probably get countless bogus issues. That you often cannot repro an issue without a test project. The current process however just leaves us not submitting bugs.

When simple code samples are provided with a clear description of an issue that should be enough Case 1419941 is a great example of something unity should be able to evaluate without the community creating an example product for them.

Creating example projects wouldn't be an issue if we were finding a bug a month, but sadly the cadence is much higher and whenever we do a major upgrade (version 2021-2022) we find multiple bugs daily. We simply don't have the time.

I'd very much like to see unity move to a single official process that was like the multiplayer teams response in git where the issues are evaluated without example projects.

My Experience with Unity:
My team spends a huge amount of time working around bugs within unity. Our last cross platform project shipped on four different versions of Unity. Different versions for each platform as each version had different bugs we could not work around. We completely stopped sending in bugs as we neared ship, we would have needed a dedicated head to create all the test projects.


My Background:
1) I am a studio head that manages a internal game studio and two external game studios.
2) I have a background in publishing/firefighting where my job was to learn and evaluate tools, pipelines, and processes. I still spend time personally evaluating new Unity features for my studios.
3) I have used Unity to ship AAA product
4) I have shipped product with unity on every major platform.

Hopefully these thoughts help improve the process.
Thanks.

 

My suspicion is actually that the cumbersome nature of bug submission is somewhat by design.

Unity is in a weird place where they're providing software tools for developers but the majority of their user base is most likely still not professional or experienced w software dev. They probably need some degree of barrier to bug reporting to prevent being buried by amateur programmers with syntax errors.

I recently went through some bug submission stuff with jetbrainz and it was a really exposed process that included direct communication w their staff. The bug got fixed, fast. It was super impressive. But I don't think you could do the same kind of thing w unity's userbase.

 

Yes. I'd suggest they have a different process for people with pro licensees vs. those with free. I think that would filter a huge portion of the less experienced community.

 

They do have *some* things differently for paying customers, but just because you cough up some money they won't be able to magically reproduce your problem if you don't show them how, usually.

 

This is basically how bugs have been logged at all companies I have worked at for the last 30+ years. If someone wants a bug to be properly investigated they need to provide something that allows it to be replicated quickly by the developer(s).

Otherwise what can happen is that the developers spend loads of time building projects to test the issue only to find that it works fine for them - simply because their way of building stuff does not fit the edge-case situation that the client is seeing.

Some places where I have worked will simply reject submitted bugs if there is no supplied project or replication steps to follow as without these it is just a waste of time for the developers.

 

This is the correct attitude by the way. But only the real showstoppers.

If you can work around the bug by, say, dropping a Unity feature completely and implementing your own solution, then it's not a show stopper.

 

I didn't say they we wouldn't show them how to repro the issue and example code, I said it wasn't worth our time to create a test project.

 

I cannot tell if you are being sarcastic.

 

I did not suggest we wouldn't provide anything. If you look at this thread as an example we provided a code sample and a repro. This wasn't enough to submit the issue through official channels:
https://forum.unity.com/threads/you...-the-onunloadeventcompleted-callback.1267208/

Any developer worth his salt could look at that code sample and do a quick evaluation. (and a unity developer did with the git process). While the multiplayer team has a unique process allowing users to submit bugs without repro projects through git and do investigate issues like I posted this process is not shared across all of unity.

 

Yeah, the multiplayer team doesn't have their package in production. The usage of their packages is limited since they are not production ready. Hence the multiplayer team shouldn't used as the baseline what's viable and what's not.

 

Yep, I don't understand how a dev can't see why unity needs a repro project. That's the sort of thing that's required by any programmer. It should be self explanatory.

Otherwise it's basically asking a doctor to diagnose your problems without ever seeing you.

If they only had you and not thousands of others to look at. If your company is doing well selling that many titles, then Unity has dedicated paid support to speed things up.

 

They didn't need a repro project if you read my entire post. They looked at the code sample I provided and identified the issue. I'm in complete agreement that sometimes a repro project makes sense. other times it does not. I don't think a one time fits all rules is ever viable.

We have paid for unity support both when I worked at Microsoft on HoloLens and Recore. At my current company we use unity as a visual editor tool rather then a engine we don't need support as we don't use the runtime.

My main point here is that when I evaluate the engine (because we are considering using the runtime) and find issues the community isn't benefiting from my findings. That was also true when I ran an indie company as we just didn't have time.

 

I totally get it and I can imagine. I'm sure you waste countless hours when the problem is on the user side. Keep in mind what I'm talking about isn't issues we care about a response for. If we wanted a response we'd create you a test project. I'm talking about all the issues we find that cost us time to identify, but where the workaround takes us less time then uploading a test project. Then we just don't bother and thus there are probably countless people hitting the same issues.

For what its worth I'd be much more apt to share test projects if we didn't have a custom dev environment. The problem is you don't have our development environment, and that is so hard to get you setup with. So we have to break out the code into a separate project, and that can take quite some time.

Thanks for listening I hope my feedback is received as someone with the best intentions who is trying to help.

 

Agreed that most issues need a test project. In the end I would just love a way to "link" issues we find to features in a way that made it easier for the community to benefit (even if an engineer didn't evaluate). Something like Unity's issue tracker but linked directly to the documentation

 

Having done projects with both, I've not found this to be the case unless it has changed at some point.

Edit: I misinterpreted that. It's a suggestion of how it could be, as opposed to how it is.

 

Many times I don't, and Unity still gets back to me on at least some of them. I read a staffer somewhere saying that they'd rather get useful info than nothing, even if we can't attach a minimal repro, so thats what I've gone by. Sometimes I attach a video showing what breaks and when, and of course a clear written report along with the logs.

Sometimes the response has been to ask for a project, and that at least cuts down my effort to the cases which Unity is interested in looking at. In those cases I sometimes make a minimal, separate repro, and other times just send them my tens-of-gigs project with a list of repro steps.

So my main thought is that even if you don't have time to hit Unity's ideal bug report level, if something is better than nothing and you've got quality data, send it anyway. The team does seem to pay attention.

I can understand why they don't advertise that and would probably even discourage it in general, of course.

 

I am not.

It's often a lot less effort to re-implement a Unity feature yourself (and frankly, you'll probably do a better job than Unity) than to try and make Unity fix the bugs for it in a timeframe that falls within your game's cycle (and I'm saying this, someone who is working on the same game for almost a decade).

So only attempt it for real show stoppers where there is absolutely nothing you can do about.

 

Our time preparing a project with a minimal test case with super clear instructions is also rarely fruitful, because a bug fix rarely comes in time for it to be useful, if at all, so we are at an impasse.

And from my point of view, Unity bugs are your problem, not ours. So you waste your time.

 

As we have to eat the impact of the bugs on our work, they're very much our problem.