Unity 5.3 Performance issues

True, but the updates happens way too fast. Update every half a year would be fine.

 

No kidding, top feedback item, a Linux editor, went unimplemented for years.

 

But that's how it was before (sorta, more random).

 

There nothing wrong with how they update things now -> we all keep speculating things. We need Unity to reply. I just think its failed testing of releases.. If they need more time to test things then so be it, but let everyone know..

 

I like the idea of weekly patch updates which fix bugs as soon as possible. It's just that these new patch releases and minor updates often introduce new bugs and performance issues so they kind of miss the point. Weekly releases are too fast for the usual QA done on full releases so maybe better automated testing is needed.

 

This is my biggest issue. It wasn't that long ago that 5.2 came out, and I'm sure everyone remembers the swarm of bugs that infested that first release and the second... and the third. Right now Unity's patch system basically serves as a deterrent for becoming an early adopter of software (and people should be deterred from that in general). You'll see countless forum warriors running into the "5.x Issues" threads saying things like "What.... you actually patched? Why didn't you wait like the rest of us! NEVER patch the day it comes out!"

On one hand it is something you learn the more you develop games, but on the other hand there isn't any sort of disclaimer to new users other than "always backup your project when upgrading." What if the bug I find doesn't happen until a week after I've upgraded patched? Do I scrub all my work this week and go backward? That might be okay for a hobbyist, but when you have deadlines you're actually held to, that kind of crap just doesn't cut it.

I've hit kind of a nice place right now where I kind of have the flow of this worked out (at least I think so, because I haven't had problems in months), but I can definitely feel the pain of those experiencing patchers remorse.

Personally, I can handle bugs and glitches and CTDs and mistakes (within reason). The thing that really pisses me off though, is when the inevitable time comes for a Unity employee to chime in (No holly, I'm not saying everyone at unity) to a thread discussing issues with a particular patch version and they take that time to try and ensure everyone that these issues either don't exist, or aren't important. It isn't damning to admit when a mistake has been made or that issues exist.

My favorite one was in a 5.2 thread a few months back where a Unity Dev came in and in one sentence said "Actually, I think 5.2 has been our most stable release. Not sure what the problem is." Well... that's easily the most helpful post ever.

Okay so I still had a little bitterness in me. I've vented. Forum warriors, do your worst!

 

FYI Hollie is actually a sith as she advocates letting the hate flow... be warned young padawans.

 

How about they do some manual testing? Huh? create some test environments?

Physics, rendering , platform, animation, etc..

I was able to downgrade to 5.1 and not really miss anything, so I'm wondering what did they do this year?

 

Check out the release notes:

https://unity3d.com/unity/whats-new/unity-5.3.1

And change the URL version number for other releases:

https://unity3d.com/unity/whats-new/unity-5.2

How about 5.1?

https://unity3d.com/unity/whats-new/unity-5.1

 

Thanks a lot for pointing this out! I didn't know about this. Alright, I have to admit, this is much better then.
But anyway, I don't think stability should be discussed on par with any features. As already mentioned by someone here, what can I use a fancy car for if its engine is missing? Well, most certainly not for driving. If you ask me, some sort of stability should be granted at any given major release, no matter what features might be added. Of course, new features may not be fleshed out completely, but with stuff breaking that worked before, you really can't count on it.
Anyway, if that's the system we have to go through to let them know the ship is leaking, I will be glad to rearrange my votes to a request that aims for better testing.

 

I will say this. I am not experiencing this issue probably because it is within the range of my normal screw ups.

But that fact that you as a community manager are not trusted to provide feedback to the developers is a very bad sign. They should do their job and trust you to do yours.

 

Unfortunately that's fairly normal developer behavior. It's been that way in every company I've worked at in the last 20 years, and Iv'e been guilty of it myself. Yes they want data to prove there is a problem I get that, but that doesn't mean they have the right to that attitude. More often then not if there is smoke there is fire. I've seen far more times where this is the case then when people were just imagining things.

The bottom line is that it's not my job to do your testing for you. I don't have the time to put together an isolated test case for every bug I find. In fact as a developer myself I think that's just plain lazy when developers demand that. If you are any good you can take a detailed description and go duplicate it yourself, why should I take my time and do that for you? If it's a bug that is seriously impacting my work maybe, but all those other semi annoying bugs forget it, my time is valuable and this is a product you make money from, you need to the do the legwork yourself.

 

Then how do you know it is an actual bug in Unity, and not bad code on your side? Have you looked at some of the bug reports, even some posted by folks in this thread? Many are not unity bugs, just bad code. Part of troubleshooting is isolating the cause, the quickest way to do that is make a new project and remove parts till you find the cause. If that ends up being a real bug, submit that project. That will help them solve a bug affecting you much quicker. If you have the time to complain about it on the forums, you have to create a useful bug report.

And the reverse can be said as well. It isn't unity's job to fix everyone's project. If it is a legitimate bug, sure. Again, many of the bugs reports aren't real bugs. (They posted numbers a while ago, and you can just look and see that many are vague and just bad code). If they treated all bug reports equally, nothing would ever get fixed. If you have found, and can reproduce a legitimate bug, do a proper bug report to get it it fixed, not just for you, but for everyone. Otherwise, non-specific complaining is useless.

 

I disagree entirely. I think Unity should actually test their software before expecting everyone else to do it for free.

 

I'm usually pretty sure I know it's a bug before I submit a bug report. Damn sure. Maybe it's pride, I hate being wrong. And in cases where I need the bug fixed AND think there is a good chance they will actually fix it, as in it's a priority, sure in that case I'll submit a report with a test case.

I'd give a bit more slack if I didn't know they don't actually test their stuff very well. Why should I pay for their QA?

But back to my main objection, which is a habit developers have of going into their corners and demanding full test cases even in situations where something is obviously wrong. It is their job to proactively explore some of this stuff and test better.

The fact is that there is a completely different standard for products that are not developer tools. You don't demand players to submit test cases, you look at the numbers and if there are enough complaints, you assume something *is* wrong and start tracking it down. The reason companies that make developer tools don't do that, is because they know they can get their customers to do their job for them. I'm ok with some of that because to a point it kind of works. But when developers for a company start to take the attitude that it is actually their customer's responsibility to track down bugs, sorry but that's out of line.

 

So Unity is not testing at all. I didn't know that...

 

I was just trying to dig it up because I remember reading a post made by someone at Unity recently where they released some numbers on how many tests they have for some part of their codebase they were talking about. I can't remember the numbers but I do remember thinking wow, they actually think that's good.

You can also look at the stuff they have released source for, pathetic testing. Even their own test tools don't have good test coverage. I tried using their test tools and was left with the strong impression that they were not made by someone that actually does test driven development. The workflow was really time consuming, not a very productive setup.

And the number of regressions that happen for stuff that would be caught by even minimal testing, like api behaviors changing.

The thing is for testing to really work it has to be something integral to your dev process. It's a way of programming that takes practice and becoming proficient at it before it really pays off. It takes time to learn how to write good tests. What I've seen is it's kind of all or nothing. You don't get half the benefit by going halfway, it doesn't work like that. So if you are just kind of testing here and there it's pretty close to not having anything.

 

If you have a look at e.g. Android, it is anything but easy to even get a half decent test coverage. There is a huge amount of features in Unity. Testing all those with various build settings using different Android versions and on the most relevant devices is a tremendous task.
Unity just made a huge step by reimplementing the OpenGL part in their code. It takes a lot of time to find out e.g. which performance tests are relevant and where a tighter test coverage is needed. Having a complete coverage for functionality and performance from the very beginning for a complicated task like this is not realistic. It would take years to get even close to that.

 

Do you work for UNITY QA.. no you do not... So why keep replying as if you know what they do..
How many times do we need to tell you... Its maybe better you contact the Unity Staff and tell them to explain it all.

 

Getting a good test coverage for things like that is very difficult and even more difficult for a complex piece of software like Unity that is present on so many platforms. It shouldn't be necessary that someone from Unity has to say that. Everyone with a basic understanding of automated testing understands the difficulties in such a task. When it comes to Unity, there are even some blog posts that explain how some of their automated testing works.

You are allowed to express you opinion about the problems in Unity. But I am allowed to make that too. There a quite some new Unity users who get the impression that Unity 5.x is completely unusable because of this kind of thread. Though, a lot of people are using it on a daily basis. I don't think it is unreasonable to chime in from time to time.

 

I don't understand this unless you are talking about going back and adding in tests for stuff that doesn't have it, and ya that's not always cost effective. I was talking about unit testing not performance testing, and in the context of test driven development. Writing tests after the fact is not test driven development, test driven development done right often gets the job done faster then not using it. And once you become proficient at it you don't really have to think about what to test it's just second nature.

And the benefits grow the more complex the systems are you are working with. Complex systems is one major reason why TDD exists. It's not any more difficult to write tests for a large system vs a small one. You still write one function at a time regardless of how big the codebase is.

 

There are many different development processes and it is not necessary to use test driven development to get a good test coverage. What's important is that there are automated tests. Obviously, there are some areas in Unity that suffer and are more prone to introduce regressions. Everyone can see that and even more the Unity developers. They are aware of those problems more than we are and since they are not stupid, I am absolutely sure they are working on that.
How they solve the problems and how they adjust their processes is up to them. It require a decent understanding of the existing processes to find out what it takes to improve them. Without that knowledge, it is impossible to say what they should change, besides having more tests. But as I already wrote, I am sure they are aware of it.
If you have a look at the Unity blog, there are several articles about automated testing at Unity and they also state that they are continuously improving that.

 

Do.... do you work for Unity? Because you've made claims over and over and over that they don't test at all. If Dantus isn't allowed to make reasonable inferences based on the available evidence then neither can you.

 

I dont really think bugs are the main issue at all, I havent encountered many.....

PERFORMANCE is where the real problems lies for everybody(thus the forum's name)

Lets not argue and keep the conversation on that topic

 

No I do not work for them and its clear I do not.. so your question is pointless.

I have not made claims that no testing is taking place, I have said I do not feel performance testing is being done. Its clear your another user who has just come into the discussion without reading the full thread.

Please find where I have said no testing is being done and quote me.

The way Dantus has spoken is as if he knows how the QA team work, and right now I do not think anyone does.
So there is a slight difference and you just trying to challenge me and say I made claims they don't test at all..I think is a small fail.

 

I've been reading the thread since day 1... maybe I've completely misunderstood your many, many statements stating outright that Unity isn't testing things.

• There nothing wrong with how they update things now -> we all keep speculating things. We need Unity to reply. I just think its failed testing of releases.. If they need more time to test things then so be it, but let everyone know..
• We don't want customers to do the work, we want Unity QA department or whatever department to the job they suppose to, this includes performance testing.
• Still many issues would have been resolved with simply doing some manual testing.
• maybe there are no manual testers on the Unity team..I can only speculate as still not had any proper reply from Unity team (holidays are no excuse)
• but its not clear to me why the QA team cannot catch simple bugs or fail to test mobile project properly
• we see bad performance and bugs which we feel could have easily been seen by proper QA testing by Unity.
• He has a valid issue that just shows that Unity not even testing/updating there own assets/demos to work fully with latest releases 5.3
• And this is exactly what I feel and been seeing. Get customers to do the QA work for them!!..
• I'm not ranting, i simply asking for information and explanation why proper QA testing is not being done on major releases.
• I personally feel the Unity QA are to blame here, they lacking performance testing before release and of course poor mobile testing.

 

http://blogs.unity3d.com/2013/12/30/monitoring-unity-performance-part-i/
http://blogs.unity3d.com/2014/01/08/monitoring-unity-performance-part-ii/

 

Shouldn't we get back on topic, and complain about how slow Unity 5.3 is? Unity has already admitted (much, much earlier in this thread) that they need to address testing and their development process.

I'll start:

Man, it's slow!

 

Dude, no.

There is 100% trust here, but there are also rules and structure it's nothing to do with trust, thanks. Unity trusts @holliebuckets and @everyone else. They probably squint a bit when it comes to me but let it slide though.... hopefully.

Plus forums.... are a terrible way to get data, because you get so much bad data. A lot of people don't even express themselves clearly, but a feedback system is super clear!

 

I certainly do not agree for the simple fact that your reasons misses the whole point of what I suggest.

The whole point is not needing the users to do anything. The cloud projects will go into test sandboxes in order to detect a massive amount of issues before hand.

 

unity is so slow, my grandma can outrun it, go back to the start point and outrun it again with eyes closed

 

Listen yes these are highlight of statements through the thread I have written, some are fact (until proven otherwise) and some are out of pure frustration. But I do not say NO testing is taking place, if you feel I have then maybe its because when in such rant mode message not always is clear.

All I'm going to say to this is simple and off-topic and I really do not have any more energy to battle Unity FanBoys.

There are two people in life, the ones who follow what the media (one example) tell them and take the word of the media and will defend the stories outlined by the media. And there are those who do not take the word of the media, instead look for evidence, proof and research, this being in the form of viewing the event unfold or physical facts. I'm the second type, so anyone can claim all they wish and make whatever statement they wish to, unless there is some proof, hence in this case, a solid/mobile performance friendly/ no simple bugs that can be seen with manual tests Unity build I do not have faith some comments are 100percent true.

Anyway I rest my case and don't worry I shall not cause any more bad feelings to anyone or upset them with my opinions and thoughts, just carry on with how things go and I await the 5.4 performance thread.

 

Yes I agree the whole point is not needing the users to do anything, but I feel Unity will not add such a feature as cloud projects into test sandboxes. Simply because this is a lot of work and the fact most user projects are in gigs these days and not everyone is confident with uploading there project to cloud some place.

 

I agree with that 100% with all things in life

 

Why did you guys could not make one scene containing all Unity features like animated characters , particles, shaders , car physics, reflections, most of the Gui functions, almost all physic features and all others common features ?
This could be a big scene or world that would validate the majority of features could work together after some update and it would serve as benchmark with each new release.
Each Unity update, you would just need to add the next features to the big scene level and see if something is break and perform some becnhmark test before adding the new coming release features.

 

Ok, if you say so.. but she said herself that she was accused of cherry-picking, which is the reason she wants people to use the voting thing.

 

The entire point of what I initially said to you is that there are multiple ways to interpret what we've seen, and if you can make your interpretation, then so can others. Just because the conclusions they come to are different from yours isn't an excuse for you to try to shut down their ability to speak, and it doesn't automatically make them a fan boy. I was just calling you out on that inconsistency.

Back to the topic at hand... I actually agree with you to a point. There is definitely a feeling that new releases are being rushed and not necessarily being tested properly. But at the same time it's entirely possible that things *are* being tested, just not the particular combination of hardware and software and deployment platform that you and some others are having problems with. It is impossible to test all of these combinations.

In my particular project I haven't had any problems with upgrades that weren't easily addressed by changing the way I was doing something because Unity changed the way they were doing something. That seems entirely reasonable to me - if I upgrade to a new version I don't have an expectation that things are going to work exactly the same way, so I adapt as necessary. If I'm doing a project that I plan to release on a schedule then I'm not going to upgrade, and I'm not going to start a scheduled project with the expectation of upgrading when some promised future feature is available - I'll implement that feature myself if necessary to avoid the tool upgrade. If that thought process makes me a "fan boy" then so be it.

I've written code in one form or another nearly every single day for the past 35 years. When presented with something that maybe appears to be a bug in a tool I'm using my first assumption is *always* that it's something I've f*cked up. In 99% of cases that has been the correct assumption.

 

I mentioned this awhile back

 

That's sad they don't take attention to some great ideas.

Unity could create a big game project that only purpose is to benchmark and test how new added features behave within a complete game environment instead on some empty scene.
Improving and making the test method more robust with a big game level any Unity tester could run could seriously improve the quality of realeases.
What is better than testing new features in a complete game scenario ?

 

must be idiots over there at unity

 

No need to become insulting to people that try their best

They have priorities and perhaps they can't spend a day making a whole test level.

 

The feedback system is a joke, what are we supposed to do? Create a feedback saying something like 'Mobile Performance on 5.x is too slow, please speed it up' Then everyone that knows about the feedback entry assigns their 10 votes, thus removing their votes from everything else that wanted fixed? The current feedback system isn't the way to fix a problem like this. This forum is the way to do it, surly this thread alone provides more than enough smoke for unity to pull it's finger out and both admit on here there is something badly wrong on mobile and that they are fixing it before adding any more features.

 

Nope. You're supposed to use bug reporter for it. If it's slower for no reason in a different version, it IS a bug. Seriously, are you trolling?

Personally I don't believe in voting on a bug. I think that's actually wrong, and all bugs need fixing, and at cost of new features.

 

Bug reporting has its place but when you don't get paid by your client because you can't deliver past the bugs, here, now today, thats the rent, thats the bills, thats food on the table.

Bug Reporting is an investment in the future (and usually other peoples projects)
Which is great for the unity company but not its stranded userbase.

 

Very much, yes!

They cannot spend a day but expect users to spend weeks on the problems caused by that? I guess the priorities should be reconsidered then. Especially when thinking about the long run. I'm pretty sure it shouldn't be a big deal to create such a project and it wouldn't take huge efforts to maintain either. It's probably even more work to test on all the platforms. But even then, it should be worth it, right? I mean, with basic stuff breaking all the time, you should watch your basic stuff so it won't break when releasing.

 

You are heavily oversimplifying the situation. You may cover some issues with that, like e.g. the currently introduced issue with the particle system. Many other issues can not be caught like that. The big scenes containing as much functionality as possible could most likely only be used for standalone testing and not for mobile platforms.

 

You could easily split that up into separate scenes. I mean, come on.

 

Not really, no. I'm seeing bugs get nailed that were only reported 2-3 weeks prior. That's pretty timely. Specially for showstoppers which can get done in days.

But OK you're having a vent, letting off steam. I get it. But if you're a professional you can't be using bleeding edge versions of *any* engine, you need to stick on a stable one for you and stay with it. Not helpful, but it's what AAA does.

 

You still have to invest time as a user, though. And if you don't want to upload your whole project, it can take a lot of time to isolate an issue.
Don't get me wrong, I think it's great that Unity tries to react quickly to essential problems. I also like the option to get a patch release if I run into minor problems that might have been fixed between major releases. But overall, there just seem to be too many holes in the bug catcher's net right now. Especially when it comes to crucial bugs.

 

Yeah I mentioned earlier in this thread about that, it's a really respectable thing and time consuming. I get a lot of bugs I don't report because I'm tired man. I got limits. If I'm sure nobody else has done much about it, I report it.