Feedback The State of Unity & Packages in 2020

@frarf Yes workflow is a major pain point, there is another thread about this were I shared a couple of workflow tools that can help out until native workflow is improved more #72.

 

1. Not really I guess I got too illustrative with those arguments, I was basically trying to show the problem of expectation about stability, using "communication" about them as way to sorts these expectation.
2. Yes, but that's not the core of the problem, communication was an illustration.

So I'll try another take, ie think like a developper, that's one thing people have been reproching unity to do, to not make game, hence not understanding what are the stakes are.

The problem is that the language used is imprecise, bleeding edge and stability mean different thing in different context, so let's outline these context when making a game. Basically when making a game and evaluating features the most important thing is workflow, or "don't waste my time or create risk" ie getting stuff done as painlessly as possible, the way unity handle features is hella risky for a dev, it create uncertainty.

What stability really mean is that I can project myself ahead and see my game done, as such there is 3 phases:

1. Evaluation: I'm going to test something to see if it's worth it and if I can risk a whole project on it, ie is it useful? I'm evaluating the promise it give, stability of code or api don't matter much, HOWEVER stability of features (or promise features) is important, because I'm betting for them to still be there or be ready when the game ships. That's experimental, we are experiment to see if it will worked out and make a decision based on outcomes, or bail out when no cost has been spent.

2. Investment: We made a decision to use the features. Since making stuff take some time, I can anticipate and go through the pain of investing in a features that isn't ready and is unstable, HOWEVER I do expect features to be on the locked down, I expect bugs to get fix, and I can help finding them, I expect workflow to get better other time, and no threatening change to happen that would jeopardize the project, I expect communication to flow such as it makes the features better and more adapted to use cases. That's the bleeding edge.

3. Risk aversion: I want something that works out of the box and is battle tested, in order to have as few surprise as possible, it solve a problem well, have great painless workflow, and help get stuff done, it's well documented so when we hit a bump it doesn't stop the project, and basically allow us to plan ahead with great precision. Most major bugs have been solved, and it's on maintenance. The features is complete, and basically upgrade are quality of life stuff. This is all about feeling safe, it should just solve problem.

These phases basically correspond to the alpha, beta and released. People might have different tolerance between 2 and 3, and most risk averse would rather skip 2. IMHO numbered released should be most 2 and 3, locked features and only improvement, with clear deadline about finishing the features.

The problem is that unity has become massively threatening to this publique, out of the blue depreciation, giving up entire features, unkept promise deadline that kept being pushed, sudden change of method or sudden obfuscations. All these problem create stress on project management.

However, it's not say there is no any merit in a wild west approach, like I have outlined, the communication and access should be different in that case. Some project of unity handle this with the github repo, these are clearly separated and understood as not ready. There is 3 publiques for that part, the tinkerer, the "student", and the NIH. They are all on the over side of "bleeding edge" and generally don't mind instability but hate obfuscation. They aren't doing risky project.

A. The tinkerer is someone who will always test new things and contrast methods, he doesn't have a project in mind necessarily, or the project is build organically by testing the features, he is going to deconstruct and propose endless alternatives and make blog post and theory about a technique. They look for problem to be solved and a pallet of solutions.

B. The student is interested is learning how the sausage is made, he look at stuff to remake it and gain an understanding, will probably make a tweet showing the result, or write a blog post to explain how it's done to layman. He can help spread awareness about the features and explain what's great about it.

C. The NIH (not invented here) will just took a peak at teh code, and redo all of it or part of it on is own. He is never satisfied with the offering, and will make some adjustment or add elements, will probably do another repo to share his implementation or sell it on the asset store. He help make alternatives and cover the niche that can't be filled. The NIH can be doing legit project, redoing a features provide the safety net and control they couldn't have otherwise, it protect them from incoherent changes. However sudden obfuscation takes away their responsibility and make them mad.

The problem is that unity conflate the two publique in analysis, so they provide Chaotic LTS for "the bleeding edge" and is surprise some people want stability even in alpha and beta, but then equate stability more as in code rather than proper workflow. Ie Unity don't make games as an organization, and therefore cannot conceptualized the real stakes and proper solution.

 

What is your understanding of the Unity tech releases (eg 19.1, 19.2…) vs Unity LTS releases?

• Tech releases add new features every release. Some others may be deprecated. Things may change reasonably between each release.
• LTS are "frozen" Tech releases that don't add new features and receive fixes and platform upgrades only.

When you see a new Unity release, what is your expectation of quality?

I should be able to install it over my previous Unity release (Tech or LTS), upgrade my projects by following some upgrade notes, and continue working normally.

I don't expect crashes, show-stopper bugs, **serious known issues** (!). If the issue is known, DON'T release the new version until it's fixed!

I can expect that things may change between releases, and some times without backwards compatibility. That's understandable. That's what the conditional compilation defines are for (#if UNITY_2018_2_OR_NEWER). you can write your code being aware of the Unity version (therefore the precise features and API available) its being used at.

Currently that's impossible with Packages. There's no way to know even if some package is installed or not, so there's no way to write code that could survive different project setups.

Do you expect different levels of quality or completeness based on the release distinction?

No. I expect both Tech and LTS releases to be completed, tested and documented.

Uncompleted / untested / work-in-progress versions can't be released as Tech. I understand that's what the alpha and beta channels are for. A beta version can't be promoted to Tech until it's feature completed, tested, documented, and all the known bugs and issues are fixed.

I can perfectly understand Tech release dates being pushed forward while all known issues are being fixed. Hell, I'd even understand not having a settled release date at all. People wanting / needing to use a specific feature as-is can get the version from the Beta channel anytime.

However, I feel there's too much pressure put on Unity versions to be released at specific dates: GDC, Unite, keynotes, etc. That makes some version to be thrown at us as a Tech release with a huge list of known issues behind it (which is subtlety moved to the bottom of the Release Notes, btw).

What is the primary motivation for you to use the LTS vs the non-LTS version of Unity?

Stability. It may not have all the bleeding edge features, but it allows me to spend my time making games instead of fighting with the engine.

When we say something is in Preview, what does that mean to you? Why do you feel that way?

I feel it like a nightly build. Like if you give me access to your own development repo. Devs are working on it and everything may change / break / malfunction anytime. My own experience is lighting being entirely broken in a HDRP project as result of a minor upgrade in the package.

Does the expectation of quality change when you see something in Preview?

Yes. As said, it feels like a nightly build to me. I don't expect any quality at all.

What drives you towards using something in Preview?

In the best scenario I may take a look at how it works or its requirements so I could start adopting design decisions in my projects that could make easier to integrate it in the future, once it's released as stable.

Otherwise I may have a peek at some upcoming feature, maybe taking some nice pics or recording some video... But I won't spend too much time on it, nor won't do anything serious with it.

What keeps you away from using something in Preview?

It's a nightly build! Everything may change, break, explode, or become completely unusable from a version to another. How could I use it for anything but some personal amusement?

What bothers me most is that lately every new Unity feature is thrown at us as soon as a nice video can be recorded of it. These features are pushed to us like they're production-ready and we're encouraged to use them, but in the best case these Preview features are 2-3 years away from being actually stable and usable in production.

Feel free to ask if I can be of further assistance.

 

This. This right here. The amount of issues that are known to unity yet the version gets released anyway is insane. It should remain in beta until all issues are resolved. Even if users keep finding new ones. ESPECIALLY if users keep finding new ones. Just pushing it out and saying itll be solved in a further patch release, minor version or major version is NOT proper software project management and will yield worse problems down the line.

Case and point: Where we stand today. We wouldnt be here had this not been happneing repeatedly for the last 18 months.

 

I expect the next marketing tagline to be "robustness by default"

 

Performance, robustness, stability and future SRP compatibilty by default.

 

I mean, if we're asking the impossible, I'd like an ostrich.

 

@smcclelland Please find below responses to your questions and a few extra followup points/issues.

Core product questions:

1. What is your understanding of the Unity tech releases (eg 19.1, 19.2…) vs Unity LTS releases?
I understand that LTS releases will have 2 years of ongoing patches to improve stability over time. The TECH releases are incremental "major feature" update releases. I understand that these will be feature complete, but that patch releases will only be available until the next TECH release lands. I risk having to upgrade to a less-stable-but-newer TECH release of Unity to receive a fix for an issue I've encountered as fixes for the now-current TECH release of Unity will not be backported to a prior TECH release.

I see TECH releases as "feature previews".

2. When you see a new Unity release, what is your expectation of quality?
When I see a NON-ALPHA or NON-BETA release of Unity, I EXPECT a very high level of quality. I expect that the baseline performance be at or above the previous major release. I do not expect to encounter many crash bugs, if any.

Recent versions of Unity have lead me to ASSUME that the quality will be far lower than what I expect.

3. Do you expect different levels of quality or completeness based on the release distinction?
No. I expect the same level of quality and completeness for all official releases of Unity, regardless of TECH or LTS distinction. I assume that an LTS will be of slightly higher quality, only because it's the same exact thing as a .3 TECH release with a few patches already applied.

The ONLY time I expect different levels of completeness is when I'm playing with an Alpha release.
The ONLY time I expect different levels of quality is when I'm playing with an Alpha or Beta release.

4. What is the primary motivation for you to use the LTS vs the non-LTS version of Unity?
I understand that the LTS version of Unity will actually get bug fixes. If I file a bug report for a TECH release, the fix may arrive, but it may arrive in a newer TECH release than the one I'm on. If I'm on LTS, there is an expectation that bugs will be addressed with a follow-up patch (at least within the 2 year cycle).

Package specific questions:

1. When we say something is in Preview, what does that mean to you? Why do you feel that way?
There are two parts to this story.

1. Before I started using packages. My understanding of the "Preview" tag was that the features were ready for public consumption; that they would have passed a "Unity Quality" check. Further, I understood that they might see changes based on user feedback, but that this would likely be rare. This would be compared with a feature referred to as "Experimental": such features should be avoided unless you simply want to see what kinds of things Unity was working on for future versions and/or wanted to provide feedback to help direct that feature's development as you saw yourself as a core user of said feature.
2. After I started using packages. I understand that Preview packages shall not be trusted for more than, say, a game jam. They are some Unity Engineer's (or Team's) pet project and will have major version increases at random times with bug fixes really only applying to the bleeding edge version (which may not be compatible with your version of Unity). The "Preview" label actually means "Experimental On Steroids".

As to why I feel this way, a lot of it has to do with trying to understand what version of a package is compatible with a particular version of Unity. The experience of going to the documentation site and having to scan multiple version Changelog pages for some inkling of understanding of compatibility and seeing no clearly documented rhyme or reason for a major or even minor version increment left me feeling like there is zero care given to semver versioning and a blatant disregard for potential downstream affects on possible package users.

Steer clear of "Preview" packages at all costs unless you need the feature only temporarily.

2. Does the expectation of quality change when you see something in Preview? What drives you towards using something in Preview? What keeps you away from using something in Preview?

As there are multiple questions in this question, I'll break them out to respond individually:

• Does the expectation of quality change when you see something in Preview?
  It changes, yes, but only marginally. I expect the occasional bug, but I expect it only when I recognize that the thing I'm attempting to do may be a bit "odd" or "unexpected". Script errors/warnings, bugs with targeted integrated systems/packages, etc. should not happen. If that is the case, then this package should not have been released. The "Preview" label should not be an excuse.
• What drives you towards using something in Preview?
  When there is a feature that I need now. As I mentioned above, I will install Preview packages if they supply a feature that I need in the short term. One really huge example is the Unity Recorder. This is a pretty amazing package that recently helped us record footage of a demo we're building at 120FPS@4K. It worked flawlessly. That said, it hasn't received an update since December 1st, 2019 and is left in the "Preview" state. Every version of Unity Recorder 2.0.x was marked as "Preview". The latest version is 2.1.0 and it is also marked as "Preview". I would never add such a package to a project where it was integral to the operation of the project itself as I have zero confidence around what the package maintainers intend to do with the package or when they'll decide to bless a specific version. In the case of Unity Recorder, this is simply an external feature that I can install, use, and then remove.
• What keeps you away from using something in Preview?
  Generally three things:
  1. The understanding that this package is probably going to be a mess.
  2. That I have no clear idea what version of Unity this package is compatible with.
  3. That I have absolutely no idea of release cadence or timeline. Is this package ever going to leave Preview? When? Will the version of Unity I'm using be supported when it does? There are no answers to these questions.

Bonus Content

On Features-As-Packages
I have to admit that I was pretty excited to hear about the Packages initiative for Unity. My original understanding of the idea here was that we would have an extremely small "Unity Kernel" installation into which we could install only the features that we wanted. Don't need physics in your game? Great! Don't include the package. Using a different audio engine? Sure! Skip on the AudioSource system.

However, I also expected that each release of that "Unity Kernel" would have a fixed version of each of the associated packages/features. What this would look like with the Unity TECH/LTS versioning system and, say, the URP package, is the following:

• Unity 2018.4 LTS: The compatible version of URP is 2018.4.x. It will continue to get bugfixes for the same duration as the 2018.4 LTS "Unity Kernel".
• Unity 2019.1: The compatible version of URP is 2019.1.x. It will only receive bugfixes while the 2019.1 TECH "Unity Kernel" version does.
• Unity 2019.4 LTS: The compatible version of URP is 2019.4.x. It will continue to get bugfixes for the same duration as the 2019.4 LTS "Unity Kernel".

As a bonus, I might further have the ability to install versions of packages that aren't specifically "in line" with my version of "Unity Kernel" but that I would be operating on my own (unless "Verified" to also work on certain previous versions). In this manner, I could conceivably update some packages to a newer version while understanding that what I'm doing may involve taking on some risk on my part. (This would look like installing a package with version 2019.2.x into a project running in "Unity Kernel" 2019.1.x.)

It remains my hope that something like what I described above can shake out of all of this current mess that we seem to be in...

On Package Versioning Issues
There's a serious issue with package version and change communication right now. The Changelogs between various versions of a package (particularly the SRPs) are inconsistent and show different information. Take, for example, this set of steps:

1. I have just read one of @jbooth's posts about compatibility issues between URP 7.1.8 and 7.2.1. I would like to see what changed in the package.
2. I toss "universal render pipeline" into Google and see that the very first hit is a link to documentation for com.unity.render-pipelines.universal@latest. Perfect.
3. That link actually took me to the page for version @8.0. I now see that version 8 is out.
4. To see what has changed, I click the "Changelog" link at the top. I start looking for changes between 7.1.8 and 7.2.1. But... wait! Neither of those versions are listed?! What the hell is happening here? I see 7.0.0, 7.0.1, and 7.1.1. It then jumps to 8.0.0. After poking around the page for a bit, I notice that there's a version dropdown in the top-left corner. I click it and see options for 7.0, 7.1, and 7.2.
5. I click the link for 7.2, which takes me a different Changelog. I now see versions 7.0.1, 7.1.1 thru 7.1.8, 7.2.0, and 7.2.1.

Why weren't those versions all listed in the same Changelog? What is going on here? It's already impossible to get any idea as to what package version is expected to be compatible with which version of Unity... now it seems we need to also jump through hoops to actually see what enhancements have been made over time... (For the record, it appears that this is due to Unity maintaining separate release branches in GitHub for the render pipelines [7.x.x; 8.x.x].)

Usability-wise, this is... less than ideal.


I hope this is helpful.

 

Wouldn't change a comma. The changelog thing is so ridiculous it makes me question how it got into production. Is the most backwards way of making a changelog I can think of.

 

As a sidenote, I'd like to point out that the 8.0 and 8.0.1 changelogs concern version which "came out" (are to come out) on may, 25th...

 

My company is developing VR experiences which are used to prevent accidents at workplaces and which are used in the medical section. Because of long term support questions for hospitals etc, we're bound to LTS versions.

To me, LTS version should contain 2 kind of changes:

a) bug fixes
b) performance improvements

Any bug that is found in an LTS version should be fixed. I don't want to see any "cannot fix in LTS, you need to switch to a later version" kind of excuse. This kind of "advice" is bad business behavior.

I wonder if Unity has any kind of ISO certification. The way bugs, requests and customers are handled wouldn't pass ANY software development ISO certification. We're currently going through the process of ISO 13485 certification (QMS/Quality management system) which means that we have to be sure that any third party software that is used to produce software for a medical device is up to a certain standard (i.e. ISO 9001 or ISO 90003 + ISO 12207 ("software lifecycle processes")).

Taking a look at how the entire 2019 cycle has been managed, I highly doubt that Unity as certified in any of those processes...

 

And this despite the fact I reported the incorrect date on the github commit where the original error was made

 

Battling packages and the editor stability and compatability issues and lack of documentation issues and trying to work out why X doesnt work and Y now does work and trying to find a single minute to actually work on your game/product, is the current unity end user experience.

Thats about as concise as I can summarize it.

 

What is your understanding of the Unity tech releases (eg 19.1, 19.2…) vs Unity LTS releases?

Tech releases are basically feature previews. They're useful to start developing features on for our next release and then we can upgrade to the LTS before delivery. This only works for features included in a Tech release that are actually out of preview of course.

When you see a new Unity release, what is your expectation of quality?
The upgrade should be as painless as possible, but more importantly it shouldn't break anything. Case in point, since upgrading to 2019.3.3 (in anticipation of the next LTS release) it turns out our VR implementation's framerate has turned into a slide show.

I realise that with the many different configurations you can now make of Unity + packages it's easy to miss one in your testing cycles, but we have the exact same issue further down the line and a much harder time tracing the causes. A new version should be tested with all packages and all packages should be updated to work with it. Or if they can't for whatever reason, they should be marked as "not verified with X" and that should be in the release information of the Tech or LTS version.

Do you expect different levels of quality or completeness based on the release distinction?
I expect all releases to be stable and high enough quality to be used to create a game and release it commercially. For me the difference lies in the LTS receiving fixes for the more obscure bugs that are only found after long term use, or that come up with changes to drivers or similar.

What is the primary motivation for you to use the LTS vs the non-LTS version of Unity?
We offer long term support to our customers so need our product to be built on a long term supported version of Unity.

When we say something is in Preview, what does that mean to you? Why do you feel that way?
It's an alpha version and I can't rely on it at all, because the API's of the ones that I have tried have shifted before the Verified versions roll around.

Does the expectation of quality change when you see something in Preview? What drives you towards using something in Preview? What keeps you away from using something in Preview?
Preview means it's a work in progress and the final version will have different functionality and a different API. I mostly use preview packages to try out new functions and give feedback on them.

 

1. What is your understanding of the Unity tech releases (eg 19.1, 19.2…) vs Unity LTS releases?
   • Each TECH release contains new features and bug/annoyance fixes, which are stabilized up to until the next TECH release, when they are dropped as a rock. Due to deprecations, feature breakage, possibility of new bugs and performance regressions on new TECH releases, upgrading from one version to another during project development is a risky business as it's not possible to estimate how much effort it will take, or even if it's viable at all (if your game rely on a feature that got removed, or a major rewrite is necessary due to significant API changes).
   • The last TECH release will get promoted to LTS at some point and will get platform SDK updates, and some bug fixes for two years: even during LTS period, there are bugs which are only fixed on newer TECH releases and are never "backported" into LTS.
   • Based on the above, I view the TECH releases as having limited usability for shipping games on. Their stability is a gamble: there's no telling which project-affecting bugs will still be unfixed when the next TECH version is suddenly released and the old version is dropped like a rock.
   • For mobile games the TECH releases are unusable since Apple and Google now have tight requirements on keeping your platform SDK updated, forcing your game into upgrading to a newer Unity version every year, unless you have deep enough pockets to buy a source license.
2. When you see a new Unity release, what is your expectation of quality?
   • I would expect a new non-beta release to be fully usable for shipping, with maybe a few bugs that should get ironed in two or three patches. But...
   • Right now I consider any non-beta Unity release to be unusable until it gets half a dozen patches or so. Based on the kinds of bugs that get reported on the forums, and the occasional staff response, it seems to me that the internal testing these releases go through is insufficient and not reflective of how Unity is used in production, and they depend heavily on adventurous developers who actually use the thing all the way to actually hit the bugs.
   • A new LTS is different, of course, as that's actually a TECH release that got through more battle testing.
3. Do you expect different levels of quality or completeness based on the release distinction?

• Yes. If something is labeled LTS, it should have a minimum guarantee of stability. However, the TECH releases are promoted as "the new Unity version" and thus should not be blowing up on people's faces after they leave beta.

​

1. What is the primary motivation for you to use the LTS vs the non-LTS version of Unity?

• Being able to work on a year-long project without being forced to upgrade Unity mid-development due to a crippling bug or an unsupported platform SDK. Upgrading a project to a newer Unity is always risky business: 3rd party assets may-or-may-not work, APIs may-or-may-not change, features may-or-may-not disappear entirely, things that used to be fast are now slow. For us who actually make games on Unity, that costs money. The last time I had to upgrade a Unity project it took one developer two whole months.

​

And more package specific questions we would like feedback on:

1. When we say something is in Preview, what does that mean to you? Why do you feel that way?
   • That stability and performance are not guaranteed, it's not rigorously tested, APIs could change massively between releases, and that the package could be abandoned at any moment.
   • I feel that way because that's how it played out with many of the preview packages.
2. Does the expectation of quality change when you see something in Preview? What drives you towards using something in Preview? What keeps you away from using something in Preview?

• Yes, I assume it's not to be trusted for production.
• I avoid using preview packages unless it's for non-critical Editor-only tasks that are unlikely to harm packaged builds.
• What keeps me away from preview packages is that there's no guarantee they have been battle tested on multiple platforms for performance and stability (or even work on multiple platforms at all), that their APIs won't be overhauled and require me to rewrite all me code over and over again.

​

IMO, the new-SRP-features-on-2019 is a special case: there was a large miscommunication during the 2019 cycle about which SRP features were being worked on, which would make into 2019 and which wouldn't, causing the impression of broken promises.

Also, the fact that something like spotlight shadows requiring "core editor changes" is troubling in regards to the stated goal of the SRP: to expose rendering to the C# side and allow users to code whichever render pipelines their projects require. If something like this requires modifying the editor C++ code, then it means the "S" in "SRP" is not doing it's job correctly.

 

Reading more of this thread and, as I see it, there are two big problems here:

- New stuff being released incomplete.
- Lousy communication & marketing making it incredibly unclear all that is incomplete.

I've been in a similar position before and I'll give this one to marketing for free: if you have no big shiny features or partnerships to announce at the next Unite, but instead just spend the entire keynote building a game live on stage your userbase will love it. Because that would show us that you've managed to stabilize the product and that the features in it are actually complete and work.

Unity doesn't need big new things. What it needs is for the things it has to be polished to a high shine. Your editor's GUI has been overhauled to make changing it easier, keep going and improve the workflows in it. Improve the package manager so it's clear what I have installed and what I can install, what needs updating, WHAT THAT UPDATE WILL DO, etc, etc.

And finally, for the love of all that's sacred, invest in getting your documentation sorted. I do not want to have to dig through the forums for links to google docs to get the upgrade notes ever again!

 

Above all: triple or quadruple your docs team.

I know it already tripled a year or two ago, but it was not even close to enough. Things are so out of hand with docs that its stopped being a running community joke how bad they are, and has seriously started to get to people.

How many times over how many years are you going to hear your customers complain about this, many of which are paying customers (or would progress to become paying customers if simple things like docs were taken care of and showed investing in the engine instead of just investing in random aquisitions all over the place) before actually doing something about the docs?

Hiring a few more people at this point will not cut it, you need to literally multiply the teams current size by a serious factor - and then be very vocal about it so the community knows you have finally invested in something the community actually consitently asks for.

Nothing out of beta should be undocumented. Beta should be when you make your docs. You should never be previewing something for public use if it is completely undocumented. Docs should evolve with the product, not be hastily made months or years after it is finished.

 

THIS! A twitter thread should not be the best source of information for ECS when you're touting it everywhere as the best new feature and are actively rewriting parts of your own engine to use it!

To be fair, I had a long conversation with the new head of documentation at Unite Copenhagen last year. He said they are working on it, but are dealing with a massive group of developers with different workflows and multiple legacy systems for documenting in that all need unifying. So please give this team more love and more priority!

 

Agreed, the docs team are trying hard and doing a good job. Effort or skill is not to blame, they are simply under-resourced.

I guess in old unity where everything followed the old way of doing things it was okay to be this lazy as the community had the knowledge to fill in the gaps. But now with everything being new (DOTS, SRP, Packages etc) its becoming super apparent how lacking the docs are, and I think that old mentality has to change.

Docs should be the number 1 priority team for hiring and growing for unity for the next year, it would do amazing things both in the short term (customer satisfaction and some backtracking of the confusion+rage being felt by many) and in the long term (ensuring unity doesnt get back into this situation as it continues to grow with the ever increasing rate of aquistions and new features piling in).

OR even better if you cant afford to grow docs team at the rate needed: allow users to make edits / suggestions to the docs. We literally have been shouting about specific docs for ages and have no way to change or update them ourselves. Turn it into a wiki and have the docs team manage the submissions, thereby increasing the amount of knowledable users who can provide info and allowing docs team breathing room to focus on what goes in and what doesnt, managing docs etc instead of playing this insane game of never-ending catchup.

 

It's not only on the doc team, the developers also need to be taught to at provide basic documentation for whatever they are writing, while they are writing it. All teams need to follow a minimum set of standards and procedures.

Public functions without even the basic C# three slashes comment docs should not be acceptable and should be flagged during code review (assuming that is even enforced) as that not only harms users, it also makes the code harder to understand for other members of the same development team.

There must be a permanent effort to write code in a way that makes it easy for others to understand it. When everyone is writing properly commented code, creating external documentation becomes much less problematic.

 

I was planning to rant about docs upthread, but I skipped over it.

There's two major issues when it comes to documentation:
1: The new package docs software you're using is garbage. The main documentation site is better in every single way, why on earth are you using a different, worse one now?

2: It seems like teams get to ship* without documenting properly, with the expectation that the docs team will clean up after them.

So I don't really think staffing up the docs team will help much. The other teams needs to be made responsible for ensuring that the stuff they ship has proper documentations and tutorials before they get to run off to play with a new, fun thing.



*For example, every single feature the animation team has shipped since I started using Unity - sub state machines, state machine behaviours, playables, animation jobs - has either had really poor or absolutely no documentation. So that team has systematically not been able to do documentation properly for nearly a decade. And that hasn't really been addressed, ever.

 

Agreed.

When I said give more love to the docs team, I should have added "and give them the power to make others do their parts". Or have some other manager enforce that. I don't care about the details, but imo the docs team should sign off on all the docs produced by other teams.

I also agree that if a feature does not have docs it isn't done and should not be shipped. Documentation is part of every feature.

 

This. It is incredibly inefficient to have someone else come along and document something after the fact, and leads to large gaps in knowledge. Any time I write a new feature or change functionality, I try to update the docs at that time. If I was working with a documentation team I would expect them to be there to review and make my changes better, not to come along and do it for me.

A while back in the 5.x cycle I asked for documentation on the shader lighting system - this was with an enterprise support account and source access, mind you. And the response I got was "The shader code changes too fast to document", which is 100% the kind of answer you get when some graphics coder just wants to be excused from doing the work. Documentation is every bit as important as designing a good API, in fact, I'd argue it's part of the task itself.

 

Honestly, I wish I could believe this, but documentation has become an increasingly sore spot since 5.0 came out in 2015, and I can remember hearing this exact same thing during the 5.x lifecycle. There exist things in the scripting reference, for instance, that have been there for years and years but say things like "CodeBargle() bargles your code" while providing no example or no explanation.

I really should start logging the individual examples. I've run into them over the last few days more often than not as I've been working on a remeshing system.

 

There should be a "suggest improvement" and comment section on the docs. Honestly the user community is being under utilized in fleshing out documentation with real world information. Areas like texture format are rife with all kinds of subtle gotcha's which are undocumented and not easy to spot unless you're actively trying to store data with some specific range in them (unsigned values, non-color HDR values, etc). Some of these are platform specific too, like some platforms clamping HDR textures, etc, and I'm pretty sure the developer wouldn't have even realized some of this stuff when writing the code.

 

BTW did anyone notice that @smcclelland stopped responding 51 messages ago?

 

Hasn't logged on since yesterday, might be a day off.

 

Fair enough

 

I feel like very early preview packages should be labeled as "Unstable-preview" or something similar, "preview" alone in my opinion does nothing to note that they should not be used in production.

 

Usually once I bring up surface shaders and the SRP fiasco Unity stops responding to that thread. So you can blame me.

 

He's probably collecting answers for all the feedback in this thread.

Spoiler

 

We already have terms for this:

1. [Experimental]
2. Alpha
3. Beta
4. Release Candidate

"Preview" simply feels like marketing designed to hide the fact that "Preview" is equivalent to Experimental or Alpha, depending on project/team.

 

Yes, that seems to match what I'm describing.

Yes, I think LTS is still required. The Stable branch would still be getting new features regularly, potentially with unforeseen bugs or regressions. The added benefit of keeping the Stable branch "pristine and stable as possible" is the added stability. You could reasonably expect to do actual production work on the Stable branch, and not have to worry about fighting bugs all the time as is currently the case with the TECH releases. The LTS branch would act as a snapshot of an already very stable Stable branch, with the additional peace of mind that no new features with potential bugs, regressions, or conflicts would be introduced. I want Nintendo 3DS meme-levels of stability for the LTS branch!





One further clarification on the desired difference between Stable and LTS in my mind.

Right now I see the current LTS releases as almost but not quite what I would like my proposed Stable releases to be. Currently, TECH releases are buggy enough that I generally avoid them if I can and just stick to LTS releases so I can get work done. Even LTS releases don't start out quite as stable as I'd like them to be. And they don't always get bug fixes as quickly as I would like them to. The fact that Unity 2017 LTS, which is nearing end-of-life, still has known issues which have persisted without a fix for many months tells me it's not really supported as well as I'd like or expect in an LTS release.

But the fact remains: If I want a mostly bug-free editor that I can get some work done in, it means I need to stick with LTS releases. By this point, the version I'm using is already on life support and I have to plan to develop and release a project within 2 years. Then when the 2 years is over and the LTS has reached EOL, I have no real chance to maintain my project without the headache of dealing with breaking changes due to upgrading to a more recent LTS.

What I would like is for the Stable release to be where all the work gets done. It can be done over the course of several years because new features are regularly added, but they are complete and stable when they are added, so I don't have to fear about upgrades! Only once per year (or two if you want to move to bi-annual LTS releases) do I have to be concerned about breaking changes, and if I don't like the breaking changes I can just stick the the LTS snapshot that took place just before the release containing the breaking changes. But assuming I've kept upgrading to the most recent Stable release for the duration of development on my project, once I've finished with a project and am no longer doing any major development on it, I can transition it to the next LTS release and put it in kind of a "maintenance" mode. This will allow me to ensure future compatibility with all supported platforms for a few years after the project is finished without having to worry about anything else changing and breaking things.

In fact, I'm starting to think I'd prefer an LTS release every two years, and having each LTS release be supported for 5 years. So if my hypothetical finished/maintenance project was using my hypothetical Unity 2020 LTS then I could safely stay on that version throughout the release of Unity 2022 LTS and still have about a full year after the release of Unity 2024 LTS to upgrade/transition my project to one of the newer LTS releases.

Personally, I'm pretty much only interested in the Stable/LTS side of things. I don't mind reporting the occasional bug I come across, but I'm not interested in feeling like I'm working for Unity's QA department without pay. Unity is a tool to accomplish a task. I want to be able to focus on the task and not fight with my tools.

That said, I understand the need for preview/beta releases to get valuable feedback on features before they ship. I think it's up to you and the data/demand you have from your customers to determine how early you want to expose those upcoming features to the public. How early in the process you want or need feedback on a new feature is beyond my expertise to give a well informed opinion.

My wish is that features that are not finished/stable do not bog down the rest of the editor and prevent people from getting work done, and also that unfinished (beta or worse) releases are clearly marked as such and not evangelized by Unity (e.g., on the blog or social media, etc.) as something that the average person should start using today.

To put it in gaming terms, the various releases should be "advertised" as such:

LTS: Easy Mode
Stable/Release: Normal Mode
Beta/Preview: Hard Mode
Alpha/Experimental: Expert Mode ("The Dark Souls of Game Dev")

I don't really care how you handle versioning, as long as it is logical and consistent. That said, it does seem like a good idea to include the year number in the LTS version. So even if the "real" version number was something like Unity 124.26.5, then the snapshot taken from that could become Unity 2021 LTS, for example. Or perhaps the year could be shortened down to just e.g., Unity 21 LTS.

I thought I was quite clear on this, but it seems I was not. I'll try to add more emphasis this time.

If a feature is not ready, it does not get promoted to the next phase.

You need to clearly and strictly define the criteria for each branch/phase. Something like this:

Alpha/Experimental: Very experimental or proof of concept. Anything and everything may change about it. It may be removed completely and never make it to beta. Expect the possibility of many bugs, changes, etc. Don't expect to be able to get any real work done at this stage. Thar be dragons.

Beta/Preview: Feature complete, relatively stable. Maybe needs some polish, documentation, and still working on minor bugs or other (non-major) known issues. API should be relatively locked down by this point, but there's wiggle room for minor changes as needed, or, rarely, a big breaking one that is needed. Better to break things here than later in the Stable branch. A developer could possibly get an early start on real work at this stage, but it is not generally encouraged by Unity and you do so at your own risk. This is bleeding edge; it has cool new features you can try out, but they might backfire and harm you or your project.

Stable/Released (and to an extent, LTS): Feature complete. No known bugs/issues. Documentation is complete. API is unchanging. No breaking changes, period (with the single exception described in my previous post). Any bugs or regressions are promptly fixed when they are discovered.


Force this mindset to really sink into Unity employees by requiring them to etch in stone, 100 times, by hand (hammer and chisel) the following:

"I will not promote a feature from one phase to the next if it does not meet all the criteria for that next phase."

Then make them sign their etching as a contract, in blood. If they break the contract, they get 1,000 lashes. And then they get a very stern look from a DeepFaked Mister Rogers, who, after pausing a moment to let the disappointment sink in, somberly tells them that everyone makes mistakes. Then he smiles, invites them to be his neighbor, and reassures them that he still loves them just the way they are.

Similar for PR/evangelists, except the etching should be something like "I will not encourage developers to use an experimental/preview feature, nor will I label it as production ready."

In short: Deferral is preferred.

Yes. This is what I want with Unity. Less emphasis on "shipping by the deadline, no matter what" and more emphasis on releasing fully realized, high quality, stable features.

 

My problem with the move to package manager is it feels like package manager is trying to be too much now.

 

I dislike experimental as a moniker for features that intend to go beyond the experimental stage as it sounds something that would get tossed into a one off github repo and then never touched again. There's no sense of long term viability in the name. The rest I agree with, but clearly unity doesnt, given that packages only have preview, a lack of preview and verified.

 

Yes but I think with Unity packages, they are at the experimental stage and then alpha when only in GitHub. Then preview=beta. I made the mistake of believing the new standard assets characters package on Github would eventually be released. But this proved not to be the case. In a sense I have no one but myself to blame for that mistake. (Though i do blame UT for discontinuing an essential package which barely saw a commit in the past 2 years).

The issue is with ‘verified’. It should mean verified to have no conflicts with ALL of the latest verified packages. But I’m not confident that’s the case. And who knows what unlabelled packages actually signify.

 

@Edy
Interesting point about packages - are there examples of situations where this has burned you upgrading a Unity version and having the package thrash a project? In the instances where you upgrade editor versions, would you expect on project launch to be informed that the package also needed to be updated? Would you expect the package remained forwards compatible so the old version worked with the new version of Unity? From the code side, what would you expect API or query wise to reason about what packages exist or are in use?

In your eyes, is our release cadence too fast? You mention tying it to specific events so would you rather see us take a more long-term stance of the product and ship when it's ready instead of fixed target dates?

Sounds like the confidence level in Preview for you is quite low? Comparing to a nightly for example is actually something we had considered as part of an 'Experimental' package cycle. How could we improve the quality of Preview packages for you and make it so you'd have a bit more confidence in our ability to deliver them?

Thanks!

 

It's not just the cadence, but the content itself. Things are being released prematurely because they lack the functionality they require; even outside of preview, releases are coming with sweeping API changes; for months outside of preview, everything feels like what we would expect from beta software; previews feel like early alphas.

 

@SonicBloomEric

So for you the tech releases are more like early access/feature previews and not necessarily viewed as stable versions of Unity? A follow-up question there is, should they be? If we could solidify tech releases, what would it take to instil confidence in our customers to use them in production environments?

That's unfortunate to hear. Are there standout examples where the quality hasn't been up to par? As much detail as possible here helps us isolate in on addressing the root problems.

That's fair. Alpha's and Beta's are there for testing purposes and to have the community help inform us if we're ready to stamp the later beta builds as RC or Gold candidates. What has your experience been with the Alpha/Beta cycle? Does it give you the ability to communicate with us on whether things are ready for launch?

Regarding your first point about Preview packages, that is absolutely the intent of them is to develop in the open with our customers and iterate with them on the package functionality. The unfortunate side effect of this has been Preview packages come in all different forms of quality, robustness, and completeness. This brings me to the next question which is - if we can correct this, is there value in getting access to more stable, high quality Preview packages? And secondly, what can we do to help build back that trust? Semver has been another big pain point where we've taken a standard and not followed it or created our own weird version - apologies for that, we're in the process of getting that under much better control.

Regarding compatibility, what would you expect the system to look like here? If a package isn't compatible with your current version of Unity, what would you expect to happen? If you tried to upgrade a package to an incompatible version? What about if you tried to upgrade one package that had dependencies on others that also needed to be upgraded? Internally, our team has been working on a couple solutions we feel will help with this but our designers have a keen interest in how we'll expose this in the editor to make a much better package experience.

When you say the occasional bug, the expectation here is still that a Preview package would be relatively stable and robust, right? If we had three versions of packages such as Experimental, Preview, Released, what would the expectation be for each? You mention support in the next part so that's also something I want to touch on - what's the expectation when you see a Preview package for support? Do you expect active development and iteration for X years? When we transition a package into the Released state, what's the expectation of support and active development from then onwards?

Interesting, so in your eyes we would basically have a series of release branches of packages and editor versions? So using URP for example we'd have a 2018.4.x LTS branch for the package that would be continuously updated with bug fixes as long as 2018.4 LTS was supported? Do you expect any functionality to be added like others above pointed out? i.e. we add SpotLight Shadows in 2020.1, would you expect that to land in 2018.4 LTS branch? When installing packages outside the "Unity Kernel" would you still expect some guidance and structure around compatibility so you were installing packages that were compatible with the rest of your project/editor environment? You mention operating on your own so I just want to understand how much of this you see as hands-off and you dealing with us versus us providing a lot of helpful UX and workflows to ensure you're still getting compatible pieces.

Changelogs are something that some teams handle really well while others can get a bit messy. Ideally, every package changelog would have every update and detail necessary for our users to make well informed decisions when evaluating new package versions. This would also adhere to strict SemVer rules so our customers could also rely on knowing that breaking changes will only be introduced in major versions and not snuck in as a patch release. I've made a note of this as something our team should dig in on at the source level to ensure teams are updating and tracking their changelogs appropriately.

This was extremely helpful. Thank you for taking the time to provide all this insightful feedback!

 

We have a bunch of release metrics and our release management is responsible for tracking the quality of a release. It's not as stringent as ISO certifications though. I will say that we're re-evaluating a lot of the release systems as well as internal systems to help build a much bigger picture around the state of things. Packages have brought with them what's often called "Dependency Hell" in the open source world and that's something we're really diving into now to ensure we have a much better infrastructure around this. Any suggestions on ways you think we can make this much better and not have a repeat 2019 cycle, please let us know!

 

Yeah agree with this point. The release cadence would not be an issue if the actual release boxes were stuck to, as in alpha for alpha style stuff, beta for beta style stuff, and release for actual release quality stuff - as in documented complete features that have been tested and are free of known bugs, with only perhaps minor bugs or unknown bugs sweeping through the cracks due to the multiple stages of internal developer, QA and user testing this should have gone through.

Right now there isnt really a big difference between versions or tags or whatever. Its all just whatever snapshot of the package or editor was available at the point people said the release would come out, thats what comes out - bugs and users be damned.

2019.3 was amazingly bad when it was originally meant to come out, everyone complained and it was pushed back and still came out too early with a ton of bugs.

@smcclelland I would be very interested to hear why a company as big and as global and dealing with as many users as yours, does not have ISO certifications? What is the reasoning behind this? Surely if your comitted to quality then that should be top of your list and would enforce you to actually take stock of this stuff with audits included... at least as far as issue tracking etc goes

 

Yup. This has been identified as a core problem for us to solve and something I've been actively working on internally. We've got a few solutions we think can really help shore this up so every developer knows the impact a change will have on the entire ecosystem (packages and editor). We then think there's an opportunity to take this compatibility data and push that to Package Manager or even the Hub to help our users make much more informed decisions i.e. you upgrade from 2019.3 to 2020.1 and we see two packages that need to be bumped, we can inform you as well as provide info about any API changes that happened between those two packages. Similarly, in package manager we can move whole sets of packages or inform users in the frontend regarding compatibility. I don't have a timeline for all of this but what you summarized is the very core of the problem we're aiming to solve with all of this.

 

That would be a great step in the right direction

 

Regarding the slowdowns you've hit in 2019.3 - do you have any Fogbugz issues or anything I can point teams towards? Was there anything specifically that you found to be the culprit or just 2019.3 in general?

Agreed, we definitely don't want to be treating our customers as the guinea pigs having to find all of this stuff.

A bit of an odd question but what if Preview didn't necessarily have a different API unless it was a new major version? For example, what if we could guarantee that the XR Interaction Kit across it's 1.0.0-preview status would not change API's until it entered into a 2.0.0-preview? Would that change anything for you?

 

Interesting point about platforms support and Tech streams. What can we do there to help make that less painful for you? You mention below that you'd expect a non-beta release to be stable for shipping so that would also involve Tech releases for you? Seems there's a bunch we could do to solidify the tech releases more going forward.

I'll ask this one again here but, what if Preview didn't so flippantly change API's? What if we could guarantee that the only time we'd ever make a breaking change was in a major version of the package? Would that help alleviate some of the pain with Preview packages? Regarding the battle-testing, is that something you feel more confident in when you see a package that's gone from Preview -> Release/Verified?

Regarding SRP, it's a real challenge because while you can abstract a lot of the rendering code into the C# layer above any core engine changes are still made at the lower level as that's where the engine itself resides. It's not so much a flaw in SRP as it is just how packages in general work - you have to really know what your core product is and then packages build on-top of that functionality. I won't speak for the graphics team here but I think if we look back on what the past two years has taught us about packages we'd probably make some different choices there.

 

Our documentation team definitely appreciates the love It's another area we're actively shoring up and working with teams to make sure we have much better documentation coverage. I won't speak for them here as I'm sure someone from that team can provide far better insights and direction than myself.

 

API changes should have a clear reason why, to me it sometimes seems like the dev team implements an API and then after the cleanup team looks at it and proposes name changes for clarity or consistency, thus creating an other API change. If this is true, and it can be avoided that would possible make API changes less occurring.

 

Still alive! Been stuck in a lot of meetings as well as digging in and processing/categorizing all of the survey feedback to distil to internal teams.

Not sure I should feel honoured, or scared, that the forum members are tracking my logged in status

 

As much as I'd love to engage there, it's something much better suited to the graphics team for feedback. I had a brief chat with them on Monday over a coffee and they're very much aware of the request and conversations. Again, I don't want to speak for them here so I've passed along the feedback and spoken with the team to engage in the graphics discussions.