Feedback The State of Unity & Packages in 2020

When the announcement of changing Tech releases from three to two first came out, there was a typo where 2020 was renamed 2021. For a moment I was like "they finally get it!", then they "fixed" it.

 

Same problem here.

You need to communicate that information better @smcclelland

 

Two things:

1. I assume that by "changelings" you mean "changelogs" (autocorrect, perhaps?)
2. I'm not sure that this accurately captures the issue we discussed. Someone on the SRP team could conclude from that writeup that their changelogs are all "standardized, accurate, and up-to-date". The problem is that they are not unified. When certain versions are released, the changelogs fork and you end up with multiples getting updated in tandem which makes tracking down recent changes you might care about really difficult.

I would suggest adding "unified" to that list of changeling () adjectives.

 

But that is not even the correct question to be asking. What you need to start asking is:

1. "What are we doing to encourage the community to contribute?"
2. "What roadblocks have we put in place (even if unintentionally) that would stop the community from contributing?"

These are extremely important questions.

Why? Well, let's take a look at the ScriptableRenderPipeline repository, one that I even suggested that some enterprising folks might fork in order to provide API access where they need it. In light of the above questions, the SRP team:

• Doesn't post roadmaps. There is no discussion about what is actively being developed and what is current priority.
• No documentation for Engine requirements. There is no documentation describing what Unity Core features the SRPs rely upon. This is extremely important, given that the team keeps claiming that "adding features to older Unity versions is hard or impossible because we're also updating the C++ side of things." Without understanding one half of that API contract, contribution is effectively impossible. It would be possible if the API contract was documented, buuuuuuut... it isn't.
• Open Issue Management. This one is pretty hilarious. The SRP readme has this extremely loud message at the tippity-top:
  "NOTE: We have migrated reported issues to FogBugz. You can only log further issues via the Unity bug tracker. To see how, read this."
  How in the world do you expect the community to contribute when they can't even see the issues, let alone comment on them?! There is no way for them (us) to effectively contribute to this at all.
• Development Documentation. There is zero documentation about the structure of the repository (which is complex, given that it simultaneously handles multiple versions of multiple "core" projects), let alone the tools and environment required to work with it. Where does one even begin with the repository?

And that's just scratching the surface.

Please see my previous note. The stats are going to be meaningless because you've done literally nothing to engender community involvement. This is the entirety of the Input System's repository's Readme:

A new Input System for Unity.
Check out the Input System documentation for more info.​

That's it. Two lines. There's no documentation on contributing, let alone a CLA (see this comment), no documentation on how to do anything, and folks posting issues are being asked to file a ticket with the Unity Bug Reporter in order to work on any problems:

could you file a ticket with the Unity bug reporter? Would like to take a closer look.​

Just for completeness' sake, let's take a look at the ShaderGraph repository. The Readme is quite simple:

We've moved!
Come visit us in the ScriptableRenderPipeline repository.​

To be fair, if you follow that link and find the current ShaderGraph git submodule, you'll find that there is at least some semblance of "instructions"... unfortunately, they're also extremely dated. They suggest that "Unity 2018.1 beta" is required and points to an archived Unity forum thread as a "Quick Start Guide". Yeesh.

To put it plainly, your questions assume that your repositories are set up for community involvement. They're decidedly not.

Your data is meaningless for the questions that you've asked.

It is all of the above.

It is also for information sharing: if I as a user have identified a bug and found a fix or workaround, I can post it on the issue. Google indexes those issues and someone else with a similar issue can locate my bug, vote for it, and even possibly make use of my workaround.

I might suggest taking a look at the ecosystem that Microsoft is building behind Visual Studio Code, TypeScript, and the network of tools through-and-around those projects as a decent model for repositories that engender community contribution.

 

• No. Better support of features and bug fixes on EACH point version.OR get rid of point versions and name them Alpha, Beta, Release and not 19.1,2,3,LTS.

• No. Provide an additional package tier meant for tech demos and experiments, and name "Preview" only those that are about to be released as final MAXIMUM in 6 months. ideally no more than 3 months before intended release. "Preview" for 2 years is not a "Preview", it's fiddling around.

 

There is a pull request that added AO to LWRP some years back.
https://github.com/beinteractive/LWRPAmbientOcclusion

Some amazing users kept updating the implementation with every new version of LWRP. It worked well and the performance impact contrary to the team's claims was negligible.

Everyone who was using LWRP was asking for it and has requested it with every opportunity in every channel. Here, on YouTube, on Discord, under your blogs, etc etc. So here we are.

A new SRP replaced LWRP. Does URP support AO?

 

I think I understand you but.

Those lines not about Unity releases, they about packages releases

//Better support of features and bug fixes in packages for LTS releases.
is must have for packages If Unity decide to go this way than we can have URP say 10.x.x and go with it may be for 5-10 years and it will support all Unity versions for 2 years back with most new feature and mostly all bugfixes. Only minor version of package will change and sometimes Unity announce deprecation of Unity x.x support. I with that deprecation must be with actual deprecation of Unity LTS.

 

I think the bottom line is "Give support for the packages in all of the supported versions of the editor (Be it TECH for a couple of months or LTS for 2 years)."

Now the question would be, how many versions of a given package are supported at the same time? I would guess the latest patch and minor for each major and majors up to the oldest supported in the oldest LTS?

For example, if the oldest LTS is 2018.4 and it supports up to package v3 (let's say v4's features needed core engine changes)

You would support V3 (latest) for 2018.4, 2019.4 and 2020.1 and V4 (latest) for 2019.4 and 2020.1, with bugfixes that apply going into both versions.

Until 2018 LTS reaches EOL. At that point you also drop support for V3?

God this is messy -_-'

 

I thinks it is no need to change major versions of package even if new features require new CoreEngine stuff.
Exactly for this reason we have defines like UNITY_2018_3_OR_NEWER new features that depend on new EngineCore just dont present for old engines because of define. All asset authors do this so it is not something new after all.

There will be no package V3 and V4. Just V3 with some minor versioning and some features but all features (a, b, c,d, e, f, g) work only for 2020.1, in 2019.3 feature a, b and c dont present, in 2018.4 feature d and e dont present either. All other stuff and all stability and bug fixes just in place for 2018 and no need for backporting. That is actually backwards compatibility, package just work as always, you just dont see part of new features created specifically for new Unity version, but all features and bug fixes that dont use new Engine stuff or can provide alternative codepath for older engine just work and arrive.

Package V4 can be created only for major changes like HybridRenderer V2 which is full redesign created from ground up of HybridRendering and (as I understand) fully depend on Unity 2020 CoreFeatures. OK it can support only Latest Unity but after release there is no new major versions and this version (with all new features and bugfixes) must support Unity 2020 right until 2020 LTS sunset.

 

First apologies for having posted the potentially demotivating expression of a feeling ECS might be suited for only a certain type of game(play) once; I now have "high hopes" for DOTS and am awaiting its final shape.

core product answers:
1. new, developed feature set fully working version vs add-on (for (very) long running big projects where potential incompatibilities pose more risk than outlook of gains)
2. as many of known bugs fixed as was feasibly possible, best performance yet
3. no
4. none existing, opposite currently: with many new (brightly looking) features in development and steadily releasing in tech, LTS lets me feel like being stuck with a dusty version

package specific answers:
1. might not work (completely, properly), might not be complete; API/usage/features can change. "Preview" sounds like the former and I might have read a disclaimer of the latter.
2. Yes, I accept it might not work completely or properly and that it might change and/or be extended. Preview for me is only relevant where I feel it can bring a "significant" gain in something I work on eg. performance, enabling features, making something easier. If the outlook of an API is that it will change a lot or it requires a lot of changes in the code and might only change little later I'm driven away because of the workload associated with (re-)adapting.

Unity has a vast array of great offerings, all for free, thank you!

 

1. What is your understanding of the Unity tech releases (eg 19.1, 19.2…) vs Unity LTS releases?
   When I have an LTS, I still don't usually get the patch I want: I'm often told to "get the latest to get the update". However, fixing 1 bug often causes many more (although it's preview, but still - this is the trend). LTS feels lacking.
   
   
2. When you see a new Unity release, what is your expectation of quality?
   Expectations or reality? In my experience, upgrading is generally hellish, but worthwhile. But focusing on the hellish part, this is what needs to be fixed.
   
   * For deprecated/obsolete stuff, why not inject some COMMENTS inside to give us 1:1 instructions what to do and add a link to top that off?
   
   * After an old project converted to the new version, why not validate serializations? Eg, I had to reimport all to wait another couple hours to port. I couldn't understand why my project was getting crazy errors: Nothing was serialized
   
   * Just some QoL features like this -- some integrity checks after upgrade. Unity is famously known for having awesome new-user experiences (new projects), but infamously known for horrible upgrades (existing projects). Throw a bone to existing projects - we're your backbone ~ and we're also likely the ones paying (rather than using the free version).
   
   
3. Do you expect different levels of quality or completeness based on the release distinction?
   You guys have been getting better at this. Compatibility is a big thing - allow us to downgrade certain modules for compatibility's sake. Eg, I want to downgrade Buffers System package. Where is Nuget to allow me to do this?
   
   From 2018>>2019, I had numerous Unicode issues with TMPro. Why not get QA to test this stuff? You guys claim Chinese support, but I don't even know how Chinese gamedevs handle it at all. I must've repro'd at least 6 scenarios and still doing it a year later. You need aggressive QA+Testers (esp for Unicode)!! Unity is a big project and since you guys don't actually make games with your own engine (why don't you?), you don't know what it's like to experience the "real issues" - don't wait until we've already run into the bug - that's what indies do, not enterprise-level, well-funded companies like Unity. Is Stephan really the only person working on TMPro? The man needs some aides - please give him 1/100000th of your budget for aggressive +QA. If not, remove the mention of Chinese/Unicode support, because at the moment, you barely have it (Alpha-level quality).
   
   
4. Do you expect different levels of quality or completeness based on the release distinction?
   Yes, since this is an enterprise product. Look at tech releases of Ubuntu and other enterprise products: Every tech preview is essentially an LTS without the liability of calling it LTS. I'd expect the same from Unity considering how much I'm paying per-seat ~
   
   
5. What is the primary motivation for you to use the LTS vs the non-LTS version of Unity?
   Bug fixes, performance, and stability. That's it. The tech versions are ironically more-stable than the LTS ones due to the vast number of bug fixes that come along with the tech prevs (although break others). I honestly don't care about features - at all - I just want editor performance and bug fixes.

And more package specific questions we would like feedback on:

1. When we say something is in Preview, what does that mean to you? Why do you feel that way?
   It means "we're not liable for new bugs that popup". Because that's ... just how it is, isn't it? However, I'm still going to get it in anyway ... because I have no choice.
   
   EXAMPLE: I've been on the last version of Unity 2019.1 for ages. I'm localizing and I can't even copy+paste Unicode (Chinese) from editor: Do you know how stressful it is to not be able to copy+paste to localization team for something that needs fixing or changing? I can't even move a character 2 slots over if there was a typo. I can't even copy a Unicode period to add one to the end. Does LTS help me? No. Does staying on 2019.1 help me? No. The only thing that helps me is to upgrade and risk my project. I've never had a choice - upgrading is inevitable since Unity rarely had a focus on stability/performance (until 2020; thank you, btw) -- you guys focus too much on new feats. Feels like Unity is a marketing company more than a game engine, sometimes.
   
   
2. Does the expectation of quality change when you see something in Preview? What drives you towards using something in Preview? What keeps you away from using something in Preview?
   I expect instability for Unity, for near-stability for others. Again, Unity is the only enterprise-level product I've used that is often very unstable for any "medium+" sized project (awesome for small projects, though). However, again I am driven towards previews to simply seek scraps of +editor performance and bug fixes. What keeps me away is incompatability to req'd 3rd-party libs (eg, Photon PUN realtime is incompatible with 2020 right now).

_______________

Important feedback about LTS:

* UNET: You implement it. Bugs happen. You need a patch.
What happens? Patch never arrives because Unity abandons it in favor of something else that you have to spend the resources to port (which is good, but not for existing projects; eg, anything long term like LTS).

* Text bugs: You implement it. Bugs happen. You need a patch
What happens? Patch never arrives because old text system gets abandoned in favor of TMPro that you have to spend the resources to port (which is good, but not for existing projects; eg, anything long term like LTS).

* IME: Chinese locale is having issue typing characters in the suggestion popup
What happens? Patch never arrives because Unity abandons it in favor of a new input system you have to spend the resources to port (which is good, but not for existing projects; eg, anything long term like LTS).

* Prefabs: Having issues copying unicode/Chinese from text areas in editor
What happens? Patch never arrives because Unity abandons this in favor of a new system that you have to spend the resources to port (which is good, but not for existing projects; eg, anything long term like LTS).

* Editor performance is sluggish to refresh assemblies, slowing performance dramatically
What happens? Patch never arrives because Unity abandons the old system in favor of DOTS/ECS (which is good, but not for existing projects; eg, anything long term like LTS).

...noticing a disturbing pattern? Major bugs don't get fixed - they just get abandoned completely. There's no point in LTS if Unity keeps brooming bugs under the carpet. This is why patches never arrive that you expect - because the feature is completely abandoned. This pattern is not a good one. If you move to a new system, at least patch the old one.

 

BTW kudos to @Stephan_B for providing outstanding support in the TMP forum around here. If there really is only a single person behind TMP, what could they achieve, if there were a team.

 

I’ve had TMP in my project over a range of Unity releases and never had any issues with it for my usage. So why has it been in preview for as long as I can remember? Having this seemingly super stable package in my project led me to believe that maybe the other preview packages were stable as well. Which of course is not the case. In other words @smcclelland, “preview” is not a term used consistently enough for us to trust what you mean by it. And perhaps UT teams would also benefit from knowing exactly what the term signifies.

 

Here's an example of the mess that are packages right now:I was toying with the idea of using Entities to optimize some areas of my project (this is a live project, so I'm on 2019.2).

- To begin with, the package manager refuses to install unless I disconnect from my work VPN (I'm WFH due to the COVID-19 situation). For unknown reasons UPM uses arcane methods to download packages instead of whatever the rest of Unity uses for internet communication.
- I now get this error when I load the project, enter or exit play mode:

Code (CSharp):

1. ReflectionTypeLoadException: Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
2. System.Reflection.Assembly.GetTypes () (at <567df3e0919241ba98db88bec4c6696f>:0)
3. Unity.Entities.TypeManager.InitializeAllComponentTypes () (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/Types/TypeManager.cs:678)
4. Unity.Entities.TypeManager.Initialize () (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/Types/TypeManager.cs:542)
5. Unity.Entities.EntityManager..ctor (Unity.Entities.World world) (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/EntityManager.cs:153)
6. Unity.Entities.World..ctor (System.String name) (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities/World.cs:62)
7. Unity.Entities.DefaultWorldInitialization.Initialize (System.String worldName, System.Boolean editorWorld) (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities.Hybrid/Injection/DefaultWorldInitialization.cs:55)
8. Unity.Entities.AutomaticWorldBootstrap.Initialize () (at Library/PackageCache/com.unity.entities@0.1.1-preview/Unity.Entities.Hybrid/Injection/AutomaticWorldBootstrap.cs:15)
9.  

- And exiting play mode prints these on my console:


Obviously the Entities package found an assembly that it shouldn't touch or something, but the developers apparently never tested in such scenarios. I'll just give up the idea of using entities altogether because I already have enough Unity bugs and quirks to work around and I'm not looking forward to adding more to the pile.

And no, I will not "open a bug report", because:

- I'm 2019.2 and I assume my reports will be thrown in the trash bin since it's no longer supported. In the rare chance they aren't, the fix will be "upgrade to 2019.3 and deal with new, unknown, bugs".
- Uploading this entire project is not viable, legally and bandwidth-wise.
- Unity development teams should do their due diligence instead of expecting users to set aside working hours to setup idiot-prof repro cases.

 

Because there's only 1 person working on TMPro - and while he's extremely productive, it's a giant module that needs additional QA budget. It's in a great state now, but originally (from my reports, alone):

• https://forum.unity.com/threads/tmpro-prev8-editor-_cullmode-material-errs.849691/#post-5609875
  
• https://forum.unity.com/threads/tmpro-unicode-cant-copy-from-any-texts.849559/#post-5605678
  
• https://forum.unity.com/threads/tmp...ethrow-errs-with-unicode.849484/#post-5605612
  
• https://forum.unity.com/threads/in-prefab-mode-tmpro-uguis-dont-display-vertical-align-diffs.847513/
  
• https://forum.unity.com/threads/tmp...n-bopomofo-taiwan-locale.717308/#post-5535391
  
• https://forum.unity.com/threads/tmp...aring-always-shows-in-editor-not-game.835048/
  
• https://forum.unity.com/threads/tmp...prougui-generatetextmesh.750398/#post-5042237
  
• https://forum.unity.com/threads/bug...-stackoverflow-exception.728672/#post-5011970
  
• https://forum.unity.com/threads/tmp...nadvertently-impacts-horizontal-align.738791/
  
• https://forum.unity.com/threads/bug...-is-changed-via-ui-event.698843/#post-4685135

Again, Stephan has been on the ball with this and fixed a great deal of them, but just an example: A lot of this stuff shouldn't need to even be reported (it's so darn common - like out of the box basic Unicode IME issues). Stephan is busy fixing things (seemingly alone) - so who is busy trying to find common, breaking bugs? And why doesn't he have help for what may be one of the most important features in Unity?

If Unity isn't going to make games with their own engine, their QA team should at least be aggressively making mock game mechanics just to find common bugs in different situations. Eg, if you say you support Chinese/Unicode as an enterprise product, you better be sure there's a dedicated QA team to aggressively test this stuff (Sorry, but Unity didn't even test the basics that anyone could've caught). Respectfully, this isn't an indie, low-budget engine: the QA budget/strategies shouldn't reflect as if it was one. Even LTS experiences this!

QA budget needs to go like this+++++++; you can't hold back on this. No one has yet to deny that there's only 1 person making, testing, AND QA'ing TMPro, for example. That's inexcusable considering how important this is. Remember UNET that had only 2 devs and 0 QA/support/doc writers? Remember how that turned out? Love you guys, but learn from your mistakes.

 

LTS releases are versions that get updates and bug fixes; the others are standalone versions.

At least the same level of quality the one I'm currently using has.

I suspect it more than I expect it. If I get more, I'm happy; if I don't, well, I can live with(out) it.

None, I don't use LTS versions.

It means: do not use because preview features or packages are not supposed to be ready to be used yet.

I've already answered the question somehow. I'd rather not use preview features or packages.

 

Indeed. I am talking about packages too.
If you are following a development paradigm, (Usually either Lean or Agile in most software houses, for some even ToC) you are following it for every aspect. and since packages have releases of their own, it applies for them too.

The way things are, (and rightly so else it would be crazy for the team to support 10 year old releases) we only really receive support for the LTS version especially since Unity officially does not consider a point release production ready. On release they declared 2019.3 NOT production ready. Which means people can't depend on Unity 19.3 for any project. If it is not production ready, it should not be a public release. This applies to everything. Core and Packages.

Providing long term support only for the LTS versions does not mean that the point releases should be buggy and unusable. It is against any development process that I know of, which makes it rather amateurish. Neither Lean nor Agile allow for such unusable buggy releases. (If it is not production ready, it is unusable as far as a serious studio which respects its developers is concerned, else by the time you are finished, you may need to scrap all your work) i.e. the Raytrace implementation or XR framework, or URP, etc.

If you are doing that, it means you are actually still use a waterfall paradigm and just slap a few "Agile/Lean" processes on top of it to make it look modern. That is not how Agile and Lean work. Hence my suggestion to avoid the .x release and call them with their respective waterfall monikers. Alpha, Beta, RC. Release. Then, nobody will have any expectations from the packages contained in these releases or the releases themselves People see every major point release as a complete offering. Only in Unity's case it is not.

 

I think it's essential to have both of these in place. When embarking on a brand new project it's not always clear which packages and just template you'll need. So we also need an option to easily find relevant packages and add them later on. Having the categories and templates use the same names would be very useful. Or perhaps even some way to add "templates" into an existing project.

 

Is there something that can be done about the responsiveness of the package manager? Even if it's just a superficial change, like caching the package list from the last startup, then putting a "getting info" indicator next to the package list. It feels kinda awful from a UX perspective to load up the package manager and then wait like 5 seconds before I can actually use it.

 

The feedback drives discussions, which in turn reveal requirements, and help keep the development flywheel turning. All of this is being directed to teams and leadership. I have little interest personally in trying to pacify the masses but I have a heavy interest in making sure our user base feels heard, respected, and wants to work with us to build a better Unity. While the list above seems massive it won't all be worked on at once.

Why the failure? Well, to be frank, that's just how we learn is through failure. We tried to do a lot of stuff in a short period of time and that seemed to create a ripple effect across the entire ecosystem. Lots of learnings that one day I hope can be covered in a retrospective blog post or conference talk.

 

While it is true that we learn from failure, it's important to consider where those points of failure occur. If they occur in the internal testing phases/public alpha/beta, they are far less severe than if they occur at, say, general release time. Alpha and Beta software is clearly not release ready and is really more in a preview state (asset developers, developers looking to see what an upgrade path will be like, etc.) and a testing state (many of the previous group and people looking to provide workflow feedback, etc.).

I fail a lot in development, but if I'm releasing a game to consumers, I don't really take an early access approach. I'm releasing finished products for people to play, but it feels weird that I'm taking those steps while I can't really say I'm seeing the same from the company that makes the engine I develop with.

 

Yeah, this was something that got completed recently on our end but it's not hooked up to all repos. There's some Github/repo level stuff we're working on now before we decide to roll it out across public repos that are accepting PR's.

 

@Murgilod - yep, many lessons learned from all of this.

 

Argh... autocorrect getting the better of me! I mean, changelings could be pretty cool though right? Theoretically, the solution here is to remove as much of the manual process as possible to prevent this from arising. The goal here would be to provide a set of automated tools and processes to aggregate and track changes, not so much have teams just simply say they're compliant. I'll add unified to make sure that's also covered.

 

The question is relevant though because hosting public repositories that accepts contributions has an inherent risk and cost associated with it. The very first thing a team needs to decide is if they're willing to work with the community and accept PR's. For some, they may not and would rather engage through issue tracking or other avenues. For others, they'll be fine with it. The risk is on our end to mitigate to ensure whatever is mirrored publicly is safe, validated, and secure. That said, the questions you've raised are absolutely valid when a team is decided should they open the repo up for public contributions.

Roadmaps - what's the expectation around having a visible roadmap for something like SRP? Is it just knowing what they're working on? Timelines? How detailed does it need to be?

Documentation - yep, there's a lot we can do via README.md and other documentation avenues to ensure people are set up for success working with the repo in the public space.

Issue Tracking - this is actually entirely fair on the team because of how the internal bug tracking system works. If we were open and accepting PR's then of course GitHub issues is where we'd want to work from but we also JIRA and FogBugz internally which are heavily used by teams. Having duplicate data spread across multiple bug systems sucks for everyone so in this instance, not opting to use Issues is better than using it and people getting pissed off that their issues aren't being interacted with.

Regarding the Input System... that must've changed recently because there used to be a very in-depth README file when I was on the project that described how to open issues against the repo and work with the codebase. Seems that's not been mirrored to the documentation. Keep in mind while these repos are publicly visible, that does not mean they are open for contributions so you're applying the assumption these teams accept PR's against them. If we did, then yes, there would need to be many changes here to ensure the success of people opening PR's against these repos.

Again, the question makes no assumptions but simply tries to understand how important is it for people to open PR's against a public Unity repository. There's no assumption there it's possible, just a simple question on is this important to our users?

 

I'm less interested in roadmaps with timelines (none of us can stick to timelines, really, timelines are like install progress bars) and more interested in roadmaps that show what's there, what's being actively worked on, what is planned to be worked on, and what's so long term that we might as well not consider it.

One thing I use even for projects that I'm the only person working on is something like Substantial's Hello Epics addon for Trello, which allows for linked card dependencies. I would love to see something like this for a Unity roadmap. I'd love to see what's being worked on, what its status is, and if it getting a status update is dependent on something else that I can also check the status of. I don't want to know what version these things might be available in, I only want to know what versions they are confirmed as locked-in.

 

You may believe that the question is relevant, but it is meaningless. To be specific, we are referring to:

The question that always comes up is how many people are actually contributing to the repos we have now?​

My point is that this implies that you guys are looking at the contribution numbers to determine whether there is any value to begin with in the approach. You're looking at the results of having some of these repos on GitHub for over a year and looking at the data saying "Well, but we don't have anyone providing PRs, so clearly no one wants to contribute."

My point is that this question is the cart that is before the horse. You CAN'T ask that question yet because it presupposes that your repositories are actually PREPARED for community contribution. My entire previous post was written to highlight the fact that they are decidedly NOT prepared for community contribution. You've simply said "Here's some source code. Good luck." and then closed down every possible standard door for community contribution.

Asking, then, "how many people are contributing to these repos?" is a ridiculous waste of time and will gain you nothing.

Apologies for all of the caps, bolds, and italics. I'm trying really hard to emphasize certain things because I fear the message isn't getting across.

Well, of course. But such teams wouldn't be asking that question anyway, right? My post wasn't trying to advocate for all teams going open source on GitHub - I was trying to point out that the questions you're asking are wrong.

If a team decides that they do want to support community-PRs then they need to take a look at successful communities/repositories on GitHub and learn about what it takes to foster community involvement. THEN you can start measuring the efficacy of the efforts by looking at the number of contributions to a given repository. Not before.

On this we agree entirely. My point is that you've said that the teams are already saying "Well, no one contributes to the repos we already do have up so why do we even bother?" And my point is that such a question is entirely valueless given the current sad state of Unity's repos (sad when viewed through a "community engagement" lens).

I would highly suggest that you take a look at some open source repositories for examples of this. There are various approaches. Some break version-specific-features/enhancements into Issues on GitHub and tag them with release targets. They will then update those issues with progress as sub-tasks are completed. In other repositories, a roadmap of "V2, V3, and V4" expectations will be laid out so the community can get an idea of what pace to expect from development and what features are taking focus.

It really shouldn't be hard for you guys as developers to figure out what to put down. I mean, imagine yourself as someone outside the company who is checking out the repository, possibly looking for something to help with: how do you know what's being done?

I fully understand this and 100% expected that this is what was going on. That said, my point stands: if you want to engage with the community, then you'll need to support a workflow that the community can engage with.

Put it another way: if one of the benefits to breaking things into Packages is seen as "fostering a co-development experience with direct involvement from the community of Unity experts" then Unity must find a way to provide access to active bugs. If this is not the case, and teams decide that they are happy to share source code but do not want to deal with community contributions or have to ask people to re-file bug reports with the Unity Bug Reporter, then a note should be made to that effect on the ReadMe and the GitHub Issues should simply be shut off.

Sure. Then the repositories should then be configured as such with an explanation. See:

The source code found within this repository is provided as-is. We are not accepting PRs on this repository and not accepting community-generated issues. If you encounter a bug with the system, please file a bug using the Unity Package Manager.​

Again, my writeup wasn't trying to suggest that all teams should be opening up for contributions. It was highlighting the fact that you cannot yet ask the question of "but how many people are actually taking advantage of our GitHub repositories to contribute?" because you have done nothing to foster or invite such things. That is all.

Great. But these two questions are entirely distinct:

1. Is [the ability to contribute to our code bases] important to our users?
2. How many people are actually contributing to the repos we have now?

The problem is that you are attempting to answer the first by measuring the second. And my point is that you cannot measure/ask the second question right now and expect to get meaningful data/results. There are false assumptions and I'm merely trying to highlight that.

A better (though still possibly flawed) question would actually be this:

• How many people are forking our repos?

You could use that to get some insight into user engagement as a fork is presumably someone willing to test out a "nightly" release or even adjust things on their own to add a feature or fix a bug. At time of writing, the Scriptable Render Pipeline repository has a healthy 773 forks which seems to indicate that there are a goodly number of users interested in delving deeper.

In summation, I think framing the question as "Is the ability to contribute to our code bases important to our users?" rather than "How many people contribute to our current repos?" is a much better because it doesn't make any assumptions about the quality of the repositories (again, "quality" from a 'community engagement' standpoint).

I hope this helps!

 

Rider uses a combination of their internal bug tracking system (on YouTrack) and Github issues, and they seem to have no problems whatsoever managing to do that.

In that instance, it doesn't suck at all. If a complaint is added both as a bug and as a github issue, they handle that by adding a comment on the issue that links the bug in the tracker.

You really need to get in touch with them, because they're doing exactly the same thing you're asking about in this thread - balancing a main product (Rider) with a bunch of plugins (Rider for Unity). Their main product is closed source, the plugins are developed out in the open on GitHub.

And they rock! You already have a partnership with them, so I really suggest you sit down with @IvanShakhov and the rest of the people working on Rider for Unity and take some suggestions on how to manage package development and mainline releases, because they're a lot better at it than you are.

 

First - My workflow now relies on creating libraries as local packages, and making custom project templates with pre-loaded packages. It's a great improvement in work efficiency, code modularity, and maintainability. Keeping packages (internal & external/3rd-party) in sync with editor/engine versions becomes a straightforward matter at that point, and rollbacks & patches are also a lot easier.

There should be a tutorial on this approach, for devs who have the requisite skills and project-maintenance needs. Shout-out goes to LotteMakesStuff, for the original tutorial that put me onto this technique.

 

Re: Community co-op, Pull requests, etc - It seems likely that only a very few devs outside Unity would actually have the time & know-how to contribute effectively to Unity feature-preview development. And how many would spend their limited time working on the plain-vanilla nuts-and-bolts stability & compatibility issues over the shiny new features?

From a management perspective, it seems like something that people might be very vocal/passionate about, but wouldn't yield enough return for the human+tech resources needed to make it happen properly.

Re: Roadmap, Preview, and Unity versioning in general - I keep abreast of the new/upcoming features and use that to inform my project implementation strategy. But as for my current working version, I only update to the last [Edit: as in final] patch of a given Unity.x release.

There are just too many moving pieces in play chasing the newest-release, and LTS is too static to support evolving needs.

 

Well, this should not be a teams decision but a company decision. If you leave it to the teams, most will not accept it...

 

Yeah, this is something being ironed out right now is re-defining the lifecycle so it's clearer to our users and also has clear guidance for internal teams to prevent the 'preview purgatory' effect.

 

Has to be at the team level because we have many teams that can't open up their repos. At the company level we can say sure, we should support this but some teams have unique requirements that prevent them from doing this.

 

We were looking at contribution numbers over the entire lifespan of public repos that were accepting pull requests. This was in discussions with the SRP and the Input System teams. Yes, we changed course midway through but even still the early phases of those repos saw little engagement. Could be there wasn't the proper support structure in place, a lack of tooling and guidance, or other factors. Now obviously how many is only one data point but what I failed to highlight in my response is the better question is how important is it for our user to contribute to the repos? That's the question that needs to be answered before we go down this path. If we see that resonates with the larger customer base, then what you're pointing out about repo preparations, guidelines, and support mechanisms is absolutely correct.

Regarding roadmaps, my question was more about the timeline and distance outwards you're looking for. Active development i.e. stuff that's being worked on now, proposed developments, etc. On the product team we've been re-evaluating how we do roadmaps and would likely apply our findings here.

Issue tracking is another internal project we're doing now so that should hopefully become much better going forward. The fragmentation is a real pain and hinders a lot of things because teams bounce between multiple source points. Fair point about communicating through the repo whether issues are completely shut off and how people should be filing bug reports. That's a good point.

To clarify here again - we're both in agreement that the first question is not the only data point, nor is it indicative of how important it is to users. The first question formed an assumption that brought us to the next question of importance. Once we know the importance or usefulness, that then informs our next steps.

 

Regarding other data points for the github repos: it's not all about us contributing to the repos; they're also a source of information for us. They allow us to check out branches, both literally and figuratively, so that we can track new features being worked on. That kind of engagement would mostly be invisible in your data.

 

Yep. So we have some insights there as far as activity goes through GitHub Insights which is helpful. Doesn't give us the qualitative data though on how helpful the repo readme's are, if the team is encouraging feedback, accepting issues, etc.

 

@smcclelland I'll be strictly technical in the following (I think your questions try to fill some metrics which overlap social, motivational and other not strictly technical issues) and will react only to recent exchange about non/contributing to public repositories (sorry about that!)

is still wrong question to ask
- unless UT solves the halting problem - which it won't

in other words you can't and won't predict *all* issues user might encounter while working with any released code
so in ideal world you would allow collaboration on publicly released code _by default_ (with the above already discussed infrastructure in place)
I personally am talking mainly about 'fixing things when they don't work' kind of issues, not necessary e.g. new/features PRs

I understand that this would be a non trivial effort if done company wide consuming time and resources which could be spent elsewhere, so I don't expect anything from this ),
but hopefully it's clear why asking that question is _technically_ completely wrong, since importance is implied automatically [as soon as you throw any code on the internet]

Point being: you can absolutely assume that it's better* to accept PRs than not -- no community research needed --
and go from there ( please do

*better meaning better for community, better for the product, and better for the internet tubes as a whole )

 

The halting problem, and everything else technically doesn't exist if the user base doesn't feel it's important or useful to contribute to our repositories. I would never assume it's better to accept PR's than to not, because resources are not infinite. There is a cost to implementing and supporting these things that take away from other initiatives. Hence, if it's not important enough to our users, it wouldn't be a priority for us to allocate resources to.

We're dancing around in circles discussing technical implementation details without addressing the critical question - do our users need the ability to open PR's against public repo's and how important is it to them against everything else?

 

I'm sure some smart people would like bug fixes they have made locally to be given back to the community

 

> do our users need the ability to open PR's against public repo's
yes, it's a technical necessity ( as explained in my first post - nothing to do with users )

> how important is it to them against everything else?
when the need arises - its'critical - since other options like fogbugz report which might need e.g. several GBs large project / or project which can't be shared / are less than practical
when everything works - is as good as non existent ( i.e. now )

I tried to _prevent_ the chicken-egg situation in the first place, but as I mentioned previously - I don't expect the whole UT will go suddenly into open collab mode for all packages

Anyway - my vote is for public PRs -

 

Pretty valuable. There's a bunch of positives;
1: I can do bugfixes instead of workarounds, and then not have to maintain my bugfix.
2: I can potentially bypass all the red tape of going through bug report -> repro case -> negotiating with QA -> bug fix. Instead, I can open a PR with a failing unit test and potentially a fix for that test. That saves everyone time.
3: The above points also makes it a lot more comfortable to opt into using a package.


The big question here is really "will the cost of allowing for PR's have a larger negative effect on the package quality than the gain from PR's". As I see it, that'll only happen if you get a lot of bad PR's, so your engineers have to spend time rejecting code instead of writing it.

 

Even if you didn't ever accept any PRs, having them available does allow users to contribute in a way that is effective for others.

As a very specific example, I recently issued a PR to the Vue project. Here's what I did:

1. Open a GitHub Issue.
2. Fork the Vue repository.
3. Fix the issue.
4. Issue a Pull Request to the mainline Vue repository.

That PR has been "accepted" but it has yet to be merged. This is because Vue@2 is only in "critical maintenance" mode at this point as the developers are focused on building Vue@3.

So what's the value in this? Anyone else who runs into the same issue as me using Vue can now point their Vue version in their npm package manifest at my PR to get the fix. One of the Vue maintainers did a cursory 5-minute review of the PR before "blessing" it before promptly leaving it in an open state, most likely to die at the feet of Vue@3.

But that's FINE because others can still refer to it and get the fix that they need. And they do. I know several users of Vue who have used my PR branch as a workaround to the Vue issue with the given platform. The Vue maintainers' involvement? Almost nothing.

Here are some differences that I see between Unity's repositories and the Vue repository:

1. Vue has official and excellent support. I expect that someone will probably look at my commits/Issues at some point.
2. Because Vue isn't in "Preview" and respects SemVer, I'm certain that it has a solid base that I can build on. Any Vue@2 version going forward will benefit from my fix and most likely won't change the guts so hard that it nullifies the work that I'm doing.
3. There are very clear instructions on how to set up my system to properly work with Vue. Even if the maintainers never looked at my contribution, I would know how to get a development environment up and running lickity-split. Having a working environment is the basis for actually working on a potential bugfix in any capacity.

So let's assume that eventually you want Unity features (e.g. the InputSystem and SRP) to get to some sort of stable point. If I as a user encounter an issue and can't wait for you guys to fix it, then I can access the code, fix it, and then possibly even share it. By doing so, I'm not only helping other developers who encounter the same issue, but I'm also offloading any ongoing maintenance for the bugfix to the project maintainers - those who should be handling it in the first place. This is my way to grease the proverbial wheels and help get the bugfixes that I need into the system.

During those times, were the projects still in "Preview"? Did they share the goals of the project? How fast was the code base being iterated upon?

Typically you are only likely to see contribution from the community once there is some sort of stability to the code base. My understanding is that neither of those packages escaped the Preview quagmire until recently, which suggests that things were in a prototype-y phase and most "users" were likely just "trying out" the technology rather than trying to fully rely upon it for a product they wanted to release.

I hope this is helpful.

 

@Baste @SonicBloomEric yes, more or less - thanks for clarifying (especially the bugreporting part)
though this whole packages situation is more specific since the whole issue and its solution would have to fall under a specific package and not under closed parts which in current packages evolving state would be probably more rare than not
so it makes sense to evaluate this per package/case, but
@smcclelland please please consider it in the future when things are more stable: the ability to fix things locally and make them available without going through full fogbugz cycle is (intergalactic) miles better

 

You're aware that we're discussing an issue (wether yes or no accept user submitted bugfixes) that has been successfully solved by a competitor. Why isn't Unity able to do the same?

The engines aren't that different complexity wise. If the competitor has the resources to successfully work with their users, Unity should be able to do it also.

 

Honestly, I'm starting to think that there needs to be a fundamental shift in how Unity is structured across the board, because I'm seeing a lot more "we can't do X because it'd be really inconvenient for Y reasons" when X is something that should be standard operating procedure.

 

https://issuetracker.unity3d.com/issues/ads-older-ads-package-is-available-in-package-manager

This issue nicely highlights that packages are not leaner at all in practice when it comes to updates.

 

About GitHub, my personal experience:

- The PPv2 source code being on GitHub was immensely helpful for the project I'm working on. As discussed on this thread there is a significant CPU overhead caused by the way the PPv2 was coded. With the help from GitHub's network graph, I was able to find forks that made their own changes and compare them against Unity's, then proceed to make my own changes, which are now available for others if they want.

- The Scriptable Build Pipeline source code not being on GitHub has made me not get anywhere near it. With a documentation that is mostly a list of uncommented methods, one liner change logs, and a dearth of community tutorials involving it, I couldn't trust such seemingly untested code with such a fundamental part of a game like creating the asset bundles and ensuring we don't have needlessly gigantic patches, missing asset references, missing shader variants, and mistakenly stripped code. It only started getting code coverage when people started to use Addressables and break things.

BTW, in the Unreal camp most PRs are rejected or stay in limbo for months. They are still useful because you can apply them to your own forks, even if they never get accepted into the engine. Some community-made features actually existed solely as continuously maintained PRs.

 

The thing that boggles my mind is why this specific bug is still not fixed. Admittedly I don't know how it works internally, but it seems like it should only take changing the default or verified version number somewhere to fix this. Yet here we are at the 7th update to the official Tech release of 2019.3 and that bug is still a known issue.

Speaking of known issues, look at how big that list is getting! My eyes lost track of which item I was on while scrolling (because they don't all fit on a single screen!), but I counted over 40 known issues in 2019.3.7f1. That's up from 11 when 2019.3.0f6 was released.

 

The beta program is working!