Unity should be Open Source

Hi,

I use Unity since two years and release some games with it. The engine is easy to use and you can create games and you do not need all skills of a game developer.
But the problem is: The Unity developers starts new projects (like HDRP, URP, Network Gaming, Entity System, Job System, ...) and do not finish theme! On top the old state of the art solutions will be deprecated and not supported anymore but the new solutions are very buggy and have a lot of technical problems.
So I believe there are too many projects for the Unity company and it should be better to make Unity completely Open Source to give the community a chance to make Unity one of the best Game Engines in the world.
Well, Unity is free for starters and all other licenses must check individual. So in this context it is not the point of interest that unity is a commercial product.
There are so much things I want to change via a Merge Request:

• Package Manager Asset Store Solution
• Play Scene with Base Scene
• World System
• Scene Global Components
• Materials depend on target platforms
  
• ...

So I think a lot of users of Unity from the community would be made it better and let Unity grown up.

Maybe, dreams come true .

Best regards!

 

The problem with Unity right now is that it is too much like an open-source project, with all sorts making an arrival via the package manager. I don't see how actually making it open source would help anyone, quite the contrary.

 

Going open source will slow down things further.

Because hallmark of opensource approach is lack of general direction, starting many things and not finishing them.

 

Does Unreal move slower than Unity?

edit: I do not advocate Unity going open source. We dont have time to dig into their source anyway. But its still a interesting topic.

 

Unreal it's not an has never been open source.
It is a propietary engine and if you tried to modify it and redistribute it, you would get a big fat lawsuit coming your way.

 

Unreal is proprietary. And not Open Source.

 

True, didn't read the inital post good enough, most question like these wants equivalent to unreal were source code is available. But here they want more real open source like godot.

I'm a big advocate for open source, there are alot project that have very clear goals and great management.

Just look at.NET core, graphana etc

 

But then there are tons of them that have fallen by the wayside.

 

Thats also true. At my dayjob almost every SDK we use are open source so probably more success stories found in the corporate world

 

Not having (free or affordable) access to the C++ part of Unity source is, by far, Unity's biggest pain in the ass for me compared to Unreal and the reason I will never willingly make any complex project using unity and strongly discourage people from doing so in every opportunity I get.

Whenever we get a hard crash, specially in built players, that originates from Unity code and not from our own code, it feels like having to disarm a bomb in complete darkness. We have to randomly poke and prod it, blindly trying to figure out which of our previous actions got Unity offended and what can we do to appease its anger. A process that can last hours, days or weeks, and could be greatly simplified if we could simply look at the crash site source while debugging.

The same happens when trying to optimize. There are some notoriously opaque systems in Unity, like the terrain, which do a lot of things in a very specific and undocumented way, and require serious trial and error to optimize.

I do get it: from the IPO financial reports we learned the bulk of Unity's income comes from large clients who purchase source licenses. Also, making the source fully available would allow people to simply compile out license checks.

But could we at least get some of the C++ source available for reference and debugging purposes, like Unreal does for Launcher builds (which cannot be used to rebuild the engine)?

 

You know, having source code access does not exactly mean you'll be able to solve the crash, though. Or that you'll even have easier time resolving the problem. Something worth keeping in mind.

The reason for that is high complexity of game engines

 

Of course, not being able to actually re-compile does limit what can be done, but being able to see what the darn thing is doing to my inputs does help immensely. This patronizing "Unity knows best, you have no need to look behind that curtain" attitude is exactly the kind problem I'm talking about.

Let me give you real world examples:

1. We have been using SphereCastCommands with good results for a while, and decided to optimize the creation of the command lists and the processing of the results using jobs. Performance gains were significant, but we now get crashes on systems with 4 cores or less. The reason? Somewhere inside the SphereCastCommand batching system there is what we believe to be a fixed size buffer that limits the number of in-flight commands per job thread, which increased as a result of our optimization.
   
   This is not documented anywhere and a quick glance at the source would cause us to promptly re-factor our system to respect this limitation. Instead, we spent two days running around doing CSI work to guess the cause of the problem.
   
   
2. We wrote a custom grass rendering system using DrawInstancedProceduralIndirect. Worked perfectly on PC, PS4 and Switch, but outright hard crashes on Xbox One. Crash site is inside the XB graphics libraries, callstack prior is full Unity code. This one took weeks of intermittent work to fix. There are other uses of procedural indirect drawing in the game, so it's obviously supported. We tried all sorts of permutations on the shader, cutting it down to the bare minimum, to no avail.
   
   I had to painstakingly read through the disassembly in Visual Studio, which fortunately retained function names and allowed me to place a breakpoint somewhere earlier in the crash call stack. Groking though the dissasembly, plus the few named variables that could actually be seen and keeping track of register and memory addresses, and comparing it with the execution of the other procedural drawing that didn't crash, we figured out that the crash was caused by Unity telling Direct3D on Xbox to use the non-procedural draw path, which caused it to try to access the vertex input layout, which was null. We then just re-factored the procedural call into using a pre-generated mesh, since without the source it was impossible to find out why the draw call was not using the procedural path.

In both cases the time to figure out what was going on would be significantly shortened by being able to see the source code, even if we couldn't be able to modify it. There's no such thing as too much transparency. Might as well give up the reference C# source, the built-in shader sources, and the packages' sources as you are at it. Those have made a heck of a difference in my workflow.

There are several parts of Unity which are absurdly opaque and do not have their implementation details documented, which would be much easier to work with if we could at least look at the source code.

 

And huge mess of a code base

 

This is an excellent example of why Unity will never be released as open source. Most if not all of it's middle-ware is behind non-disclosure agreements as is all code related to console platforms.

I know what you're going to say though. You're going to mention that Unreal 4's source code is available but the difference between the two engines is that Unreal 4's middle-ware is implemented entirely through plugins while Unity's middle-ware is built directly into the engine.

There are only two ways Unity's source code would be able to be open source. One, Unity would need to refactor all of its middle-ware implemented with a plugin approach. Two, all of the middle-ware would have to become open source. Neither of these is likely to ever happen.

 

The platform support in UE4 are not plugins. I do have access to them all: they require you to unzip a folder full of source files on top of the engine source code and re-compile the whole thing. Meanwhile, all the NDA'd platform support in Unity are actual plugins. It's actually a better implementation of multiple platform support: you cannot build for consoles on UE4 with the "launcher" version. In both engines you only get access to them after confirming your licensed developer status.

Also, for simple reference and debugging purposes, Unity doesn't actually need to make NDA'd source available.

But you have a point in that Unity was way too happy to simply shop around for stuff like Umbra and Enlighteen (R.I.P.) until recently, and there might be stuff there that could require going through some red-tape even if their headers are not included.

 

Yes.

It is not "patronizing" or "Unity knows Best", it is speaking from experience with Unreal engine where source code access is provided.

The main issue is that the engine is a Behemoth. It is a 300 megabyte body of program text. When you hit a crash, you do not always get a simple "oh, there's a little bug here, you can fix it, if you change this parameter!", but very often you get something that involves multiple threads, is spread through multiple components, and is for practical purposes, operating in ancient sumerian, using alien logic, and someone spent 10 years writing it.

Basically, while you can peek under the hood, below the hood, there will bee a swarm of writhing eldritch horrors and no documentation.

For example, off the top of my head, stuff I experienced:

If you create a landscape in currently open scene through C++ code, the editor will crash once it is done recompiling shaders with 75% certainty. The crash will lead to some sort of sanity check akin to "if shader.dirty == 0", which won't help you pinpoint a problem. To get a rough idea of what is going on, you'd need to comb through roughly half of the editor, as there's no documentation and this is a multithreaded problem. Likewise, if it doesn't crash, chances are grass layer won't appear until you poke landscape with a brush at random point, which implies that there's some sort of "dirty" flag... somewhere.

Or... trying to rebuild skin mesh from code past certain version will lead to a hard editor crash once the editor attempts to render said mesh for the first time. Investigating would lead you to certain render-related structure that has to be initialized for all meshes, but isn't being initialized for your mesh specifically, and there does not appear any api function for initializing it.

Or... this is actually one of the easier ones. For example, when importing pngs, Unreal 4 at least in some versions sets RGB data to zero in transparent regions (A == 0). You'd expect there to be a switch for disabling this, but it does not exist, this is hardcoded. This is not what you want to do if you're storing information in transparent region, but hey, at least in this case you can write your own texture factory, as the stuff which deals with direct png access is just a few levels deep within the original one.

Stuff like that.

Yeah, source code helps. No, it does not mean you'll solve the problem.

 

But it at least makes it possible to fix the problem at all. Or at the very least make a more informed workaround if it happens somewhere too cryptic to touch.

With Unity the only option is to report a bug, upgrade your project, pray for the bug to be gone in the latest version, and for no new crippling bugs to come along. It's the great gamedev equalizer: it doesn't matter how much coding talent you have available, once you hit the closed source wall you're as stuck as everyone else (unless you have enough budget to pay tens of thousands for a source license).

 

I believe that is untrue.

Having source code access does not mean that the problem can be fixed. And not having source code access does not mean the problem cannot be fixed. If you can sidestep it, then you can sidestep it without source. What's more, having source code access does not even necessarily mean you'll have easier time fixing it, because rather than poking it with a stick and figuring out what makes it die, you'll spend time trying to fathom mysteries of alien logic used in the problematic spot, which can take longer.

Working with closed source platform is a very common thing, and when developing for windows, you CAN end up in a situation where you have a stack trace which ends up deep in the guts of ntdll. In this case you do not usually bemoan lack of source code access to ntdll, but look for way to figure out what happens and sidestep it. Because windows is even a bigger Leviathan than Unreal Engine (300 gigabytes repository, as they say), and you will not have time to do someone else' job trying to fix it.

Likewise, if working on linux, you somehow hit a kernel level issue, if you aren't already kernel developer, you won't be fixing that. Because it takes prohibitive amount of time, and won't be applied to every system you deploy on. So you won't be diving into source, and instead will sidestep it.

That makes me wonder if you really need source code in the first place. Basically, the "under the hood" stuff does not have polish of the outer API, and is in "proceed at your own risk" category. Or even in "Abandon all hope" category.

It also makes me remember time of Accolade which reverse engineered a console to make games without acquiring proper license. Or when I, myself was writing a plugin for a closed source engine which implemented faux post-processing effect which the engine did not officially support.

In the end most likely scenario is that source code access matters when the source code is designed as a product, but in reality "under the hood" stuff is eldritch horrors, and is messy.

Also regarding this:

your other option is to replace faulty system with your own. That will be more efficient than praying, in my opinion.

I would advise to study techniques for debugging without source code access. Decompiling and disassembly are useful tools, same goes for tracing system calls, and so on, so it makes sense to add more items to your toolbox.

Rather than "I need source to fix it!", I think "How can I diagnose and sidestep it if source code is not available" is a more useful approach.

 

I'm speaking from experience too. I use UE4. I've worked with it's source since version 4.2. Modified a damn lot of it, did tons of feature back-porting, console SDK upgrades, and "we need to ship ASAP" hacks, all impossible in vanilla Unity. Between "eldritch horrors" and broken black boxes, I'll take the horrors 100% of the time, thank you very much.

Just because it's "possible" to debug things without source doesn't make it nice.

Replace the faulty systems... sounds great, I would love to replace Unity with something else, but this isn't my call, unfortunately. Enlighten, how should I replace the asset bundle async loading system by one that doesn't hitches when it randomly decides to load stuff synchronously mid streaming? Or the shader compiler itself? Maybe it's time to go nuclear and do all rendering via native plugins now?

And you said you were not patronizing. People will indeed defend anything on forums, damn. I should posting because if I continue I'll get myself banned.

 

Frankly the middle path they have chosen seems like the worst of both worlds, like their development speed has already slowed to a crawl, since all the stuff they really care about are packages, but also there aren't nearly enough open stuff that I could potentially decide to fix thing X that has bothered me for years if I really wanted to.

 

I'm not trying to defend anything, I'm merely voicing my opinion, in hopes, that maybe, one day, it will be useful to someone, and will prevent them from wasting inordinate amount of time on their gamedev journey. If that makes you believe that I'm "defending" something, then you are projecting product of your imagination onto my actions, make assumptions which are incorrect, and that, as far as I'm aware, is not my fault.

Now please explain why the hell should I put up with this kind of thing.

The way I see it, you're having a bad day. And in a situation when you're having a bad day, rather than trying to vent on forums,. you should fix your mood.

Have a nice day.

 

Great, but that just makes you the exception to the rule that most developers won't be able to make use of the source code, and it's working with the assumption that having access to the Unity source will enable you in the same ways that having the Unreal source enables you.

Have you looked into the new system built for exactly that? Includes full source code.

https://docs.unity3d.com/Packages/com.unity.addressables@1.16/manual/index.html

 

I mean isn't a game engine supposed to facilitate all sorts of edge cases because it's supposed to help me create the game I want to make?

The "most people won't use X" mentality quickly leads to software being a one trick pony, and if we take that to it's logical conclusion and extreme for Unity, then Unity should be a one button software, that once you press it, it buys a bunch of assets from the asset store and slaps them together to create a cartoony looking free 2 play ad supported monetisation hell type game that it then uploads to Google Play.

 

I'm fine with source code coming out if it has a positive effect but I have no faith that the source is in a reasonable state that wouldn't cost a lot of time and money to be brought to that point. Unity is already struggling to adapt to and with the competition. You need to weigh the ups and downs to making this transition.

 

Yeah, fair enough, I think I agree.

 

No, it is supposed to facilitate most common cases instead, so you won't need to spend six months reinventing the wheel and writing your own animation system.

Edge cases is something you implement yourself on top of existing framework.

And trying to make anything possible would lead to higher bloat. As flexibility costs resources.

 

I agree whats being said. For 95% of all indie source code does not matter. We simply dont have the time to invest in trying to find bugs in the source. Its better invested in creating a solid repro

edit: scratch my number above. 95% of all indies that use unity are noobs,. of the 5% pros, 95% do not benefit from source code

 

That mentality is why Timeline and Animator are not robust enough and if I just sneeze at them in the wrong direction, things fall apart, fast, which leads to having to write my own systems and reinventing the wheel.

 

The engine as it is, with all its flaws already saved you couple of years of development time.

Also, it is not "mentality", but common sense.
Lastly, unity is business and there's not much reason for them to implement edge cases, if nobody is paying for that.

P.S. What the hell is with all the negativity on this forum lately?

 

Demonstrably false.

Which makes me wonder, why am I paying them?

 

It is demonstrably true.

Before I tried unity, I spent a lot of time studying related programming material. Physics, graphics, shaders, export plugins, skeletal animation, shading algorithms. Everything.

And I did learn all of this. But.

Then I discovered unity, studied it for something like two months, and after two months, as a test wrote a simple basic combat 3d game prototype with simple model.... and I was able to pull it off in a DAY.

That was one of the cases where life presents you an unpleasant lesson you can't do a thing about. "AN IMPORTANT LESSON', as Terri Pratchett's Death said.

So.

By using unity engine, you save from 3 to 7 years of development time necessary to learn necessary know-how, related algorithms, write everything yourself, and arrive at inferior product compared to unity, which you'll then need to babysit.

For me, that is, pretty much, indisputable. And the way I see it, if you insist that unity doesn't save you any time, then you are letting your emotion get better of you, and aren't seeing the situation clearly.

You're paying them for that. For the time you would've wasted implementing every single system by yourself, then by testing it on Windows, Linux, MacOS, Android, and bazzillion different platforms and technologies, ensuring everythign works well enough and look identical which would take ridiculous amount of time.

If you are not happy with what you're getting, however, switch to another product.

 

Yes, we all know that Unity is fast at prototyping, which is why people are confused and think it also saves them time when releasing games.

Please don’t tell me what Unity has or hasn’t done for me and don’t mistake your personal experiences as universal.

 

Another person in bad mood?

This is unfortunate.

 

I have looked, but have you? Because the actual building of asset bundles and discovery of asset dependencies is done inside Unity's black box through the ContentPipeline API. Addressables is nothing more than yet another asset bundle manager, it fixes none of its fundamental issues.

And again, several developers do have access to Unity's source code. Any successful Unity game that looks reasonably impressive is most likely using source code modifications. You literally cannot make a game that looks and run like Ori does on Switch or Call of Duty on mobile without source code access.

I can't see how making something that is currently available for very few people available for more people is a bad thing. Or how writing a terrain+folieage+trees system from scratch is somehow less laborious than trying to modify the existing one.

But again, people will defend and downplay anything on these forums. Yes, I am in a bad mood because of Unity and that doesn't invalidate any of my points. If the food tastes bad, I will complain.

 

On the contrary I'm focused solely on the negatives because you're focused solely on the posiitves. Someone has to balance out the suggestion by providing reasons for why it's not a sound approach just like you've stated why it is a good idea.

 

Unity is nothing close to open source(or Unreal, source-available), there's merely few packages on GitHub (like https://github.com/Unity-Technologies/Graphics). If you want to contribute to other ones you have to write essays on the forums. There's a good chance that no one from Unity will read them.

These work pretty good for me: https://github.com/Cysharp/UniTask, https://github.com/Facepunch/Facepunch.Steamworks, https://github.com/vis2k/Mirror. Much more stable and finished than any other package from Unity.

 

Didn’t know about UniTask, seems pretty great, thanks!

 

Check out other packages from @neuecc , probably the most advanced wizard of open source C#. For ex with this one https://github.com/Cysharp/ZString you can create and combine strings without allocations by setting them directly to TextMeshPro using TMP_Text.SetText() extension that uses TMP_Text.SetCharArray().

Imo if Unity will ever have something similar to Epic Grants this should be the 1st person to receive it together with @vis2k

Unfortunately Unity rather spends money buying Vivox, Multiplay, Artomatix etc that makes no difference to most of us.

 

Having worked with the Unity source before, it can be fairly time intensive and presents its own set of challenges. The biggest one being merging with official updates and changes. If you make changes that get deep into the player portion, and they do updates for platforms or such that are mission critical to deployment, you have to merge. Sometimes that is a breeze, other times... not. Our changes, at the time, were related to networking and rendering and constantly had to be brought forward with engine updates when available. And we had 2-3 folks dedicated to maintain our engine integration. We didn't do that again.

 

The most basic rule in open/available source is to keep your changes minimal or make sure they are merged to official repo. It's a simple deal of we give you the source you give us your changes.

I guess here where companies paid to get source rather say: we bought the source so we don't give you our changes. With how fast Unity is being developed these companies are rather expected to make their completely custom forks.

Maybe I'm wrong but to me it seems like relationship between Unreal and developers is a lot better than what we have here.

 

Second part is not true

 

We didn't get source to "fix" the engine, we didn't bother with any of that stuff, if we had problem/bug we would let Unity know, and they would address, our bug reports were given high priority. (plus we had unity staff onsite all the time) But the reason we did the source agreement to was to support a different platform and add low level features that weren't there already. I don't recall any of our work was ever pushed back to the engine, it just wasn't stuff that would have broad application and it is not remotely open source, I don't think we would legally been able share our code with them.

 

Not "Freely". Restrictions apply.

https://www.unrealengine.com/en-US/eula/publishing?sessionInvalidated=true

 

Their core runtime with the most crucial stuff is completely isolated from external dependencies except PhysX (I had access to it).

 

I would like it open source right now, I see allocation from empty OnCollsionStay method. And change log for Unity 2019 says it should not allocate. Also I want it open source with git read access so I can see history for changes over time on that method.