Question Where to find to the source code of .NET classes used in Unity?

A few years ago, StringBuilder weren't generating much garbage, since then there's been some changes in the framework used by Unity, and now they generate more garbage.
I've checked ConcurrentQueue, and it doesn't generate garbage in Unity.
But it does generate garbage in a Visual Studio standalone executable (meaning that I'm afraid that in the future it will generate garbage in Unity...).


I have 2 questions:
- is there a way to access the source code of .NET classes (ConcurrentQueue.cs or StringBuilder.cs for example) used by a given version of Unity?
- will Unity continue to go for more recent frameworks?


About the first question, finding some source code for .NET classes is easy (https://referencesource.microsoft.com/ for example), but these source aren't usually the ones used in Unity.
It would be nice to be able to know what is actually happening under the hood when you use a .NET class.


About the second question, here is my motivation for it:
It seems that newer frameworks are optimized for mobile by Microsoft, specifically they're trying to decrease memory pressure of the collections (at least that's the case for Stringbuilder and ConcurrentQueue).
While it's a good thing when your target is very memory limited, it has the problem of increasing garbage collection.
So when your target is standalone you have the incredible luck of having your collections releasing / reallocating memory constantly as you add / remove elements from it (which are 2 very standard features of collection), so that you may not use a few thousands precious octets.
I don't care about it for non-thread safe collections as I can rewrite them quite easily, but when it comes to thread-safe collections, especially the lock-free ones, I do not have the expertise to rewrite them from scratch.

It's a bit like if going for a newer framework was 'de-pooling', by that I mean that collections were previously doing something like pooling, and in the future they seem to pool less and less, and allocate more and more.
In the latest frameworks, when a ConcurrentQueue grows, it will put the data in chained arrays, and when you dequeue elements, the empty arrays are discarded, so when it grows again you have to re-allocate the discarded arrays.

I've seen the same thing in TextMeshPro, the underlying array containing the text will be reallocated when the text it contains is reduced, and reallocated again if the text in it expands.
The worse being to clear a text in TextMeshPro before putting a new text, here you maximize the garbage created.
Fortunately, having access to the code of TextMeshPro helps a lot because we can find out the root of the problem and correct it ourselves.


I hope that my fears are misplaced and that Unity teams will not use more recent code for ConcurrentQueue (and maybe other concurrent collections, I haven't checked them yet), and maybe have the good idea to use an older version of the StringBuilder class (one can dream...).

 

I'm not sure but this might be what you are looking for https://github.com/Unity-Technologi...tem/collections/Concurrent/ConcurrentQueue.cs

 

Thank you for this link.
It is quite interesting, but unfortunately it isn't the source code used in Unity 2012.2.18, according to the deep profiler the underlying method called when enqueuing is 'TryEnqueue()' while in this source code it's 'TryAppend()'.

In fact, this source code seem to be the same as Microsoft's one, and it does generate garbage as the 'TryRemove()' method will release a Segment when it's empty instead of pooling it to re-use it in the future.
It's a shame actually, all they have to do is to pool the released Segments, and when growing to use the pool to get an empty segment.
Well, maybe it's quite hard to do with thread-safety.


My guess is that we have to select another branch in the link you provided, but there are dozens of branches, and their names aren't always very explicit.
I would hope that someone from Unity see that thread and indicate how to find the branch corresponding to a Unity version, they surely know how to that.

For information, I have found this:
https://github.com/dotnet/msbuild/blob/main/src/MSBuildTaskHost/Concurrent/ConcurrentQueue.cs
It's probably not the correct one because according to the comments it's only a back-ported version with only a few features.

 

https://github.com/dotnet/runtime/b...tem/Collections/Concurrent/ConcurrentQueue.cs seems to match what you saw in the profiler.

But to be sure what they use, you should make a build and then decompile the mscorlib.dll in the build directory (or in il2cpp cache in Temp or Library folder if you built with that) Don't know what dll they use in the editor. Probably something from the Unity installation folder. There's multiple mscorlib.dll there

 

It's much closer, not sure it's the one but it may well be it.
The 'TryDequeueSlow()' method does release a segment, generating garbage, but maybe it's just never called in my basic test.
That said, I hope it is not the one, as it is not lockless... but maybe there's no way to avoid locks and garbage at the same time.

You're right, that's what I should do, but I hoped to find out where Unity provides its source code, it's much easier than dnSpying especially since the comments do help a lot.

 

Thank you for your answer, I'll look deeper in the Unity github to try to understand how it works, now that I'm sure that what I'm looking for is somewhere in here.


As for moving to more recent frameworks, while I'm usually very happy of having more and more recent features, I'm also afraid of losing performance with newer frameworks.
As said in my initial post, the StringBuilder class is generating more garbage since a few years, because it now release memory when you remove text from the StringBuilder, before it wasn't releasing memory except to grow (which is unavoidable).
And the newer ConcurrentQueue source code generate garbage while older implementations doesn't, so going for the new framework will increase garbage generation for every game using ConcurrentQueue.

I understand that platforms with very small memory prefer having collections which release memory when their number of elements is reduced, despite the fact that it generates garbage.
But standalone platform have no use for more collections which release memory when reduced.
With several GB of memory we don't care to lose a few thousands octets in reduced StringBuilders or collections, but we do have a problem with uncontrollable garbage collection coming from the framework itself.

So please, do not take newer code blindly, .NET hasn't been designed to create video games, Unity has, it's normal that .NET does things in a way which doesn't suits the needs of Unity.
In addition, I think that making a pass to optimize the collections in .NET (which are often at the heart of the game) would help optimize games a bit.
Preventing them to release memory of course, and perhaps also adding them some cool features.
For example, I've seen a nice increase of performance by adding a 'AddIfNotExists()' to the 'Dictionary<T>' class, much better than having to check the existence and then add the element if not present.