To Linq or not to Linq

I've been exploring Linq recently and it is pretty cool how compact I can make my code with it.

That being said I read something about it being potentially dangerous if you don't understand it fully, and I certainly don't.

I'm curious whether I should continue down the Linq path and use it wherever I can, or if it is better to just use the standard iteration techniques I am more familiar with.

 

As far as I know, Unity deploy in .NET 2.0, which does not have LINQ.

 

Unity doesn't actually use .NET at all. Instead, it uses Mono - which is an open source reimplementation of the .NET framework. Because of that, it's not possible to directly compare Mono versions with .NET versions. However, I believe the closest comparison (assuming Unity 4.3) would be to .NET 3.5, with some features of .NET 4.0 thrown in.

Regarding your usage of LINQ... I wouldn't consider it dangerous per-say, though it's likely slower than other mechanisms that can accomplish the same thing. So, I'd recommend that you be conscious of where and how you use it - and only do so in places where it makes sense. I certainly wouldn't use it in performance-critical portions of code.

Jeff

 

LightStriker I am already using Linq in Unity so it definitely is supported, although I have read some features are not fully implemented on all platforms, like the sorting options on iOS.

 

Ok makes sense, I will have to reevaluate my usage of it somewhat as I was initially using it to pull items from very large lists.

 

LightStriker, .NET 3.5 is a superset of .NET 2.0. It is still considered base .NET 2.0 but it adds features on top of it. The version of Mono that Unity 3.5+ uses is equivalent to .NET 3.5, but with a few things stripped out that Unity doesn't support (such as System.Data).

That being said, LINQ works just fine. However, if you're planning on targeting iOS which uses Full AOT, you need to be aware that there are some limitations that will cause LINQ to break with AOT errors. This is because of the way LINQ does some things dynamically and that it uses an expression builder for.

 

Linq is awesome, will make you more productive, reduce maintenance costs, and increase your productivity over time as you start seeing problems as list comprehensions rather than loops and solve more problems using less code that directly expresses intent instead of having intent hidden by layers of imperative bookkeeping.

However, Linq allocates like crazy, which is pretty much the worst thing you can do in Unity due to the non-generational-and-generally-horrible garbage collector. This is the reason people avoid it, not because of any lack of understanding.

On that note, I've just released Smooth.Slinq, which provides a faster-than-Linq-and-allocation-free enumeration / comprehension API for Unity.

Asset Store Link

Asset Forums Announcement

 

I use Linq all of the time in editor scripts and this works fine on both Windows and OS X.

Like Smooth P mentioned, Linq does allocate like crazy... this isn't always a bad thing, but I would certainly avoid usage of Linq within your "Update" functions.

I do not know how true this next point is, but a while back I was led to believe that not all Linq functionality worked properly on certain mobile devices. I am uncertain on this, but it might be worth looking out for.

 

I'm still too old school. I've never had need for Linq in games, but I would like to know more why I would need Linq? what's the use cases?

 

It just makes your code cleaner, that's all really:

Code (csharp):

1.  
2. // The following array contains a collection of numbers:
3. var someNumbers = GetRandomCollectionOfNumbers();
4.  
5. // Now, using Linq you can get all unique even numbers and sort ascending with the following:
6. var uniqueEvenSorted = someNumbers
7.     .Where(n => n % 2 == 0)
8.     .Distinct()
9.     .OrderBy(n => n)
10.     .ToArray();
11.  

You do not need to use ToArray, you could use ToList, or even just accept the IEnumerable<int> result!

Linq doesn't work in UnityScript though, you need to be using C# for this!

 

Increased productivity, reduced maintenance, faster time to market.

Basically, get more done in less time with less code that declaratively expresses intent rather than being mired in imperative logic.

 

I'm assuming that by "allocates like crazy", you mean creates objects on the heap quite a bit. Which it does... relatively speaking.

But compare some of those actions to the amount of memory that would have to be allocated in a none linq setting. And you'll have comparable amount of memory usage. Sometimes lower for super simple linq statement, often times larger for things like sorting.

For example:

non-linq

Code (csharp):

1.  
2. foreach (var obj in someList)
3. {
4.     if(obj.SomeBooleanTest) //do stuff
5. }
6.  

linq

Code (csharp):

1.  
2. foreach(var obj in from o in someList where o.SomeBooleanTest select o)
3. {
4.     //do stuff
5. }
6.  

Sure the non-linq will be less, there's no extra anonymous enumerable being created in memory.

But lets say that list needed to be sorted, made distinct, like in numberkruncher's example.

Code (csharp):

1.  
2. // The following array contains a collection of numbers:
3. var someNumbers = GetRandomCollectionOfNumbers();
4.  
5. // Now, using Linq you can get all unique even numbers and sort ascending with the following:
6. var uniqueEvenSorted = someNumbers
7.     .Where(n => n % 2 == 0)
8.     .Distinct()
9.     .OrderBy(n => n);
10. //note I removed the ToArray because that processes the entire list immediately, I often don't need it as an array
11. //actually I use arrays very rarely in my code... when I do they're mostly static in size and don't change much if at all after construction... such as field members for a component that the inspector needs.
12.  

(I don't even want to post the alternative non-linq code, it'd be so damn messy)

Well, that'll require a bit more objects in created on the heap. Both in the linq and non-linq methods. Thing is... how efficient will the average programmers sort technique be? They'll probably end up using the built in sorting algorithms supplied by .Net (just like linq will use), but done with arrays most likely. And will probably need copies of the arrays made as well and so on. You'll be creating a lot of objects, taking up lots of memory, and having to manage all those objects.

Allocating like crazy...

And you have to do all the work yourself.

And then comes the last thing, which is the main reason I like Linq so much. If you do this... I bet you'll probably write the code that does this all up front. Processing the list, sorting it, etc, all BEFORE you go and actually loop over it. Requiring a bunch of up front processing time.

The linq alternative, it processes as you need it. Say I loop through some linq statement, and I stop mid-way. It stops... it doesn't process anymore, it doesn't allocate anymore, it stops with me. I sort, and process half way through the list, it has no need to sort the back end of the list... so it doesn't.

Sure all this functionality could be done, by hand, but will require a whole bunch of leg work, and is hardly readable. Causing you to create your own linq like library... like Smooth P did (which may or may not over come of the flaws in linq, and may or may not introduce its own flaws).

That last bit though, the processing of the list as I need it. That's the part that sells me the most.

I also use linq in Update methods all over the place and barely see a performance hit relative to other options... but my linq statement also are in the realm of things that other options would be just as costly. And if the performance hit is so bad, but can be pre calculated on say load, I would do so instead.

 

Interesting, I didn't realize that, thanks!

 

Note it hinges on the sort algorithm, which means where you stop it may have sorted further past it.

Watch this visualization of quicksort (the algorithm used by linq OrderBy):

https://www.youtube.com/watch?v=m1PS8IR6Td0

You'll notice how it sorts over the collection and partitions it into groups, and then sorts each partition. When you break into a new partition, is when more sorting will be done. Say you loop up to where it's at at 10 seconds, all that unsorted stuff afterward won't ever be touched again. It's been touched once, in a cursory way to partition out the list, but it won't do the individual heavy lifting necessary to get it in perfect order.

 

Thanks for the explanations. It's still kind of not typical in games and seems to be lower performance. I can see it being useful for editor scripts, or perhaps an RPG though. For action games, or games in general, I'm just not seeing these database style queries being a super workflow improvement over simple code. But that's just my inexperience with Linq talking, no doubt. Granted, sorting the list seems easy. But sorting in realtime is the sort of thing you want optimised.

 

Oh another thing I really like with linq is the 'from from from' statements.

2 froms translates to non-linq relatively simple. But 3 froms translates to non-linq really messy. I had to write this one recently... so much nicer in linq:

Code (csharp):

1.  
2. var slips = (from s in _allSlips from bId in customer.BoatsOwnedByCustomer from b in s.BoatsInSlip where bId == b.Id select s).Distinct();
3.  

I had a collection of the boat ids that a customer owns, I had a collection of all the slips in the marina, and those slips have a collection of the boats in it (the slip could be linear and contain multiple boats). I had to find the slips that contain a boat that is owned by the customer who can own multiple boats.

Rewrite that as a for loop...

Code (csharp):

1.  
2. List<SlipInfo> slips = new List<SlipInfo>();
3.  
4. foreach(var slp in _allSlips)
5. {
6.     foreach(var bId in customer.BoatsOwnedByCustomer)
7.     {
8.         bool bAdded = false;
9.         foreach(var b in slp.BoatsInSlip)
10.         {
11.             if(b.Id == bId)
12.             {
13.                 slips.Add(slp);
14.                 bAdded = true;
15.                 break;
16.             }
17.         }
18.        
19.         //if we added this slip, break so we go to next slip and remain distinct
20.         //note this assumes allSlips is distinct already
21.         if(bAdded) break;
22.     }
23. }
24.  

More readable in my opinion. Which has been brought up.

Note my code isn't efficient, it could be made more so with out linq... but becomes even more unreadable.

This though is an example where linq is going to allocate more memory and take more processing to enumerate the entire list. But it's a situation where I don't do it frequently as well.

 

I would very much disagree!

Here's some examples of my linq in my current game, linq I'd hate to write not in it, and operates very efficiently:

Code (csharp):

1.  
2.         private void OnEarlyUpdate(object sender, System.EventArgs e)
3.         {
4.             foreach (var go in from m in ApocMovementMotor.LiveObjects where m.gameObject.activeInHierarchy  m.IsEnabled()  !m.HasTag(Constants.TAG_PLAYER) select m.gameObject)
5.             {
6.                 if (go.transform.position.y < TERMINAL_DEPTH) go.Kill();
7.                 ...
8.  

Code (csharp):

1.  
2.             ...
3.             var prior = GameObject.FindObjectsOfType<GameObject>();
4.             yield return asyncOp;
5.  
6.             var objects = new GameObjectCollection((from o in Object.FindObjectsOfType<GameObject>().Except(prior) select o.transform.root.gameObject).Distinct());
7.             ...
8.  

Code (csharp):

1.  
2.     void IMovementStyle.Activate()
3.     {
4.         _objectToLayerCache = new Dictionary<GameObject, int>();
5.  
6.         //GetAllChildrenAndSelf is an extension method with some deeper linq going on
7.         foreach (var go in from t in this.gameObject.GetAllChildrenAndSelf() select t.gameObject)
8.         {
9.             if (go.IntersectsLayerMask(Constants.MASK_MOBILELAYERS))
10.             {
11.                 _objectToLayerCache[go] = go.layer;
12.                 go.ChangeLayer(Constants.LAYER_PROJECTILE, false);
13.             }
14.             else if (go.IntersectsLayerMask(Constants.MASK_MOBILEHITBOXLAYERS))
15.             {
16.                 _objectToLayerCache[go] = go.layer;
17.                 go.ChangeLayer(Constants.LAYER_PROJECTILEHIT, false);
18.             }
19.         }
20.  
21.     }
22.  

This is in a platformer.

 

I suspect you're right. For an RPG where the player might have a large stash of items, you could implement the meat of a "show me all of my shoulder armor pieces sorted by level, then quality" feature with one line of code. That sort of thing is less applicable to simpler games of course.

 

My game is a networked FPS, and believe me when I say nearly every class uses Slinq. Thinking of something like LINQ / Slinq as database queries hides what it's real power is, and that's for clean, declarative, high productivity / low maintenance collection enumerations and comprehensions. In general, if you're doing stuff with collections and understand lambdas (and generics, though generic parameters can nearly always be inferred), using LINQ / Slinq will save you development time and reduce maintenance costs.

Part of the reason LINQ is often overlooked and/or misunderstood is because of Microsoft's decision to equate LINQ with SQL / "queries" and use SQL inspired names for the operations, eg: Select vs Map, and the inclusion of the "query syntax" which is less powerful than chained method syntax.

Nearly all live, modern languages are moving towards a functional style of programming, including (non-ancient versions of) C# and Java. A good collection enumeration / comprehension API isn't something you use once in a while, it's fundamental to nearly all code.

Some common things that are loops or nested loops in imperative code, but become clean one-liners in Linq / Slinq:

• Finding a specific element in a collection (First, Last, Min, Max, etc... and in Slinq these type of operations return an Option<T>, which can be considered an enumerable of size 0 or 1, and provides its own chainable enumeration API)
• Performing an action on each element of a collection (ForEach)
• Folding / Aggregating a collection (Sum, Average, Aggregate, etc)
• Doing any of these things to a filtered subset of a collection (Where, While, Take, TakeRight, Skip)
• Doing any of these things to mapped collection (Select)
• Doing any of these things to a collection of collections (SelectMany, Flatten)
• Removing particular element(s) from a collection (Remove, Slinq only)

Sorting is a complex issue that is best left to another post, but it is important to point out that you may want to sort filtered and or mapped subset(s) of the original collection, that the total cost of sorting the subsets of a partitioned collection is cheaper than sorting the entire collection, and that the OrderBy() operations in LINQ / Slinq aren't sloppy bubble sort code thrown together by a "how to make game" forum poster...

 

I'm going to go out on a limb, and say for most people using LINQ isn't going to be the main performance bottleneck, or even close to the top of the list. And I think code that is very clean and clear is worth its weight in gold.

 

I love LINQ and can't live without it anymore. Unity's broken GC makes it slightly worse in Unity than it is in the real world, but like jc said, I doubt it will show up as a performance bottleneck for most people. You can also avoid some of the allocations by compiling first in Visual Studio with the Microsoft compiler and then importing the DLL into Unity, because Unity's compiler is also broken.

 

I've really never been a big fan of linq. I don't think that linq itself is the performance problem, but I do think that people start to neglect little things since linq makes so many things so easy.

I think it's because you don't notice that you're repeating that same one liner all over the place nearly as much as you would if you had to write the loop out. So some of those little incremental improvements or optimizations are just never made. It's also a bit harder to optimize than you might think, since often it's not so much a single bad usage, it's pervasive usage: select's happening literally everywhere is one common example, repeated filters on a similar field instead of just using a dictionary is another one.

 

That's exactly what I like about it, and exactly what a high-level language like C# should be all about. Devs who enjoy sweating about the little things and making their own lives difficult have C++.

 

The fact it has garbage turns me off at the gate though. If Unity can sort out the GC so its virtually painless then count me in. Otherwise I'd rather not (in a large game with lots of cross references) run into hitches. If anything, Linq is saying "You can be more productive, if Unity will fix the GC"

I have used middleware that uses Linq, and the performance was pretty bad with collections happening quite frequently. Probably not a problem on high powered desktops.

 

And thus, Smooth.Slinq.

More methods than LINQ, faster than LINQ, and no per-frame allocations.

 

Question, does Slinq's method names mirror Linq names? So that all you have to do is swap out the reference between the two?

 

The Slinq documentation is available online:

• Smooth.Slinq
• Smooh.Base (Shared classes such as Option<T>, Tuple<T1, ..., Tn>, etc)

The main user-facing classes for Slinq are in the Smooth.Slinq namespace, namely:

• Slinq<T, C> - The main Slinq class, analogous to an IEnumerator, and is generally what is meant by a "Slinq".
• Slinq - Static class with (extension) methods for creating chained Slinqs along with stuff like Average(), Sum(), etc.
• Slinqable - Static class with (extension) methods for creating Slinqs from other types like IList<T>, LinkedList<T>, etc, as well as other Slinq creation methods.

Also important is:

• Option<T> - Like a Nullable<T> on steroids. Option<T> defines many chainable, LINQ-like methods that make it a very powerful data type.

For one-liners, the Slinq API is generally the same as LINQ's but with more methods and relaxed delegates with variant type parameters. Of course, since Slinqs are enumerators as opposed to enumerables, you have to create the Slinq.


So something like:

Code (csharp):

1.  
2. var countLinq = list.Where(x => x > 0).Count();
3. var countSlinq = list.Slinq().Where(x => x > 0).Count();
4.  

Note that one of the ways that real-world LINQ usage leads to allocations is due to closures. To avoid closures Slinq provides method signatures that take an extra, user-supplied parameter that is passed to any delegates and can be used to capture state:

Code (csharp):

1.  
2. var y = 42;
3.  
4. var countLinqClosureOverY = list.Where(x => x > y).Count();
5. var countSlinqClosureOverY = list.Slinq().Where(x => x > y).Count();
6.  
7. var countSlinqAllocationFree = list.Slinq().Where((x, p) => x > p, y).Count();
8. // y is passed to the delegate as the second parameter
9.  

A Slinq is equivalent to an IEnumerator, not an IEnumerable, so manually looped Slinq usage would look like:

Code (csharp):

1.  
2. var slinq = list.Slinq();
3. while (slinq.current.isSome) {
4.     //
5.     // slinq.current is an Option<T>
6.     //
7.     // if slinq.current.isSome is true, the Slinq has elements remaining
8.     // if slinq.current.isSome is false, the Slinq is empty
9.     //
10.  
11.     //
12.     // Do stuff with slinq.current.value, it is a field so you can access it as much as you want without method calls
13.     //
14.  
15.     var z = slinq.current.value + 1;
16.     var m = 10 * slinq.current.value;
17.  
18.     //
19.     // Move to the next element
20.     //
21.  
22.     slinq.Skip();
23.  
24.     // or, slinq.Remove(), if supported by the underlying collection
25.     // or, slinq.Dispose(), if you're done early and don't care about the remaining elements
26.     //
27.     //
28.     // There are also other partially consuming operations like:
29.     //
30.     // Skip(int)
31.     // SkipWhile(predicate)
32.     // SkipWhile(seed, aggregationFunc)
33.     //
34.     // as well as corresponding Remove() counterparts.
35.     //
36.     //
37.     // You can also use a fully-consuming operation on the remainder of the Slinq, like:
38.     //
39.     // var maxRemaining = slinq.Max();
40.     // do stuff
41.     // loop will end as slinq will now be empty
42.     //
43. }
44.  
45.  
46. // Cleaner than a where loop, with no dangling var:
47.  
48. for (var slinq = list.Slinq(); slinq.current.isSome; ) {
49.     // Do stuff
50. }
51.  

Of course, Slinq has ForEach() methods, so those can also be one-liners.

"Loop syntax" is only needed when you want to consume part of a list with a particular operation and leave the rest for other operation(s).

 

Ok, so it's not directly compatible.

 

Oh... 95 bucks... yeah... Linq is speedy enough for me.

 

The main value proposition of Slinq is in the "GC Alloc" column, not the "Time" column.

In fact, the non-constant space operations in Slinq are not optimized for speed, they are optimized for consistent, predictable, strictly linear memory use with the flexibility for "zero-copy" grouping / reordering of pooled data.

When I decided to put everything on hold and convert my ever-growing collection of enumeration / comprehension code into a full-fledged replacement for LINQ I figured I'd be perfectly content if Slinq ended up slower than LINQ to get that nice, round, non-FPS spiking, studder-free 0 in the GC column. The fact that Slinq is slightly faster than LINQ for constant space operations and potentially an order of magnitude faster for pool backed operations is a just a very, very pleasant bonus.

 

I've decided to release Slinq (and Smooth.Compare) as part of Smooth.Foundations under the MIT License.

The latest version isn't up on the asset store yet, but direct download links to the code and documentation are available on the Smooth Games forums.

 

Well.. 2 years later! Is it safe to use LINQ with Unity or Not?

 

Yes. Have you tried?

 

I'm trying it right now and I'm in love! =)

I have one small question.... FirstOrDefault... How do I define the Default Value, it will return null if not found?

 

It returns default(T) which is null for reference types and the default value for value types (ie false for boolean, 0 for numerical values, etc)

 

I'm using Linq only for .Last / .First / .ToList/ToArray and rarely to do DB-like stuff.

 

Unity Technologies recommends to avoid LINQ where performance is a concern:
https://unity3d.com/de/learn/tutori...ion/optimizing-garbage-collection-unity-games

Microsoft recommends not to use LINQ in Unity games, as it causes heavy allocations:
https://developer.microsoft.com/en-us/windows/mixed-reality/performance_recommendations_for_unity

 

Be aware that most of those problems aren't created by Linq itself, but rather the run-time delegate allocation (via lambda expressions widely used in Linq).
If something's being called every frame, make sure to cache the delegates (like you do with components), or avoid lambda expressions altogether.

TL;DR; If you get rid of Linq, but still use lambda expressions - it's going to be nearly just as bad.

 

The DOTS-code extensively uses the Lambert operator along with foreach:

https://github.com/UnityTechnologie...pts/Systems/IncreaseVelocityOverTimeSystem.cs
https://github.com/UnityTechnologie...ssets/Scripts/Systems/PaddleMovementSystem.cs
https://github.com/UnityTechnologie...g/Assets/Scripts/Systems/PlayerInputSystem.cs

 

I assume you mean "Lambda" by saying "Lambert", but anyway...

Code (csharp):

1. https://youtu.be/gyqIZP_zLtU?t=1913

(forum eats the timecode, had to paste it like that)

 

That's a little confusing.

 

I've used linq extensively inside my game, though granted never in an update loop.

Nevertheless, I've slowly had to remove it due to gc issues (`average` creates garbage, really!?), and I'll probably strip it from everywhere except outside loading screens and my games buildin map editor.

I wish more voices were critical of it, rather than having to learn about it the hard way.

 

Old post, I know, I've same problem, people think like LINQ is just easy , which it is, but GC is terrible, its not noticable on PC, thats their execuse, and its awful one

 

It's not black and white. There is no right or wrong answer. Not all LINQ methods are created equal. Optimize for your target platform accordingly. I'm currently working on a project targeting Oculus Quest 2. I'm writing a fairly sophisticated rules engine that supports a lot of permutations and I'm doing it all old school because I'm targeting essentially a mobile platform.

 

That's actually spot on. It really depends on the usecase. Things which are done once when a button is clicked most likely don't matter. However if something needs to be done several times per frame it's usually not acceptable anymore. Though as you said, it may depend on the targetplatform.

Just as an example: When you create a quiz game with 4 choices and you use Linq to scramble the choices or doing some awkward inefficient lookup when an answer is selected, it literally doesn't matter. We can run for loops that iterate through millions of vertices or pixels every frame, so a list or array with 4 elements doesn't really matter.

I most the time avoid Linq in general because when using more complex things I'm often times not sure if it actually does the right thing. So one may spend some time checking that a certain function / combination works as expected. Of course if you work with Linq all the time you get more competent with it. Though I like to easily see what a certain piece of code does. Yes, we can look into the Linq with ILSpy but that's not very convenient.

I generally follow the philosophy: If I don't understand it I probably shouldn't use it ^^. (That's why I've just written my own DNS server from scratch in C# and have it running on my pi with mono . No, really, it's a rabbit hole, especially watching all the strange requests it accumulates).