Question Nano Optimizations : Overhead and Allocations

Me, personally? I dont even sleep at night.

Which one is better in terms of overhead? I think the first one?

Dictionary<string, Guns> Ships;

Guns guns; // struct

GunDeck gunDeck; // struct

int decksCount;

int decksReloading;



void FixedUpdate(){

foreach (KeyValuePair<string, Guns> shipGuns in Ships)

       {

           guns = shipGuns.Value;

           if (!guns.Reloading) continue;



           decksCount = guns.GunDecks.Length;

           decksReloading = 0;



           for (int d = 0; d < decksCount; d++)

           {

               gunDeck = guns.GunDecks[d];

               if (!gunDeck.Reloading) continue;

               ...

OR

// Same global parameters as above

void FixedUpdate(){

foreach (KeyValuePair<string, Guns> shipGuns in Ships)

       {

           // Switched these two statements around

           if (!shipGuns.Reloading) continue;

           guns = shipGuns;

 

           decksCount = guns.GunDecks.Length;

           decksReloading = 0;



           for (int d = 0; d < decksCount; d++)

           {

               // Also switched these two around

               if (!guns.GunDecks[d].Reloading) continue;

               gunDeck = guns.GunDecks[d];

               ...

Since we are calling

shipGuns

(big garbage) and

guns.GunDecks[d]

(not as garbagefull) twice instead of just once as in the snippet above, this creates more overhead/allocation/garbage?
Meaning method nr1 is better?

Unity says:
To get around this, you should try to reduce the amount of frequently managed heap allocations as possible: ideally to 0 bytes per frame, or as close to zero as you can get.
(Edit: Lol nice sentence structure there)

source:
https://docs.unity3d.com/Manual/performance-garbage-collection-best-practices.html

 

if I remember correctly, value types create garbage, like:

but reference types don't, like:

if shipGuns is a cached class.

So if I understood the explanation I received, even though it looks like a mess, it's better to not simplify things. Although it's still "dark energy" magic either way I look at it, lol

 

Its all structs, I modified the OP to clarify that

 

well good, that'll help speed for sure!

But I'd wait to hear from someone else, who eats machine code for breakfast, so they can give a better and clearer explanation

 

@Bunny83 @angrypenguin @Kurt-Dekker (the man the myth the legend)

 

The point of values types is they don't allocate.

You generally only allocate when you instance a new reference type. The foreach loop in the example will allocate (because it generates a state machine class under the hood).

In any case, measure. We have the profiler for a reason. There's probably no major difference in your example and you're wasting your time worrying about it.

 

Wait isnt it the other way around?
ref doesnt allocate because you are referencing existing memory allocation whereas value types especially immutable ones like strings always allocate?

Furthermore I dont think there is a way to for loop through a dictionary, only way would be to use an Enumerator, but thats what a foreach does under the hood anyway

 

That's not what I said in the slightest.

When you instance a new reference type you allocate. And strings are reference types, they just try to act like value types as far as usage goes.

 

Im even more confused now

 

Then I wouldn't worry too much about optimisation when you haven't got a full understanding of C# then.

It comes down to the stack vs the heap. You want to do some reading on that: https://www.c-sharpcorner.com/article/stack-vs-heap-memory-c-sharp/

 

That's a good representation of explanation, Thanks!

So I read the differences, then I read them again.. Then after reading it 22 times, I eventually scrolled down to look at the bottom diagram:

So where my memory screwed up, is where it says Obj(ref) on the stack, as I mistook that as a "reference" type. Apparently it's just "a reference", as I was still confused on making any "temporary" variable something that needs to be garbage collected. Which it's not, I assume it just gets over-written since the apocalyptic warning of "a stack can't hold much" still clouds over my head(or it deleting from LIFO isn't somehow technically garbage).

But as Spiney points out, things like lists or even a foreach loop(maybe not a for loop?) use the heap for all their shenanigans, and therefore would need garbage collected(if not cached). Which we all know, trashmen(thread) don't like to deal with large amounts of garbage per house(per frame)!

But I'm still grasping all the nuances of the finite details, all the things had I tried to learn in the beginning, would've completely turned me off to coding in the first place. Talk about a dull life... Not being able to create worlds, and of course blow them up.. lol

 

The local variable

obj

is on the stack, which itself is just a memory address to the actual instance on the heap.

But half the point of languages like C# is that we don't need to worry about this stuff in a majority of cases. I've only really had to do some real micro-optimisations once so far, and that was for some mesh deformation stuff.

In the example in the post, if the dictionary only contains a few KVP's, then I highly doubt this is a code hot-spot that needs worrying about.

Also - and this depends on what the rest of the code does - but unless there's physics stuff happening here, then this should not be in FixedUpdate either.

 

I think your confusion comes from the fact that there are different "kinds" of memory and when we talk about allocations we specifically talk about memory on the managed heap. The managed heap is controlled and managed by the memory manager / garbage collector. Any reference type lives on the heap. That's true for all reference types that includes strings. Strings are not value types as you said. Strings are objects and live on the managed heap like all the other reference types.

Value types on the other hand live directly inside the memory of the variable itself. Where that is can vary depending on where the variable is declared. However as soon as the variable exists, the memory that holds the value type exists. So storing a value type inside a class means the data of the value type actually lives on the heap. However it can not be collected or freed independently since the variable belongs to a class that lives on the heap. So as long as the class exists, the variable would exist and you can store and read data in that variable and it won't allocate any additional memory.

Local variables, which includes method arguments, live on the stack. The stack is NOT managed memory. The stack is a continuous memory section of a fix size. When you enter a method, the stack pointer is just moved to a different place on the stack to reserve a certain memory portion. When you exit that method, the stack pointer is moved back and the whole stack frame is "gone".

So structs on their own do not allocate managed memory. An array or Dictionary are objects and live on the heap. The creation of an array or Dictionary of course requires an allocation of memory on the heap. An array of value types / structs also live on the heap. So creating an array of size x would allocate memory on the heap. As long as you re-use that array, no additional allocations are taking place.

A Dictionary stores the actual elements inside an internal array. That internal array has a certain length which is the capacity of the dictionary. When that capacity is exceeded, the dictionary would create a new array with a larger size and the old array would be up for garbage collection. Though as long as the dictionary has a large enough capacity to hold all elements, no new garbage / memory would be generated / allocated when you add or remove elements.

Reading and writing values types means that you actually copy the data of that values type from one place to another. As I said earlier, local variables live on the stack and do not allocate memory. Though keep in mind when you "pass around" value types, you always copy the whole content of that value type. That's why value types should be kept small. Having value types that are several hundreds of bytes in size would not be great for performance as the whole data would be copied all the time.

Iterating through a dictionary with foreach does not allocate memory as the generic Dictionary class as well as the List class have struct enumerators. A common misconception is that foreach works on the IEnumerable / IEnumerator interfaces. However that's not true. In C# the foreach loop actually uses pattern matching during compile time. So it looks for a method called GetEnumerator on the given object / struct and checks if the returned type has a MoveNext method that returns a boolean and if it has a Current property. That's all. If it has the compiler will directly call that method, stores the result in a local variable and runs MoveNext in a loop. So a type does not need to implement the IEnumerator / IEnumerable interfaces in order to be used in a foreach loop. (In the past Unity uses an older version of Mono that did allocate memory for struct enumerators because it had a faulty implementation. That's because IEnumerator<T> implements IDisposable and mono boxed the enumerator just to call Dispose on it).

Note that it makes no sense to declare temporary variables as member variables of a class. They would be fix part of the memory footprint of the class. Just use local variables when you want to store things temporarily. Local variables are implicitly reserved all at once on the stack when you enter a method and are released automatically when you leave the method.

I don't get where you got the idea from that there are any memory allocations in the code you've shown. It makes no difference if the types "Guns" and GunDeck are a structs or classes. Nothing in that code snippet allocates memory since the actual instances already exists. So there is no "optimisation" here, at all.

Spoiler: Passing by value / reference

Another common misconception is the difference between passing a variable "by reference" or "by value". This has been taught and explained wrong for decades, even by some credible sources. A lot confusion comes from comparison to other languages which don't have the concept of passing by reference like javascript for example. You will still find countless of articles which talk about "passing by reference" in javascript, but that's not really a thing. Passing a variable by reference is not the same as passing a reference to a reference type instance around.
I just found a great article that talks about how javascript does not support passing by reference. It does a great job explaining the basic difference better than my following paragraphs ^^

In C# ( and C and C++) by default, arguments of methods are passed by value. This is true regardless of the variable type. Value type as well as reference type variables are all passed by value. The key thing you have to understand that we talk about variables here. The value of a reference type variable is the reference that is stored in that variable. That could be a reference to an object or null. When you pass a variable as an argument to a method, you actually copy the value of that variable into a local variable of that method. Just to make that clear: On a 64bit system a reference or pointer has 64 bits, so consists of 8 bytes. So a reference type variable occupies 8 bytes whereever it's declared. If it's a member variable of a class, 8 bytes in the memory of that class are designated to hold a reference to some other object. If it's a local variable 8 bytes on the stack would hold that reference. So inside the method you have a local variable on the stack that holds a copy of what was stored in the variable that got passed into the method.

Passing by reference on the other hand can only be done through the

ref

or

out

keyword. When you pass a variable "by reference" it means that we do NOT copy the value of the original variable into the local variable of the method, but an actual pointer to where the original variable is stored. So an actual pointer to the memory where the variable itself is stored. So inside the method the local variable acts like an alias name for the variable that was passed in. The big difference between passing by reference and passing by value is that when you pass by reference, the method can actually change the content of the variable that was passed to the method.

Yes, passing a reference to a class instance either by value or reference both allow the method to "follow the reference" to reach out to the referenced object. However that's not what passing by reference is about. That's just how references work in general. Assigning a one reference variable to another reference variable also just copies the value (the reference itself) into the other variable.

In C and C++ we would talk about "double pointers" or pointers to pointers. Though that's kinda off topic as we talk about C# here.

 

Huh, I was under the impression that even struct enumerators would always get boxed and thus always allocate, but I suppose I was reading old/incorrect information.

Good to know as I was often avoiding foreach loops in favour of for loops...

 

"Variables" never needs to be garbage collected. Only objects are. Variables can be part of an object and make up the memory footprint of that object. However individual "variables" do not live on the managed heap.

Individual values can be stored on the heap in wrapper objects which are called "boxes". That's essentially what happens when you "box" a value type. It's literally just like putting the content into a "box" and store that on the heap. So whenever you do

object obj = 5;

You will allocate managed memory on the heap as C# will essentially create something like that

Code (CSharp):

1. public class IntBox
2. {
3.      public int value;
4. }

and the line of code above would essentially do this

object obj = new IntBox{value = 5};

Of course the concept of boxing is a fix part of the .NET ecosystem, so this is just an analogy.

The heap can only store objects / reference types.

 

So just to be crystal clear on this, say using a static list, and you call:

Code (CSharp):

1. int a = Class.staticList[i].someValue;

still passes by reference, without technically using the "ref" keyword? Or maybe I'm asking it wrong, that it just creates a pointer to said list and index, and doesn't copy anything?

 

Note that I'm not sure about how the compiler will now treat struct enumerators that implement the

IEnumerator<T>

interface. However I know for sure that when a struct does NOT implement the interface, it won't allocate memory Over here I wrote a very simple enumerator / enumerable to iterate through the bits in a bitflag value. Since it's a struct it doesn't allocate any garbage.

 

int

is a value type, thus it's copied, not passed by reference. Value types are only passed by reference when you box them, or specifically passed them by reference with

ref

.

It should be noted that properties, including indexers, are just methods with some extra syntax sugar.

Sounds like I should do some tests then.

 

I meant from the dot hell from getting it from the list. As I read that passing reference part, it said the only way to "get" something without the data being copied was only by "ref" or "out".

So was confused if calling to an index of a list needed a ref keyword, or something weird like that

 

That part ^ ^

 

I'm... not sure what you're actually going on about now.

All that code does is access a static member/property of a static class, and returns the member/property at index

i

. There's no passing by reference going on here, just a few method calls potentially, at minimum one method call from the indexer.

 

Ohh ok, just when I saw "copy" I'm assuming that's something that needs to be cleaned up by GC. I originally thought there was no problem with doing it that way, just reading that one part made it seem like more was happening under the hood.. thanks for clearing that up

 

The term "passing by reference" only applies to method calls and only applies to variables.

You also confuse the concept of passing variables to method with just using a reference value. They are not related. In the code you posted there's no method call with the exception of the indexer of the list. So technically you pass the variable

i

by value to the indexer get method. Though apart from that:

Class.staticList

just reads the value that is stored in that static variable. That "value" is a reference to a List object that is stored on the heap like every object.

On that value you now call the getter method

(list reference).GetItem(i)

The getter method returns a copy of the element. If it's a reference type that is stored in the list, it would return the reference value that is stored at that index. If it's a value type, it returns the whole instance of the value type.

(value type instance).someValue

This final bit just reads a part of the actual value type. Essentially the data at a certain offset.

I always wanted to make a visualisation that shows the actual memory and what's stored inside the individual bytes. However it's difficult to make it clear and understandable.

No, that's not what I said. I was only talking about the concept of passing variables by reference or by value. Not about "something". When you have a reference value that points to an object on the heap, you can copy that reference as often as you want. It won't allocate any new memory and you do not copy the "data" of the referenced class. You just copy the reference itself. The reference is the content of a reference variable.

 

It's 3:30am now and I need some sleep ^^ Have to work tomorrow... ehh "later"

 

The only things that get cleaned up are partitions of the managed heap. Ergo, when you allocate when instancing a new reference type and when the garbage man determines it's no longer in use.

That's why folks chase allocation free code, as the act of allocating memory itself, and the act of cleaning up said memory is what has overhead.

 

Don‘t ask us. You have a profiler!

 

I can confirm that it does not allocate even if the struct implements

IEnumerator<T>

.

List<T>.Enumerator

for example implements the interface but does not allocate when a

List<T>

is iterated using foreach.

Proof

 

Only allocation issue you can get in both examples is if some methods you're calling accessing your structs directly by interfaces. Since interface implies reference passing, and to have a reference you need object, and value isn't object, so one is created automatically.
...and this is highly unlikely since there no straight up method calls, and it takes quite purposeful approach to make

.Reloading

do this.

But since I've mentioned it, generics is the way around:

Code (CSharp):

1. interface IFace {}
2. struct ValueFace : IFace {}
3.  
4. void Foo(IFace arg) {} // DON'T
5. void Bar<TFace>(TFace arg) where TFace : IFace {} // OKAY

 

Thanks for all the insightful replies everyone. I appreciate it.

Really I just want to know how to make this loop garbage/allocation free.
Can we figure out together what the actual code/loop should look like to make this happen?

Dictionary<string, Guns> Ships; // global scope variables = much less garbage? The class will be running always so I dont care if these things are kept in memory for the whole lifetime of the app.

Guns guns; // struct

GunDeck gunDeck; // struct

int decksCount;

int decksReloading;



void FixedUpdate(){

foreach (KeyValuePair<string, Guns> shipGuns in Ships) // foreach correct?

      {

          guns = shipGuns.Value;  // Lets figure it out conclusively?

          if (!guns.Reloading) continue;



          decksCount = guns.GunDecks.Length; // This is def the way to go alloc free

          decksReloading = 0;



          for (int d = 0; d < decksCount; d++)

          {

              gunDeck = guns.GunDecks[d];

              if (!gunDeck.Reloading) continue;

              decksReloading++;

              ...

Some more info, there can be 1000s of Ships, this is just the Guns/Reloading loop.
Each Ship/Guns.cs has 1-4 GunDecks.
Each GunDeck has 2-4 CannonGroups.
Each CannonGroup has 2 - 40 Cannons.

1000ships * 2 GunDecks * 2 CannonGroups * 10 Cannons
40.000 Cannons that need their reloadtime adjusted. Which will never happen at once tbh.

The rest of the loop that you cant see actually goes in a quadtree structure, which is why we are recursively checking Reloading= true/false for each level/layer. This way if there is no Cannons reloading the CannonGroup isnt Reloading if there is no Groups Reloading on that Deck then the Deck isnt reloading and if there are no Decks reloading on the ship then the whole ship wont be reloading and it will skip that whole ship.

So it only loops through ships/decks/groups/cannons that actually are reloading, the rest it skips for performance.

Thanks men

 

The first questions you have to ask yourself is:

• Is it actually allocating? Ergo, have you even profiled this yet?
• And does it matter?

Honestly, don't worry about performance until you actually have an issue. Your progress will stagnate otherwise.

And if you read these posts you probably would notice that we established that its most likely not allocating anything.

I think the better question is why this code is in FixedUpdate when it's not physics related.

 

FixedUpdate runs at 50fps, Update can run at 100s of FPS, for my use case, a reload its good enough, actually I might put this loop into a Coroutine and update it at 20 FPS for even better perf.

 

FixedUpdate is not a source of a regular time interval. It plays catch-up with Update using an accumulator, and can sometimes run multiple times between Update calls. Despite the name, it should be used for physics only.

You're better off just having a timer in Update that only executes this logic at specific intervals.

In any case, please profile your code before you ask us to optimise it for you.

 

Code (CSharp):

1. void FixedUpdate()
2. {
3.     Dictionary<string, Guns> ships = Ships; // no allocation
4.  
5.     foreach(KeyValuePair<string, Guns> shipGuns in Ships) // no allocation
6.     {
7.         Guns guns = shipGuns.Value; // no allocation
8.  
9.         if(!guns.Reloading) // no allocation
10.         {
11.             continue;
12.         }
13.  
14.         IGuns iguns = guns; // <- allocates on the heap (value type gets boxed into an interface type)
15.  
16.         if(!iguns.Reloading) // no allocation
17.         {
18.             continue;
19.         }
20.  
21.         object oguns = guns; // <- allocates on the heap (value type gets boxed into an object type)
22.  
23.            
24.         guns = new Guns(); // no allocation, since it's a value type
25.         oguns = new object(); // <- allocates on the heap, since it's an object type
26.  
27.         decksCount = guns.GunDecks.Length; // no allocation
28.  
29.         decksReloading = 0; // no allocation
30.  
31.         for(int d = 0; d < decksCount; d++) // no allocation
32.         {
33.             if(!guns.GunDecks[d].Reloading) // no allocation
34.             {
35.                 continue;
36.             }
37.  
38.             gunDeck = guns.GunDecks[d]; // no allocation
39.         }
40.     }
41. }

 

Thanks, so my code without any interfaces or objects, since its all structs is allocation freeee

 

I will make a new thread for this question since Ive always wondered about performance between Update Fixed and Coroutines

(while=true) yield return null;

 

Ohhh boy... I can already see a new post about an error, "why Unity crashes after using coroutine?" lol...

 

Lmao this is the standard way to make an infinite update loop inside a Coroutine, yes it will crash your Unity if you do it wrong, but my coroutine code in the new thread should be 100% safe and work.