IL2CPP super SLOW vs Mono Standalone

Hello guys, i working on a WebRTC in full c#.

In last week i finally finished my H264 Codec Decoder (100% C#) but i facing a super strange problem when building the project with IL2CPP.

In mono Build i can archive around 30~40fps while decoding a 1920x1020 H264 Stream, but when i swap to IL2CPP i can only reach half of this (around 15~20fps).

I tried everything in project settings options (Master C++ Compiler Configuration, Strip Engine Code, etc) but seen that IL2CPP can't handle well the heavy CPU byte[] operations.

Here my results after trying everything:

Code (CSharp):

1.  
2. //Tested in Unity 2019.4.1f1 with 8 threads in intel i7 6700HQ
3. --------------------------------------
4. | MONO          | 38,42 frames/sec   |
5. --------------------------------------
6. | IL2CPP        | 19,55 frames/sec   |
7. --------------------------------------

Maybe someone has a tip to improve perfomance in IL2CPP? Is mono really much better optimized to heavy byte[] operations?

PS: I Already tried to replace byte[] to unsafe byte* calls, or optimize for(int i=0; i<array.Length; i++) to for(int i=0; i<n; i++) but no improvements noticed in final decoding speed.

EDIT:
Well i already made another benchmark with a simple loop with 600000000 iterations
and seems that IL2CPP is super slow with generics. The weird thing is that i just use byte[] and MemoryStreams in H242 operations... maybe related to memory stream, or Array.Copy / Buffer.BlockCopy?

 

Thanks @JoshPeterson
I already submited a bug report today.

 

@JoshPeterson number 1264028

Another tip from my investigations releaved that one generic method that simplificate BlockCopy perform a huge overhead in IL2CPP

ArrayHelper.BlockCopy<T>

Maybe generics dont perform well in IL2CPP?

EDIT:

Okey, finally found that all the problems was related to functions with IList<T> as parameter.
Changing IList<T> to T[] i finally fixed the performance problems in IL2CPP (and now is running faster than Mono)

EDIT 2: Net Core seems to keep running twice faster than final build in IL2CPP, is that correct? RyuJIT compiler seem to be much faster than unity.

EDIT 3:

Unity Editor runs 20 times slower than IL2CPP Build and my code is useless inside editor

 

I think IList<T> looping isn't generics being slow, it's interfaces.
A method like:
void Loop<T> (IList<T> list) { }
will pass an interface and use virtual calls it can't inline etc, unless you're lucky and the entire method gets inlined away

I'd be interested to see how it performs if you adjust it to
void Loop<T, U>(U list) where U : IList<T> { }
as I believe that'll instantiate functions for every list type you pass in, getting rid of the virtual calls (unless your U is an interface? idk if that works) and making inlining etc easier for the compiler

 

Nice tip, let-me try your suggestion

 

Thanks for reply @JoshPeterson , i using unity 2019.4

Where can i find this option? i never saw this property in editor preferences. I tryed to find this "debug code generation" option but i failed to find it.

 

Thanks @JoshPeterson i will upgrade to Untiy 2020 to access this better workflow. Thank you for your time.

@Zuntatos i tested your approuch but it seems to run ultra slow too.

The problem is with IList<T> ... even if i pass parameter as U with U : IList<T> the methods used will continue be from IList interface, even if i pass, for example, byte[]. The performance is the same as using simple IList<T> as parameter

 

Issue track accepted by unity

https://issuetracker.unity3d.com/is...an-mono-when-performing-operations-with-ilist

 

Its great that you got a response back to us, but this seems like a pretty painful limitation (its easy to work around, but the workaround is likely at the expense of the reusability of your code).

Is there any idea of if/when this might change?

 

When can we expect that to be fixed? Having to avoid interfaces obviously is a major issue, some of us have client projects that are already live and at scale that use interfaces extensively as there was no information beforehand to say otherwise. Rewriting these would not be a small task, and would be at a serious cost to our business not only financially but in terms of time.

 

Thanks, unfortunately I dont think I have the rights to do that with our codebase but I will talk to management and see if we can get some sort of reports together for you so that you have some data. Thanks for getting back to us!

 

I also want to emphasize that interfaces are fairly important for us - and sometimes they're unavoidable. It's good to know that they can potentially cause performance issues, so we will take a closer look at the parts of our code which use them extensively (e.g. our networking implementation).

 

I don't get why issue like that is marked as "By Design". It makes no sense, users should be able to upvote it.

 

He explains what you can do, to give this issue more priority, in this post:
https://forum.unity.com/threads/il2cpp-super-slow-vs-mono-standalone.932499/#post-6489914

 

We have done some benching on il2cpp vs net 5 and in all cases net 5 is faster than il2cpp. In many cases its also faster than cpp. Pretty cool what hardware specific just in time compilation can do ä for performance.

Not being able to use interfaces in a tight loop is a showstopper if you want to keep a abstract and maintainable domain.

 

Still doesn't make sense imo. I thought this was the reason for the issue tracker and upvoting (users decide Unity's priority to a certain extent via voting for stuff they want fixed).

"View bugs we have successfully reproduced, and vote for the bugs you want to see fixed most urgently."

But we can't vote for it. Instead we are supposed to deep profile and show specific use-cases where it has an obvious detrimental impact.... and then Unity might consider bumping it up their priority list.

It's just kind of weird because it's marked as "By Design".

RESOLUTION NOTE:
Interface methods calls in IL2CPP are slower then they are in Mono. This is a known issue that we plan to address in the future, although we don't have any ETA for that work to be completed

So.... the initial IL2CPP implementation is slower by design? I assume there was some trade-off made where it is slower than mono but there are other benefits that outweigh this reduction in speed.

 

It should be obviously marked as Postponed that doesn't disable voting.

 

Thanks @JoshPeterson

For everyone who wants this issue fixed, please vote

https://issuetracker.unity3d.com/is...an-mono-when-performing-operations-with-ilist

 

It has been a whole year or more. Any news on this? I have obviously voted for this as other than pc platform I dont have choice for not using il2cpp...

 

Yes.. this is a super problematic problem in my Project too...

 

This is kind of issue where it is so easy to be overlooked at, unless devs do very extensive testing on their game comparing il2cpp to mono.

But I suspect that a lot of devs simply does not even realize that there is such an issue. It is something that devs simply does not even suspect anything like this is happening in the background. Feels like a... there is catch on using il2cpp but not so well known, kinda like my bill that feels like it should be less but not suspecting that anything that could be possibly taking away from me...

I bet if this is more widely commented, documented then we would get so many more votes.

It maybe harsh words but, it is sad that something like this is being ignored by Unity for a whole year... just makes me to think, ok, "is there anything else" that Unity is ignoring that I should be aware of? All because, I want to avoid taking that path, or using that particular feature of Unity. If I knew about this a much earlier, or thought it won't be fixed for whole year, (and no eta on anything happening in the future) I would have not used interface for tight loop calls.....

 

i have that problem in my project that's build on windows...so said.

 

Ah, so this is why the Netcode for Game Objects team changed from INetworkedVar (interface) to NetworkedVariableBase (abstract class).

Yay, another C# feature we need to be wary of when optimizing for consoles/mobile (where only IL2CPP is supported).