Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

Suggestion for reducing the size of IL2CPP generated executable

Discussion in 'iOS and tvOS' started by United-Unity, Jul 7, 2015.

  1. United-Unity

    United-Unity

    Joined:
    Mar 25, 2015
    Posts:
    20
    Hello!

    Recently my team has been struggling for reducing iOS IPA size, especially code size.

    It was hard to find and remove unnecessary code. But even with this work, IPA size was not enough small to download on the air.

    Next we were to investigate how much space each code take. And found out that CustomAttributesCache related functions took lots of space in __TEXT section. (in my case 6% of __TEXT)

    With Unity 4.6.6p1 CustomAttributesCache looks like this.

    Code (csharp):
    1. extern TypeInfo* DebuggerHiddenAttribute_t102_il2cpp_TypeInfo_var;
    2. void BombAiBehaviour_t2418_CustomAttributesCacheGenerator_BombAiBehaviour_ProcessAction_m12693(CustomAttributesCache* cache)
    3. {
    4.     static bool s_Il2CppMethodIntialized;
    5.     if (!s_Il2CppMethodIntialized)
    6.     {
    7.         DebuggerHiddenAttribute_t102_il2cpp_TypeInfo_var = il2cpp_codegen_type_info_from_index(54);
    8.         s_Il2CppMethodIntialized = true;
    9.     }
    10.     cache->count = 1;
    11.     cache->attributes = (Il2CppObject**)il2cpp_gc_alloc_fixed(sizeof(Object_t *) * cache->count, 0);
    12.     {
    13.         DebuggerHiddenAttribute_t102 * tmp;
    14.         tmp = (DebuggerHiddenAttribute_t102 *)il2cpp_codegen_object_new (DebuggerHiddenAttribute_t102_il2cpp_TypeInfo_var);
    15.         DebuggerHiddenAttribute__ctor_m466(tmp, NULL);
    16.         cache->attributes[0] = (Il2CppObject*)tmp;
    17.     }
    18. }
    This code seems to work for caching custom attributes data to be queried later. It’s fairly ok to support this feature in a static fashion. And following is summarized list of attributes supported by custom attribute caches.

    Code (csharp):
    1.              Name              Count   Ratio  
    2.  
    3. ---------------------------- ------- --------
    4.   DebuggerHiddenAttribute       3691   45.82%
    5.   CompilerGeneratedAttribute    3121   38.74%
    6.   SerializeField                 284   3.53%  
    7.   ExtensionAttribute             240   2.98%  
    8.   HideInInspector                183   2.27%  
    9.   AddComponentMenu               182   2.26%  
    10.   ObsoleteAttribute               99   1.23%  
    11.   ExecuteInEditMode               57   0.71%  
    12.  
    DebuggerHidden and CompiledGenerated takes almost 85% of attributes. And these attributes may be generated for coroutine and that means these attributes don’t has any significant role in running codes, right?

    So I would like to suggest that il2cpp has the option to remove these useless attributes in advance to suppress generating large cache code.

    Thanks!
     
    icyaway and miskoltrans like this.
  2. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,424
    @United Unity

    Thanks for the suggestion. We have looked at moving custom attributes to an external file which is not part of the executable, but decided it was not worth the cost. This is an interesting data point though. I'll add this to our backlog of possible improvements.
     
  3. United-Unity

    United-Unity

    Joined:
    Mar 25, 2015
    Posts:
    20
    Hello again!

    I've done making an experimental tool for reducing executable size because it's quite critical issue for me.
    Even it is a dirty hack, it seems good to share the result.

    As I posted earlier, top majority of my executable is code that makes attribute caching data and most of them are
    `CompilerGeneratedAttribute` and `DebuggerHiddenAttribute` which are not necessary for running app.

    To reduce the amount of code, a tool removing unnecessary attributes is put in front of il2cpp. Those attributes are
    Code (csharp):
    1.  
    2. System.Runtime.CompilerServices.CompilerGeneratedAttribute
    3. System.Runtime.CompilerServices.ExtensionAttribute
    4. System.ParamArrayAttribute
    5. System.Reflection.DefaultMemberAttribute
    6. System.Diagnostics.DebuggerStepThroughAttribute
    7. System.Diagnostics.DebuggerHiddenAttribute
    8. System.Diagnostics.CodeAnalysis.SuppressMessageAttribute
    9. System.ObsoleteAttribute
    10. UnityEngine.AddComponentMenu
    11. UnityEngine.ExecuteInEditMode
    12. UnityEngine.HideInInspector
    13. UnityEngine.TooltipAttribute
    14. UnityEngine.DisallowMultipleComponent
    15. UnityEditor.MenuItem
    16.  
    My project had 14000 attributes related functions but number was decreased to 2000 after this processing.
    And the change of app size is following.
    Code (csharp):
    1.  
    2.                            Before         After         Delta  
    3. --------------------- -------------- ------------- -------------
    4.   IPA Size               71,047,667     69,351,885    -1,695,782
    5.   Raw Executable Size   133,363,328    119,887,936   -13,475,392
    6.   32bit __TEXT Size      24,739,840     23,822,336      -917,504
    7.   32bit __DATA Size       4,046,848      3,981,312       -65,536
    8.   32bit others Size      39,034,880     31,768,576    -7,266,304
    9.  
    It worked nicely for my project and expect this feature will be included in il2cpp :)
     
    Last edited: Jul 21, 2015
  4. United-Unity

    United-Unity

    Joined:
    Mar 25, 2015
    Posts:
    20
    I think that it's good to share my code to build tool. I build this code with mono.cecil (in UnusedByteCodeStripper2)
    And result exe is renamed as UnusedByteCodeStripper2.exe and moved to UnusedByteCodeStripper2 directory.
    (before this, original UnusedByteCodeStripper2.exe is renamed as UnusedByteCodeStripper2.org.exe)

    Code (CSharp):
    1. using System;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using System.Diagnostics;
    5. using System.IO;
    6. using System.Linq;
    7. using System.Reflection;
    8. using System.Text;
    9. using Mono.Cecil;
    10. using Mono.Collections.Generic;
    11.  
    12. namespace RemoveAttributesTool
    13. {
    14.     internal class Program
    15.     {
    16.         private static readonly string[] RemoveAttributesNames =
    17.         {
    18.             // Just information
    19.             "System.Runtime.CompilerServices.CompilerGeneratedAttribute",
    20.             "System.Runtime.CompilerServices.ExtensionAttribute",
    21.             "System.ParamArrayAttribute",
    22.             "System.Reflection.DefaultMemberAttribute",
    23.             "System.Diagnostics.DebuggerStepThroughAttribute",
    24.             "System.Diagnostics.DebuggerHiddenAttribute",
    25.             "System.Diagnostics.DebuggerDisplayAttribute",
    26.             "System.Diagnostics.CodeAnalysis.SuppressMessageAttribute",
    27.             "System.ObsoleteAttribute",
    28.             "System.AttributeUsageAttribute",
    29.             "System.MonoTODOAttribute",
    30.             // Not relative
    31.             "System.CLSCompliantAttribute",
    32.             "System.Runtime.InteropServices.ComVisibleAttribute",
    33.             "System.Runtime.ConstrainedExecution.ReliabilityContractAttribute",
    34.             // Editor only
    35.             "UnityEngine.AddComponentMenu",
    36.             "UnityEditor.MenuItem",
    37.             "UnityEngine.ContextMenu",
    38.             "UnityEngine.ExecuteInEditMode",
    39.             "UnityEngine.HideInInspector",
    40.             "UnityEngine.TooltipAttribute",
    41.             "UnityEngine.DisallowMultipleComponent",
    42.             "UnityEngine.Internal.ExcludeFromDocsAttribute",
    43.         };
    44.  
    45.         private static readonly string[] AdditionalDllFileNames =
    46.         {
    47.             "UnityEngine.dll",
    48.             "mscorlib.dll",
    49.             "System.dll",
    50.             "System.Core.dll",
    51.             "System.Xml.dll",
    52.             "Mono.Security.dll",
    53.         };
    54.  
    55.         private static void Main(string[] args)
    56.         {
    57.             // Process
    58.  
    59.             for (var i = 0; i < args.Length; i++)
    60.             {
    61.                 switch (args[i])
    62.                 {
    63.                     case "-a":
    64.                         ProcessDll(args[i + 1]);
    65.                         break;
    66.                 }
    67.             }
    68.  
    69.             foreach (var fileName in AdditionalDllFileNames)
    70.             {
    71.                 if (File.Exists(fileName))
    72.                     ProcessDll(fileName);
    73.             }
    74.  
    75.             // Run original executables
    76.  
    77.             var monoCfgDir = Environment.GetEnvironmentVariable("MONO_CFG_DIR");
    78.             var monoPath = monoCfgDir.Substring(0, monoCfgDir.Length - 3) + "bin/mono";
    79.  
    80.             var currentModulePath = Assembly.GetExecutingAssembly().Location;
    81.             var orgModulePath = currentModulePath.Substring(0, currentModulePath.Length - 3) + "org.exe";
    82.  
    83.             var orgArgs = '"' + orgModulePath + '"' + ' ' + string.Join(" ", args.Select(a => '"' + a + '"'));
    84.             var handle = Process.Start(monoPath, orgArgs);
    85.             handle.WaitForExit();
    86.         }
    87.  
    88.         private static void ProcessDll(string dllPath)
    89.         {
    90.             AssemblyDefinition assemblyDef;
    91.  
    92.             using (var assemblyStream = new MemoryStream(File.ReadAllBytes(dllPath)))
    93.             {
    94.                 assemblyDef = AssemblyDefinition.ReadAssembly(assemblyStream);
    95.             }
    96.  
    97.             ProcessAssembly(new[] {assemblyDef});
    98.  
    99.             using (var assemblyStream = File.Create(dllPath))
    100.             {
    101.                 assemblyDef.Write(assemblyStream);
    102.             }
    103.         }
    104.  
    105.         private static void ProcessAssembly(AssemblyDefinition[] assemblyDefs)
    106.         {
    107.             foreach (var assemblyDef in assemblyDefs)
    108.             {
    109.                 foreach (var moduleDef in assemblyDef.Modules)
    110.                 {
    111.                     foreach (var type in moduleDef.Types)
    112.                         RemoveAttributes(type);
    113.                 }
    114.             }
    115.         }
    116.  
    117.         private static void RemoveAttributes(TypeDefinition typeDef)
    118.         {
    119.             RemoveAttributes(typeDef.FullName, typeDef.CustomAttributes);
    120.  
    121.             foreach (var field in typeDef.Fields)
    122.                 RemoveAttributes(field.Name, field.CustomAttributes);
    123.  
    124.             foreach (var property in typeDef.Properties)
    125.                 RemoveAttributes(property.Name, property.CustomAttributes);
    126.  
    127.             foreach (var method in typeDef.Methods)
    128.                 RemoveAttributes(method.Name, method.CustomAttributes);
    129.  
    130.             foreach (var type in typeDef.NestedTypes)
    131.                 RemoveAttributes(type);
    132.         }
    133.  
    134.         private static void RemoveAttributes(string ownerName, Collection<CustomAttribute> customAttributes)
    135.         {
    136.             foreach (var attrName in RemoveAttributesNames)
    137.             {
    138.                 var index = -1;
    139.                 for (var i = 0; i < customAttributes.Count; i++)
    140.                 {
    141.                     var attr = customAttributes[i];
    142.                     if (attr.Constructor != null && attr.Constructor.DeclaringType.FullName == attrName)
    143.                     {
    144.                         index = i;
    145.                         break;
    146.                     }
    147.                 }
    148.  
    149.                 if (index != -1)
    150.                     customAttributes.RemoveAt(index);
    151.             }
    152.         }
    153.     }
    154. }
     
    Last edited: Sep 5, 2015
  5. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,424
    @United Unity

    Thanks for sharing your results and your tool. We will certainly consider making this feature a part of the Unity toolchain.
     
    Shushustorm likes this.
  6. dchau_hh

    dchau_hh

    Joined:
    Jan 22, 2014
    Posts:
    24
    @United Unity

    I'd like to try out your stripping tool but the code you posted doesn't compile. How are you invoking the original UnusedByteCodeStripper2 after you do your initial strip?

    Thanks!
     
  7. United-Unity

    United-Unity

    Joined:
    Mar 25, 2015
    Posts:
    20
    Oops! I made a mistake to post previous source. "Process.Start" statement was stipped out by mistake.
    I fixed source to compile correctly. Please try it again.
     
  8. nbaris

    nbaris

    Joined:
    Jan 13, 2015
    Posts:
    18
    Thanks for sharing this. May I ask how you calculate this break-down of the code segment?
     
  9. United-Unity

    United-Unity

    Joined:
    Mar 25, 2015
    Posts:
    20
    I remember that I used nm utility for estimating the reduction of __TEXT. And after making stripping tool it became easy because you can measure the size of __TEXT of before & after.
     
  10. larku

    larku

    Joined:
    Mar 14, 2013
    Posts:
    1,412

    Hey United Unity, any chance you can provide the details on how to build this? I've got no idea what mono.cecil is or what I'd do with it. I see there's a Mono.Cecil.dll in the same folder as UnusedByteCodeStripper2.exe.. Any links to how I'd do this?
     
  11. larku

    larku

    Joined:
    Mar 14, 2013
    Posts:
    1,412
    Hey JoshPeterson,

    Did this get into the Unity build pipeline? I'm considering using this technique as I need to shave a couple of mb off my IPA...

    Specifically, is this already being done with 4.6.9f1?
     
  12. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,424
    @larku

    No, this tool is not in the Unity build pipeline, it will still need to be done separately. Note that Mono.Cecil is a C# tool used to inspect and modify IL Assemblies. We use it for a number of tool in Unity, like il2cpp.exe and UnusedByteCodeStripper2.exe. Also, UnusedByteCodeStripper2.exe is a tool we use in the IL2CPP build process to remove IL code from assemblies that we can prove is not called at runtime. You can see the command line we use to invoke it if you build for IL2CPP and search for UnusedByteCodeStripper2 in the editor log.
     
  13. larku

    larku

    Joined:
    Mar 14, 2013
    Posts:
    1,412
    Thanks JoshPeterson,

    I did finally work out that the original posters comment about "Mono.Cecil" was meaning that they added it as a reference when building the tool..

    I've done this but unfortunately it only gave me about 100kb saving :( I need about 4mb
     
    Last edited: Dec 12, 2015
  14. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,424
    @larku

    Yes, the impact of this seems to vary depending on the project, unfortunately.
     
  15. United-Unity

    United-Unity

    Joined:
    Mar 25, 2015
    Posts:
    20
    Hi, there.

    First time I wrote this tiny tool, I assumed that this code will be expired for any reason. But still my project uses it (up until Unity 5.3.1) There are maybe someones that need this tool to reduce the size of app. I uploaded this tool to github and made it more easier :)

    Check this one:
    https://github.com/SaladbowlCreative/Unity3D.UselessAttributeStripper/
     
unityunity