Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

C# HasAFlag method extension: how to not create garbage allocation?

Discussion in 'Scripting' started by FeastSC2, Jan 22, 2019.

  1. FeastSC2

    FeastSC2

    Joined:
    Sep 30, 2016
    Posts:
    978
    I would like to have an extension method on my flags to know if they have a certain flag in them or not.
    The problem is that the HasAFlagOf<T> code allocates garbage and I just use flags too much to allow this. How can I not allocate garbage and use a method that works for my flags?

    Code (CSharp):
    1. public static class EnumExt{
    2.     public static bool HasAFlagOf<T>(this T value, T flag) where T : struct // allocates garbage
    3.     {
    4.         return (Convert.ToInt64(value) & Convert.ToInt64(flag)) != 0;
    5.     }
    6.     public static bool HasAFlagNoGarbage(this SH_UberFx.XY value, SH_UberFx.XY flags) // NO garbage allocated but I must create many different variations of these because it is type dependant
    7.     {
    8.         return (value & flags) != 0;
    9.     }
    10.     public static bool HasAFlagNoGarbage(this SH_UberFx.MainTex value, SH_UberFx.MainTex flags)// NO garbage allocated but I must create many different variations of these because it is type dependant
    11.     {
    12.         return (value & flags) != 0;
    13.     }
    14. }

    Here's how I create all my flags ( I just need the method to work with that kind of flag enum ):

    Code (CSharp):
    1.     [System.Flags]
    2.     public enum VegetationBitmaskEnum
    3.     {
    4.         Wind = 1 << 0,
    5.         Ondulation = 1 << 1,
    6.         Rotation = 1 << 2,
    7.         Scale = 1 << 3,
    8.         //All = Wind | Ondulation | Rotation | Scale
    9.     }
     
  2. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    I guess that the culprit is 'Convert.ToInt64()', did you tried just casting?

    The way I used to manage the Enums without garbage is to dynamically create the cast methods:
    Code (CSharp):
    1. DynamicMethod AreEqualMethod = new DynamicMethod("AreEqual", MethodAttributes.Public | MethodAttributes.HideBySig, CallingConventions.Standard, typeof(Boolean), new Type[] { TheType, TheType }, typeof(EnumEqualityComparer), true);
    2. ILGenerator TheIlGenerator = AreEqualMethod.GetILGenerator();
    3. TheIlGenerator.Emit(OpCodes.Ldarg_0);
    4. TheIlGenerator.Emit(OpCodes.Ldarg_1);
    5. TheIlGenerator.Emit(OpCodes.Ceq);
    6. TheIlGenerator.Emit(OpCodes.Ret);
    7. AreEqual = (Func<TEnum, TEnum, Boolean>)AreEqualMethod.CreateDelegate(typeof(Func<TEnum, TEnum, Boolean>));
    8.  
    9. DynamicMethod CastToInt32Method = new DynamicMethod("CastToInt32", MethodAttributes.Public | MethodAttributes.HideBySig, CallingConventions.Standard, typeof(Int32), new Type[] { TheType }, typeof(EnumEqualityComparer), true);
    10. TheIlGenerator = CastToInt32Method.GetILGenerator();
    11. TheIlGenerator.Emit(OpCodes.Ldarg_0);
    12. TheIlGenerator.Emit(OpCodes.Ret);
    13. CastToInt32 = (Func<TEnum, Int32>)CastToInt32Method.CreateDelegate(typeof(Func<TEnum, Int32>));
    14.  
    15. DynamicMethod CastToValueMethod = new DynamicMethod("CastToValue", MethodAttributes.Public | MethodAttributes.HideBySig, CallingConventions.Standard, TheType, new Type[] { typeof(Int32) }, typeof(EnumEqualityComparer), true);
    16. TheIlGenerator = CastToValueMethod.GetILGenerator();
    17. TheIlGenerator.Emit(OpCodes.Ldarg_0);
    18. TheIlGenerator.Emit(OpCodes.Ret);
    19. CastToValue = (Func<Int32, TEnum>)CastToValueMethod.CreateDelegate(typeof(Func<Int32, TEnum>));
    20.  
    The problem is that it won't work on mobile, as I think it's not possible to generate IL code at runtime on mobile.
    It also probably won't work with IL2CPP.

    And to use it, I created a generic static class:
    Code (CSharp):
    1. static public class EnumExt<TEnum>
    That way I can 'remember' everything I need for each type of enum, including all the values and names, that way I can even have a 'toString()' which is garbage-less.
     
    the_real_ijed and FeastSC2 like this.
  3. FeastSC2

    FeastSC2

    Joined:
    Sep 30, 2016
    Posts:
    978
    Indeed it is, but casting doesn't work for some reason.

    I'm not familiar enough with your code to use it but when I'll be better in programming I just might use it.

    Could there be a simpler solution by any chance?
     
  4. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    You're right, just casting doesn't work, which explain why I had to use the code in my previous post (it was long ago, I wasn't sure if it was the reason or not).

    I'm afraid that there's not really any other way. In fact, it's not a .NET limitation, it's a C# one. Such a casting is allowed in IL code but not in C# because of a syntax limitation.


    That said, there is another way, which is not very pretty, but which works quite well.
    You can generate C# code with all the Enums you need.
    I mean, you just have to write a program which writes the source code you need in plain C#, as if you wrote it yourself. It will simply write the same functions over and over again, just changing the type of the enum.
    You name the created file as a '.cs' file and you include it in the Unity asset folder as you would for any source file you write yourself.

    The problem is that each time you need to have another type of enum, you need to to add it to the list of enums your code generator generates, and generate the code again.


    Note that you can also use reflection in order to scan you source code to find all the enums you have, and automatically generate the source file for all the 'HasAFlagNoGarbage()' methods your source code requires.
    That takes a bit more skills, but honestly not a lot. Reflection is a lot easier than the code generation I put in my previous post.
    Take a look there:
    https://stackoverflow.com/questions/4247410/c-sharp-or-vb-net-iterate-all-public-enums
    It contains that method:
    Code (CSharp):
    1. var query = Assembly.GetExecutingAssembly()
    2.                     .GetTypes()
    3.                     .Where(t => t.IsEnum && t.IsPublic);
    4.  
    5. foreach (Type t in query)
    6. {
    7.     Console.WriteLine(t);
    8. }
    That will find out all the Enum types in the currently executing assembly, and as you can see, it's not that complex.
    Instead of the 'WriteLine()' call, you'll have to write in a text file the text corresponding to the content of your 'HasAFlagNoGarbage()' method ('t.name' will contain the name of the enum type).
     
    FeastSC2 and xVergilx like this.
  5. xVergilx

    xVergilx

    Joined:
    Dec 22, 2014
    Posts:
    3,296
    There's not much you can do.
    About the allocation, I don't think the Convert.ToInt64 does that.
    It's the boxing allocation that is performed on the value in order to convert it from generic to the object and then pass it to the convert method.

    Writing the code manually or generating the code is the solution unfortunately.
    As you can't really apply constraints for the value types.
     
    FeastSC2 likes this.
  6. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
    "The implementation of an allocation-free version of HasFlags is fairly straightforward:"

    From "dissecting new generic constraints in C# 7.3". The core change that enables this is the where T : Enum constraint introduced in C# 7.3
     
  7. lordofduct

    lordofduct

    Joined:
    Oct 3, 2011
    Posts:
    8,513
    I've played with the same idea of having enum extension's for flags and the sort.

    And well... it just don't work. You will get boxing all the time because the compiler doesn't know what type it is. It's just going to happen. It doesn't help that when treated as System.Enum it gets boxed.

    I gave up on the idea and just inline my flag checks into the code.

    That or you convert them to long's before passing them into the function you write. Or using newer versions of C# that do support the proper constraints, which I'm pretty sure aren't in Unity yet.
     
    FeastSC2 likes this.
  8. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
    C# 7.3 is supported in 2018.3 and 2019.1. You can get access in 2018.2 with the (deprecated) incremental compiler package.
     
    FeastSC2 and lordofduct like this.
  9. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    It's not the 'T : Enum' which makes it garbage-free, it's the IL code generated, which also work with .NET 3.5.
    Unfortunately it has some limitation with mobiles and IL2CPP probably.

    Also, it seem to be too complex for the OP, which is why I think that generating C# code in plain text instead of IL code is probably the best way to go.


    Thanks for the link, it's very interesting to know about the new constraints, that can avoid some runtime checks.
     
    Baste and FeastSC2 like this.
  10. FeastSC2

    FeastSC2

    Joined:
    Sep 30, 2016
    Posts:
    978
    That seems all very interesting, thanks for the many answers.

    Where did you guys learn this?
    Where would you go to learn C# online as thoroughly as possible nowadays?
     
  11. xVergilx

    xVergilx

    Joined:
    Dec 22, 2014
    Posts:
    3,296
    Good to know. Now I can finally avoid boxing on the generic enums.
     
  12. Gladyon

    Gladyon

    Joined:
    Sep 10, 2015
    Posts:
    389
    I'm sorry, I can't really help you on that one.
    I started to code more about 35 years ago, even before I got access to a BBS with a 9600 bauds modem, before the Internet era... so I had to learn by myself, without any forum or friends who knew how to code.

    Also I do not know C#, or any other language, thoroughly. There are always more to know, it's an endless task.
    In addition, knowing how the compiler work can be important in some situation, as can be how the CPU works, or the RAM, etc.


    My best advice would be for you to be curious about everything.
    When you see some bit of code which corresponds to your needs in a forum, do not just copy/paste it. You have to understand it, dissect every line until you're confident you know the purpose of every line.
    It takes time, as I said you'll never stop learning things, but the best way to know things is to understand them completely.

    In order to do that, I usually do not use copy/paste. I write it down character by character, line by line, and if there's a line which has no meaning to me, I just do not copy it, to see what happen.
    Usually it doesn't work, so I look into it with the debugger, and very often the meaning of the line/instruction becomes clear, or at least some of the fog disappear.
     
    the_real_ijed and FeastSC2 like this.
  13. Suddoha

    Suddoha

    Joined:
    Nov 9, 2013
    Posts:
    2,824
    Without the new enum constraint, one could try the following:

    Enumerations in C# are always based on integer types.

    So that one thing that needs the workaround is getting that underlying value of any enumeration type generically into an actual integer representation, so that we can benefit from all the operators.
    As we figured out, there's no way to use usual casting approaches without additional type checks, boxing etc, but thinking on a lower level as you'd do in C++, we can leverage some features that are available in C# as well, at least to some extent: Unions in C++, or explicit struct layouts in C#.

    That is, we set the offset of an enumeration field to the same as a 64bit wide integer field, in order to read/write the values from/to the same location. (As the set of unsigned long values contains all the potential enumeration values, i.e. the enum's set is much smaller, the unsigned long will only be writable privately - otherwise a consumer could quickly mess things up)

    The result could be this:
    (only added the stuff that gets you going, there's potentially a lot more to add like bitwise OR, other extension methods etc):
    Code (CSharp):
    1. using System;
    2. using System.Runtime.InteropServices;
    3.  
    4. public static class EnumExtensions
    5. {
    6.     [StructLayout(LayoutKind.Explicit)]
    7.     private struct EnumValue<TEnum> where TEnum : struct // could add some more interfaces to exclude more value types
    8.     {
    9.         static EnumValue()
    10.         {
    11.             if (!typeof(TEnum).IsEnum)
    12.             {
    13.                 throw new Exception("Generic type argument must be an enum.");
    14.             }
    15.         }
    16.  
    17.         [FieldOffset(0)]
    18.         private ulong _backingValue;
    19.  
    20.         [FieldOffset(0)]
    21.         public TEnum Value;
    22.  
    23.         // this one is only ever going to be used internally, and that's within the overloaded operators
    24.         private EnumValue(ulong value)
    25.         {
    26.             Value = default(TEnum);
    27.             _backingValue = value;
    28.         }
    29.  
    30.         public EnumValue(TEnum value)
    31.         {
    32.             _backingValue = 0;
    33.             Value = value;
    34.         }
    35.  
    36.         // example for the bitwise AND operator
    37.         public static EnumValue<TEnum> operator &(EnumValue<TEnum> lhs, EnumValue<TEnum> rhs)
    38.         {
    39.             return new EnumValue<TEnum>(lhs._backingValue & rhs._backingValue);
    40.         }
    41.  
    42.         public static implicit operator ulong(EnumValue<TEnum> rhs)
    43.         {
    44.             return rhs._backingValue;
    45.         }
    46.     }
    47.  
    48.     public static bool HasFlagOf<TValue>(this TValue value, TValue flag) where TValue : struct
    49.     {
    50.         var lhs = new EnumValue<TValue>() { Value = value };
    51.         var rhs = new EnumValue<TValue>() { Value = flag };
    52.         return (lhs & rhs) != 0ul;
    53.     }
    54. }
    Note that this allows to write and possibly compile code with any value type (if you add some more interfaces as constraint you can reduce the number of types again).
    The static initialization block will throw at runtime when an invalid generic type is used (you could extend this to support other numeric types if you wanted to).

    I left out the other operators and potential extension methods, as that will be easy to add.
    Might rename everything, but it's just a quick example.
     
    Last edited: Jan 22, 2019
  14. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    @Suddoha You're code compiles but throws runtime exceptions because generic types can't have an Explicit layout.
    This implementation is simpler, also GC.Free, should still be burst compilable, and when profiled in editor, 10x faster than .HasFlag:
    Code (CSharp):
    1. using System;
    2. using Unity.Collections.LowLevel.Unsafe;
    3. static class EnumTools
    4. {
    5.     static uint NumericType<TEnum>(TEnum t) where TEnum : struct, Enum
    6.     {
    7.         unsafe
    8.         {
    9.             void* ptr = UnsafeUtility.AddressOf(ref t);
    10.             return *((uint*)ptr);
    11.         }
    12.     }
    13.     static ulong NumericTypeULong<TEnum>(TEnum t) where TEnum : struct, Enum
    14.     {
    15.         unsafe
    16.         {
    17.             void* ptr = UnsafeUtility.AddressOf(ref t);
    18.             return *((ulong*)ptr);
    19.         }
    20.     }
    21.     static byte NumericTypeByte<TEnum>(TEnum t) where TEnum : struct, Enum
    22.     {
    23.         unsafe
    24.         {
    25.             void* ptr = UnsafeUtility.AddressOf(ref t);
    26.             return *((byte*)ptr);
    27.         }
    28.     }
    29.     static int s_UIntSize = UnsafeUtility.SizeOf<uint>();
    30.     static int s_ULongSize = UnsafeUtility.SizeOf<ulong>();
    31.     static int s_ByteSize = UnsafeUtility.SizeOf<byte>();
    32.     public static bool HasFlagUnsafe<TEnum>(TEnum lhs, TEnum rhs) where TEnum : struct, Enum
    33.     {
    34.         int size = UnsafeUtility.SizeOf<TEnum>();
    35.         if (size == s_UIntSize)
    36.         {
    37.             return (NumericType(lhs) & NumericType(rhs)) > 0;
    38.         }
    39.         else if (size == s_ULongSize)
    40.         {
    41.             return (NumericTypeULong(lhs) & NumericTypeULong(rhs)) > 0;
    42.         }
    43.         else if (size == s_ByteSize)
    44.         {
    45.             return (NumericTypeByte(lhs) & NumericTypeByte(rhs)) > 0;
    46.         }
    47.         throw new Exception("No matching conversion function found for an Enum of size: " + size);
    48.     }
    49. }
    Note: you could get around the size checks by leaving it to the caller to specify the type. In my benchmark however, the size check cost is negligible so I'd go with the safe option.

    Also Note: you don't actually need the Enum constraints so if you're on .Net 3.5 you can just remove it.

    Thanks to @alexrvn for the unsafe magic :D
     
    Last edited: Jan 24, 2019
    Suddoha and xVergilx like this.
  15. Suddoha

    Suddoha

    Joined:
    Nov 9, 2013
    Posts:
    2,824
    @MartinTilo

    That's weird, it compiles and runs just fine for me in 2018.1. I've ran it myself to ensure it works.

    But yea, I get why it wouldn't be allowed, just looked it up.

    The unsafe one looks great. Thanks for sharing. :)
     
    Last edited: Jan 24, 2019
  16. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
    Running on Mono? I've seen Mono have bugs where stuff that's not permitted by the C# specs compiles and runs.
     
    Suddoha likes this.
  17. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    @Suddoha Using 2018.3 with .Net 4.x I get:

    Code (CSharp):
    1. Error while loading types from assembly Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null: System.Reflection.ReflectionTypeLoadException: Exception of type 'System.Reflection.ReflectionTypeLoadException' was thrown.
    2.   at (wrapper managed-to-native) System.Reflection.Assembly.GetTypes(System.Reflection.Assembly,bool)
    3.   at System.Reflection.Assembly.GetTypes () [0x00000] in <ac823e2bb42b41bda67924a45a0173c3>:0
    4.   at UnityEngine.Experimental.UIElements.VisualElementFactoryRegistry.DiscoverFactories () [0x0006d] in C:\buildslave\unity\build\Modules\UIElements\UXML\VisualElementFactoryRegistry.cs:54
    5.  
    6. EnumExtensions/EnumValue`1 : Generic class cannot have explicit layout.
    7.  
    8. UnityEditor.EditorAssemblies:ProcessInitializeOnLoadAttributes()
    and

    Code (CSharp):
    1. TypeLoadException: Generic Type Definition failed to init, due to: Generic class cannot have explicit layout. assembly:C:\Users\martinsch\Documents\Projects\AdventureGame\Library\ScriptAssemblies\Assembly-CSharp.dll type:EnumValue`1 member:(null) signature:<none>
    2. EnumTestScript.Update () (at Assets/Scripts/EnumTestScript.cs:54)
    3.  
    it does work with .Net 3.5 and performance is quite similar to the unsafe solution.
     
    Suddoha likes this.
  18. Suddoha

    Suddoha

    Joined:
    Nov 9, 2013
    Posts:
    2,824
    Oh I see. Good catch both of you @Baste and @MartinTilo.

    That might explain it. Once you told me it does throw at runtime, my first thought was that it could be the other way around.

    Just re-checked the project I used and it's got
    Scripting Runtime Version: .Net 3.5 Equivalent
    Scripting Backend: Mono

    Always great to learn more about such specific cases and "features". Thanks again.
     
    Last edited: Jan 24, 2019
    MartinTilo likes this.
  19. OndrejP

    OndrejP

    Joined:
    Jul 19, 2017
    Posts:
    303
    Would it be possible to use UnsafeUtility to copy value from enum to ulong without branching?

    EDIT:
    It seems to me you should be able to use it like this:

    Code (CSharp):
    1. ulong val = 0;
    2. UnsafeUtility.CopyStructureToPtr(ref enumValue, &val);
    Then you can easily do any bit operations and if needed write it back using similar method.
    Even endianness does not matter as long as you need only bit operations.
     
    Last edited: Mar 8, 2019
  20. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Yep that works.
    But if we're already getting rid of the branches, let's also get rid of the copies ;)

    Code (CSharp):
    1. public static class EnumTool
    2. {
    3.     public static bool HasFlagUnsafe<TEnum>(TEnum lhs, TEnum rhs) where TEnum :
    4. #if CSHARP_7_3_OR_NEWER
    5.     unmanaged, Enum
    6. #else
    7.     struct
    8. #endif
    9.     {
    10.         unsafe
    11.         {
    12. #if CSHARP_7_3_OR_NEWER
    13.             return (*(ulong*)(&lhs) & *(ulong*)(&rhs)) > 0;
    14. #else
    15.             ulong valLhs = 0;
    16.             UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    17.             ulong valRhs = 0;
    18.             UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    19.             return (valLhs & valRhs) > 0;
    20. #endif
    21.         }
    22.     }
    23. }
    Edit: The initial snippet did not show that TEnum was constraint to unmanaged, or that this only works with C# >7.3
     
    Last edited: Mar 8, 2019
    xVergilx likes this.
  21. OndrejP

    OndrejP

    Joined:
    Jul 19, 2017
    Posts:
    303
    The reason for copy was to NOT read other memory accidentally.
    CopyStructureToPtr, copies size of TEnum whatever size it is and writes it to ulong.
    For TEnum which is based on int, it overwrites half of the bits, the other half remains empty (was set to zero).

    Doesn't your solution read "other" memory in case TEnum is smaller than ulong?
    And if it does, doesn't it use it in comparison as well?
     
  22. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Good catch!

    Alright, revised edition:

    Code (CSharp):
    1.  
    2.         public static bool HasFlagUnsafe<TEnum>(TEnum lhs, TEnum rhs) where TEnum :
    3. #if CSHARP_7_3_OR_NEWER
    4.             unmanaged, Enum
    5. #else
    6.             struct
    7. #endif
    8.         {
    9.  
    10.             unsafe
    11.             {
    12. #if CSHARP_7_3_OR_NEWER
    13.                 switch (sizeof(TEnum))
    14.                 {
    15.                     case 1:
    16.                         return (*(byte*)(&lhs) & *(byte*)(&rhs)) > 0;
    17.                     case 2:
    18.                         return (*(ushort*)(&lhs) & *(ushort*)(&rhs)) > 0;
    19.                     case 4:
    20.                         return (*(uint*)(&lhs) & *(uint*)(&rhs)) > 0;
    21.                     case 8:
    22.                         return (*(ulong*)(&lhs) & *(ulong*)(&rhs)) > 0;
    23.                     default:
    24.                         throw new Exception("Size does not match a known Enum backing type.");
    25.                 }
    26.  
    27. #else
    28.                 switch (UnsafeUtility.SizeOf<TEnum>())
    29.                 {
    30.                     case 1:
    31.                         {
    32.                             byte valLhs = 0;
    33.                             UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    34.                             byte valRhs = 0;
    35.                             UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    36.                             return (valLhs & valRhs) > 0;
    37.                         }
    38.                     case 2:
    39.                         {
    40.                             ushort valLhs = 0;
    41.                             UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    42.                             ushort valRhs = 0;
    43.                             UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    44.                             return (valLhs & valRhs) > 0;
    45.                         }
    46.                     case 4:
    47.                         {
    48.                             uint valLhs = 0;
    49.                             UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    50.                             uint valRhs = 0;
    51.                             UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    52.                             return (valLhs & valRhs) > 0;
    53.                         }
    54.                     case 8:
    55.                         {
    56.                             ulong valLhs = 0;
    57.                             UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    58.                             ulong valRhs = 0;
    59.                             UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    60.                             return (valLhs & valRhs) > 0;
    61.                         }
    62.                     default:
    63.                         throw new Exception("Size does not match a known Enum backing type.");
    64.                 }
    65. #endif
    66.             }
    67.         }
    68.  
    Yes, I've brought back the branching to the copy approach, because I did some profiling.
    Setup: 2019.2a9 Mono in the Editor (I know, should've made a build) with Editor Attaching disabled in the preferences.
    In an Update method I ran 40000 iterations of loops such as this one.

    Code (CSharp):
    1.  
    2.     ProfilerMarker HasFlagUnsafe = new ProfilerMarker("HasFlagUnsafe");
    3.     [Flags]
    4.     public enum Test
    5.     {
    6.         Bla = 1<<0,
    7.         Blub = 1 << 1,
    8.         Blub2 = 1 << 2,
    9.         All = Bla | Blub | Blub2,
    10.     }
    11.  
    12.         //...
    13.  
    14.         using (HasFlagUnsafe.Auto())
    15.         {
    16.             for (int i = 0; i < iterations; i++)
    17.             {
    18.                 EnumTool.HasFlagUnsafe(Test.All, Test.Bla);
    19.                 EnumTool.HasFlagUnsafe(Test.Blub, Test.Bla);
    20.             }
    21.         }
    I did 2 runs of this captured within the 300 frames of one profiler capture, then took >140 frames from each run and compared them in Profile Analyzer.
    In the first run, I compared the
    CSHARP_7_3_OR_NEWER
    approach against just

    Code (CSharp):
    1.                             ulong valLhs = 0;
    2.                             UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    3.                             ulong valRhs = 0;
    4.                             UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    5.                             return (valLhs & valRhs) > 0;
    2019-04-12_09-41-25.png
    In the second run I added the switch to the copy comparison:
    2019-04-12_09-41-57.png
    As you can see, the switches don't seem to matter that much, or at least not in this particular setup. What does hurt is using the copy approach for anything that is not of the size of ulong. Also, ulong backed Enums generally perform best and the non-copying approach is also the best one I've seen so far, though only possible with C# 7.3+.
    What might still be interesting would be to throw this at Burst...
     
    Alic, myloran, quabug and 1 other person like this.
  23. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    FYI, I had informed the VM team about this and it has now been fixed in Unity's version of Mono (and IL2CPP). It's been a while since it was fixed so I can't currently recall which versions exactly but: if in doubt, profile first, then use the workaround if needed.
     
    FeastSC2 likes this.
  24. jGate99

    jGate99

    Joined:
    Oct 22, 2013
    Posts:
    1,935
    Thanks, what about Clear/Remove Flag and addFlag?
     
  25. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Afaik these aren't part of the existing .Net APIs and easily covered by bit-wise operators. I don't think the function call overhead (yes we could strongly suggest that the compiler inline these), nor the size check overhead would be worth it to add explicit APIs for these?

    Anyways it's:
    Code (CSharp):
    1. flags = (MyFlagType)0; // Clear all Flags
    2. flags |= MyFlagType.Flag1; // Add Flag
    3. flags &= ~MyFlagType.Flag1; // Remove Flag
    Maybe the IDEs should have snippets for these so one doesn't have to remember how bit wise operators work every time this is needed? Not something I think we should (or meaningfully can) be doing anything about via our Mono/IL2CPP implementations though, as opposed to HasFlags which is indeed a tad cumbersome to write in bit-wise operators and already existed as API.
     
    Alic and jGate99 like this.
  26. jGate99

    jGate99

    Joined:
    Oct 22, 2013
    Posts:
    1,935
    I was actually looking for some generic extension methods which we could use in Unity's C# 7.3 (generic constraint)
    I found few online but they use Span which is not in unity
     
  27. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    2,431
    Yeah, the ones I saw were also generating allocations as if they came for free. Here, this should do it:
    (It doesn't alloc but I didn't compare performance as I did with the HasFlag solution.)

    Code (CSharp):
    1. using System;
    2. using System.Runtime.CompilerServices;
    3. #if !CSHARP_7_3_OR_NEWER
    4. using Unity.Collections.LowLevel.Unsafe;
    5. #endif
    6.  
    7. public static class EnumFlagExtensions
    8. {
    9. #if CSHARP_7_3_OR_NEWER
    10.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    11. #else
    12.     [MethodImpl((MethodImplOptions)256)]
    13. #endif
    14.     public static TEnum AddFlag<TEnum>(this TEnum lhs, TEnum rhs) where TEnum :
    15. #if CSHARP_7_3_OR_NEWER
    16.                 unmanaged, Enum
    17. #else
    18.                 struct
    19. #endif
    20.     {
    21.  
    22.         unsafe
    23.         {
    24. #if CSHARP_7_3_OR_NEWER
    25.             switch (sizeof(TEnum))
    26.             {
    27.                 case 1:
    28.                     {
    29.                         var r = *(byte*)(&lhs) | *(byte*)(&rhs);
    30.                         return *(TEnum*)&r;
    31.                     }
    32.                 case 2:
    33.                     {
    34.                         var r = *(ushort*)(&lhs) | *(ushort*)(&rhs);
    35.                         return *(TEnum*)&r;
    36.                     }
    37.                 case 4:
    38.                     {
    39.                         var r = *(uint*)(&lhs) | *(uint*)(&rhs);
    40.                         return *(TEnum*)&r;
    41.                     }
    42.                 case 8:
    43.                     {
    44.                         var r = *(ulong*)(&lhs) | *(ulong*)(&rhs);
    45.                         return *(TEnum*)&r;
    46.                     }
    47.                 default:
    48.                     throw new Exception("Size does not match a known Enum backing type.");
    49.             }
    50.  
    51. #else
    52.                  
    53.             switch (UnsafeUtility.SizeOf<TEnum>())
    54.             {
    55.                 case 1:
    56.                     {
    57.                         byte valLhs = 0;
    58.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    59.                         byte valRhs = 0;
    60.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    61.                         var result = (valLhs | valRhs);
    62.                         void * r = &result;
    63.                         TEnum o;
    64.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    65.                         return o;
    66.                     }
    67.                 case 2:
    68.                     {
    69.                         ushort valLhs = 0;
    70.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    71.                         ushort valRhs = 0;
    72.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    73.                         var result = (valLhs | valRhs);
    74.                         void* r = &result;
    75.                         TEnum o;
    76.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    77.                         return o;
    78.                     }
    79.                 case 4:
    80.                     {
    81.                         uint valLhs = 0;
    82.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    83.                         uint valRhs = 0;
    84.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    85.                         var result = (valLhs | valRhs);
    86.                         void* r = &result;
    87.                         TEnum o;
    88.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    89.                         return o;
    90.                     }
    91.                 case 8:
    92.                     {
    93.                         ulong valLhs = 0;
    94.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    95.                         ulong valRhs = 0;
    96.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    97.                         var result = (valLhs | valRhs);
    98.                         void* r = &result;
    99.                         TEnum o;
    100.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    101.                         return o;
    102.                     }
    103.                 default:
    104.                     throw new Exception("Size does not match a known Enum backing type.");
    105.             }
    106. #endif
    107.         }
    108.     }
    109.  
    110.  
    111. #if CSHARP_7_3_OR_NEWER
    112.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    113. #else
    114.     [MethodImpl((MethodImplOptions)256)]
    115. #endif
    116.     public static TEnum RemoveFlag<TEnum>(this TEnum lhs, TEnum rhs) where TEnum :
    117. #if CSHARP_7_3_OR_NEWER
    118.                 unmanaged, Enum
    119. #else
    120.                 struct
    121. #endif
    122.     {
    123.  
    124.         unsafe
    125.         {
    126. #if CSHARP_7_3_OR_NEWER
    127.             switch (sizeof(TEnum))
    128.             {
    129.                 case 1:
    130.                     {
    131.                         var r = *(byte*)(&lhs) & ~*(byte*)(&rhs);
    132.                         return *(TEnum*)&r;
    133.                     }
    134.                 case 2:
    135.                     {
    136.                         var r = *(ushort*)(&lhs) & ~*(ushort*)(&rhs);
    137.                         return *(TEnum*)&r;
    138.                     }
    139.                 case 4:
    140.                     {
    141.                         var r = *(uint*)(&lhs) & ~*(uint*)(&rhs);
    142.                         return *(TEnum*)&r;
    143.                     }
    144.                 case 8:
    145.                     {
    146.                         var r = *(ulong*)(&lhs) & ~*(ulong*)(&rhs);
    147.                         return *(TEnum*)&r;
    148.                     }
    149.                 default:
    150.                     throw new Exception("Size does not match a known Enum backing type.");
    151.             }
    152.  
    153. #else
    154.                  
    155.             switch (UnsafeUtility.SizeOf<TEnum>())
    156.             {
    157.                 case 1:
    158.                     {
    159.                         byte valLhs = 0;
    160.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    161.                         byte valRhs = 0;
    162.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    163.                         var result = (valLhs & ~valRhs);
    164.                         void * r = &result;
    165.                         TEnum o;
    166.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    167.                         return o;
    168.                     }
    169.                 case 2:
    170.                     {
    171.                         ushort valLhs = 0;
    172.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    173.                         ushort valRhs = 0;
    174.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    175.                         var result = (valLhs & ~valRhs);
    176.                         void* r = &result;
    177.                         TEnum o;
    178.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    179.                         return o;
    180.                     }
    181.                 case 4:
    182.                     {
    183.                         uint valLhs = 0;
    184.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    185.                         uint valRhs = 0;
    186.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    187.                         var result = (valLhs & ~valRhs);
    188.                         void* r = &result;
    189.                         TEnum o;
    190.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    191.                         return o;
    192.                     }
    193.                 case 8:
    194.                     {
    195.                         ulong valLhs = 0;
    196.                         UnsafeUtility.CopyStructureToPtr(ref lhs, &valLhs);
    197.                         ulong valRhs = 0;
    198.                         UnsafeUtility.CopyStructureToPtr(ref rhs, &valRhs);
    199.                         var result = (valLhs & ~valRhs);
    200.                         void* r = &result;
    201.                         TEnum o;
    202.                         UnsafeUtility.CopyPtrToStructure(r, out o);
    203.                         return o;
    204.                     }
    205.                 default:
    206.                     throw new Exception("Size does not match a known Enum backing type.");
    207.             }
    208. #endif
    209.         }
    210.  
    211.     }
    212.  
    213. #if CSHARP_7_3_OR_NEWER
    214.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    215.     public static void SetFlag<TEnum>(ref this TEnum lhs, TEnum rhs) where TEnum : unmanaged, Enum
    216.     {
    217.  
    218.         unsafe
    219.         {
    220.             fixed (TEnum* lhs1 = &lhs)
    221.             {
    222.                 switch (sizeof(TEnum))
    223.                 {
    224.                     case 1:
    225.                         {
    226.                             var r = *(byte*)(lhs1) | *(byte*)(&rhs);
    227.                             *lhs1 = *(TEnum*)&r;
    228.                             return;
    229.                         }
    230.                     case 2:
    231.                         {
    232.                             var r = *(ushort*)(lhs1) | *(ushort*)(&rhs);
    233.                             *lhs1 = *(TEnum*)&r;
    234.                             return;
    235.                         }
    236.                     case 4:
    237.                         {
    238.                             var r = *(uint*)(lhs1) | *(uint*)(&rhs);
    239.                             *lhs1 = *(TEnum*)&r;
    240.                             return;
    241.                         }
    242.                     case 8:
    243.                         {
    244.                             var r = *(ulong*)(lhs1) | *(ulong*)(&rhs);
    245.                             *lhs1 = *(TEnum*)&r;
    246.                             return;
    247.                         }
    248.                     default:
    249.                         throw new Exception("Size does not match a known Enum backing type.");
    250.                 }
    251.             }
    252.         }
    253.     }
    254.  
    255.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    256.     public static void ClearFlag<TEnum>(this ref TEnum lhs, TEnum rhs) where TEnum : unmanaged, Enum
    257.     {
    258.  
    259.         unsafe
    260.         {
    261.             fixed (TEnum* lhs1 = &lhs)
    262.             {
    263.                 switch (sizeof(TEnum))
    264.                 {
    265.                     case 1:
    266.                         {
    267.                             var r = *(byte*)(lhs1) & ~*(byte*)(&rhs);
    268.                             *lhs1 = *(TEnum*)&r;
    269.                             return;
    270.                         }
    271.                     case 2:
    272.                         {
    273.                             var r = *(ushort*)(lhs1) & ~*(ushort*)(&rhs);
    274.                             *lhs1 = *(TEnum*)&r;
    275.                             return;
    276.                         }
    277.                     case 4:
    278.                         {
    279.                             var r = *(uint*)(lhs1) & ~*(uint*)(&rhs);
    280.                             *lhs1 = *(TEnum*)&r;
    281.                             return;
    282.                         }
    283.                     case 8:
    284.                         {
    285.                             var r = *(ulong*)(lhs1) & ~*(ulong*)(&rhs);
    286.                             *lhs1 = *(TEnum*)&r;
    287.                             return;
    288.                         }
    289.                     default:
    290.                         throw new Exception("Size does not match a known Enum backing type.");
    291.                 }
    292.             }
    293.         }
    294.     }
    295. #endif
    296. }
    297.  
    AddFlag and RemoveFlag don't change the value of the enum variable they where called on but just return the new value, SetFlag and ClearFlag modify the value directly (which doesn't work pre C# 7.3). So you can choose whichever flavor of this suits you best ;)
     
    Ultroman and jGate99 like this.
  28. jGate99

    jGate99

    Joined:
    Oct 22, 2013
    Posts:
    1,935
    Thank you very much, this is gonna solve a lot of time :D
     
    MartinTilo likes this.
  29. OndrejP

    OndrejP

    Joined:
    Jul 19, 2017
    Posts:
    303
    I was surprised that switch-statement was that fast and decided to test it myself.

    upload_2022-5-8_13-4-45.png

    Legend:
    CopyPtrToStructure - copies enum to ulong with
    UnsafeUtility.CopyStructureToPtr

    CopyUnmanaged - copies enum to ulong with
    ulong uValue = 0; *(TEnum*)&uValue = enumValue;

    Switch - uses above code by @MartinTilo
    SwitchNoThrow - same as switch, but returns false for unknown sizes instead of throwing exception
    If - similar to SwitchNoThrow, but uses
    If
    and
    Else if

    Direct - classic old non-generic way wrapped in generic method, each size requires different method
    Raw - classic old non-generic way
    (value & flag) > 0


    Raw and Direct are merely for comparison
    Tested on Release, all methods have
    [MethodImpl(MethodImplOptions.AggressiveInlining)]


    Analysis:
    After running these tests, I was even more surprised that branching version is a lot faster than switch version.
    It seems that JIT can completely optimize-out other branches (performance is comparable to Raw).
    For some reason JIT can't do that for switch, I'm not sure why. Maybe it sees Switch OpCode and generates jump table without realizing it can be optimized to a single case?

    Another interesting point is ULong_CopyUnmanaged, which has same performance as Raw. It's likely that JIT removes redundant copying (copying ulong to ulong).

    Lastly Burst - it simply does not care :D
    Having full type information during compilation, it optimizes the hell out of it and even vectorizes my testing loops.

    Summary:
    Branching version seems to be the fastest.
    Can someone confirm these results?
    @MartinTilo