Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Enter the 2020.2 Beta Sweepstakes for a chance to win an Oculus Quest 2.
    Dismiss Notice

NativeArray<DynamicComponentTypeHandle> not possible?

Discussion in 'Data Oriented Technology Stack' started by Guedez, Oct 16, 2020 at 3:01 PM.

  1. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    520
    Is it a version thing or I just can't? I was about to praise
    DynamicComponentTypeHandle 
    as the ultimate solution to replacing all of my Reflection code, I can even
    Chunk.GetDynamicComponentDataArrayReinterpret
    which is insanely powerful. But then, why can't I make a
    NativeArray<DynamicComponentTypeHandle>
    ?

    it throws
    InvalidOperationException: Unity.Entities.DynamicComponentTypeHandle used in NativeArray<Unity.Entities.DynamicComponentTypeHandle> must be unmanaged (contain no managed types) and cannot itself be a native container type.


    EDIT: Updated all packages and still can't.

    My packages:
    Code (CSharp):
    1.  
    2. {
    3.   "dependencies": {
    4.     "com.unity.2d.sprite": "1.0.0",
    5.     "com.unity.animation.rigging": "0.3.4-preview",
    6.     "com.unity.burst": "1.3.9",
    7.     "com.unity.cinemachine": "2.6.3",
    8.     "com.unity.collections": "0.8.0-preview.5",
    9.     "com.unity.dots.editor": "0.10.0-preview",
    10.     "com.unity.editorcoroutines": "1.0.0",
    11.     "com.unity.entities": "0.14.0-preview.19",
    12.     "com.unity.ide.visualstudio": "2.0.3",
    13.     "com.unity.jobs": "0.2.9-preview.15",
    14.     "com.unity.mathematics": "1.1.0",
    15.     "com.unity.performance.profile-analyzer": "1.0.3",
    16.     "com.unity.platforms": "0.5.0-preview.6",
    17.     "com.unity.quicksearch": "1.5.4",
    18.     "com.unity.rendering.hybrid": "0.8.0-preview.19",
    19.     "com.unity.scriptablebuildpipeline": "1.7.3",
    20.     "com.unity.searcher": "4.3.1",
    21.     "com.unity.test-framework": "1.1.16",
    22.     "com.unity.test-framework.performance": "2.2.0-preview",
    23.     "com.unity.textmeshpro": "3.0.1",
    24.     "com.unity.timeline": "1.3.5",
    25.     "com.unity.ui": "1.0.0-preview.6",
    26.     "com.unity.ui.builder": "1.0.0-preview.3",
    27.     "nuget.mono-cecil": "0.1.6-preview",
    28.     "com.unity.modules.ai": "1.0.0",
    29.     "com.unity.modules.animation": "1.0.0",
    30.     "com.unity.modules.assetbundle": "1.0.0",
    31.     "com.unity.modules.audio": "1.0.0",
    32.     "com.unity.modules.director": "1.0.0",
    33.     "com.unity.modules.imageconversion": "1.0.0",
    34.     "com.unity.modules.imgui": "1.0.0",
    35.     "com.unity.modules.jsonserialize": "1.0.0",
    36.     "com.unity.modules.particlesystem": "1.0.0",
    37.     "com.unity.modules.physics": "1.0.0",
    38.     "com.unity.modules.physics2d": "1.0.0",
    39.     "com.unity.modules.ui": "1.0.0",
    40.     "com.unity.modules.uielements": "1.0.0",
    41.     "com.unity.modules.umbra": "1.0.0",
    42.     "com.unity.modules.vr": "1.0.0",
    43.     "com.unity.modules.wind": "1.0.0",
    44.     "com.unity.modules.xr": "1.0.0"
    45.   }
    46. }
    47.  
    48.  
     
    Last edited: Oct 16, 2020 at 3:53 PM
  2. brunocoimbra

    brunocoimbra

    Joined:
    Sep 2, 2015
    Posts:
    461
    DynamicComponentTypeHandle should be used in the same way that ComponentTypeHandle<T> is used
     
  3. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    391
    DynamicComponentTypeHandle uses SafetyHandle internally so you cannot put it in a NativeArray.
    But if you just want a unique type ID. You can simply use TypeIndex which is int.

    int typeIndex=TypeManager.GetTypeIndex<Translation>();

    TypeManager and ComponentType works well on TypeIndex.
     
  4. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    520
    I wanted to access an arbitrary amount of values from a chunk, which and how many of those are read from a XML during runtime. Currently I am running single threaded a bunch of jobs in sequence, each using it's own DynamicComponentTypeHandle. It works, but could be so much better.

    I can create a EntityQuery from an arbitrary number of runtime-decided types, but I can't do the same to access data from chunks without concatenating a bunch of jobs that each reads/writes a single type? I guess it still beats using reflection, but it frustrates me to be so close yet so far.

    If there is absolutely no way to send an array of types and use it to read/write a chunk, then I think I will manually make a 1, 2, 4, 8 and 16 types jobs and run them as needed to 'emulate' a 31 slots array while needing to concatenate as few jobs as possible.
     
  5. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    1,767
    Hybrid Renderer V2 uses a struct of 128 DynamicComponentTypeHandle instances. It doesn't populate all of them. The job just doesn't touch the non-populated ones.
     
    Guedez likes this.
  6. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    391
    You need internal access to Unity.Entities.
    And you can do that with TypeIndex

    This codes is how I use TypeIndex to access Raw data (void *)
    Code (CSharp):
    1.         unsafe public static NativeArray<T> ToOptionalDataArray<T>(this ref EntityQuery query, Allocator allocator, T defaultValue = default)
    2.             where T : unmanaged, IComponentData
    3.         {
    4.             int totalEntityCount = query.CalculateEntityCount();
    5.             var dataOut = new NativeArray<T>(totalEntityCount, allocator, NativeArrayOptions.UninitializedMemory);
    6.             var pDst = (T*)dataOut.GetUnsafePtr();
    7.             //var dataSize = UnsafeUtility.SizeOf<T>();
    8.             var dataSize = sizeof(T);
    9.  
    10.             var entityCounter = 0;
    11.             var imp = query._GetImpl();
    12.             var filter = imp->_Filter;
    13.             var matches = imp->_QueryData->MatchingArchetypes;
    14.             var matchCount = matches.Length;
    15.             var typeIndex = TypeManager.GetTypeIndex<T>();
    16.             for (int ia = 0; ia < matchCount; ia++)
    17.             {
    18.                 var archetype = (*matches.Ptr) + ia;
    19.                 var types = (int*)archetype->Archetype->Types;
    20.                 var typesCount = archetype->Archetype->TypesCount;
    21.                 //var indexInArchetype = typeIndex.BinarySearchIn(types, typesCount);
    22.                 var indexInArchetype = NativeArrayExtensions.IndexOf<int, int>(types, typesCount, typeIndex);
    23.  
    24.                 if (indexInArchetype < 0)//Type not found in Archetype
    25.                 {
    26.                     var matchChunkCount = archetype->Archetype->Chunks.Count;
    27.                     for (int ic = 0; ic < matchChunkCount; ic++)
    28.                     {
    29.                         var chunk = archetype->Archetype->Chunks[ic];
    30.                         var entityCount = chunk->Count;
    31.                         UnsafeUtility.MemCpyReplicate(pDst, &defaultValue, dataSize, entityCount);
    32.                         entityCounter += entityCount;
    33.                     }
    34.                 }
    35.                 else//Type found in Archetype
    36.                 {
    37.                     var matchChunkCount = archetype->Archetype->Chunks.Count;
    38.                     for (int ic = 0; ic < matchChunkCount; ic++)
    39.                     {
    40.                         if (!archetype->ChunkMatchesFilter(ic, ref filter)) continue;
    41.                         var chunk = archetype->Archetype->Chunks[ic];
    42.                         var entityCount = chunk->Count;
    43.                         var pSrc = ((T*)ChunkDataUtility.GetComponentDataRO(chunk, 0, indexInArchetype)) + entityCounter;
    44.                         UnsafeUtility.MemCpy(pDst, pSrc, dataSize * entityCount);
    45.                         entityCounter += entityCount;
    46.                     }
    47.                 }
    48.             }
    49.             return dataOut;
    50.         }
    This function check if a Type exist on the quertied chunks and fill the container with the data in chunk or default value. so I can have a container of data that match query entity size(and index).
     
    Last edited: Oct 18, 2020 at 4:01 PM
    Guedez likes this.
  7. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    391
    TypeIndex is generated by TypeManager. And they are ordered and fixed at Runtime.
    you can use an offset bitmask to mark types.
    Here's how I do it.
     

    Attached Files:

    Guedez likes this.
  8. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    391
    And I have a DataBase of Data by level (blob asset loaded from Excel)
    This is how I set data by TypeIndex.
    Code (CSharp):
    1.             var entityMgrCache = EntityManager;
    2.             Entities.WithName("UpdateDataLevel")
    3.             .WithBurst()
    4.             .WithEntityQueryOptions(EntityQueryOptions.IncludeDisabled | EntityQueryOptions.IncludePrefab)
    5.             .WithChangeFilter<LeveledComponentID>()
    6.             .WithStoreEntityQueryInField(ref LevelChangeQuery)
    7.             .ForEach((Entity e, in DynamicBuffer<LeveledComponentID> IDs) =>
    8.             {
    9.                 ref var dataBlob = ref dataAsset.Value;
    10.                 for (int i = 0, len = IDs.Length; i < len; i++)
    11.                 {
    12.                     var id_Lv = IDs[i];
    13.                     var locator = dataBlob.GetLocatorByID(id_Lv.ID);
    14.                     var cType = locator.ComponentType();
    15.                     entityMgrCache.SetComponentLevelRaw(e, ref dataBlob, locator, id_Lv.Level);
    16.                 }
    17.             }).Run();
    Code (CSharp):
    1.         unsafe public static void SetComponentLevelRaw(this EntityManager em, Entity entity, ref LeveledDatabaseBlob blobData, Locator locator, int level = 1)
    2.             => em.SetComponentDataRaw(entity, locator.TypeIndex, blobData.GetDataRaw(locator, level).UnsafePtr, locator.DataSize);
    3.  
    4.         unsafe public static void SetComponentLevelRaw(this EntityManager em, Entity entity, ref LeveledDatabaseBlob blobData, int id, int level = 1)
    5.             => em.SetComponentLevelRaw(entity, ref blobData, blobData.GetLocatorByID(id), level);
    6.  
    7.         unsafe public static void SetComponentLevelRaw(this EntityManager em, Entity entity, ref LeveledDatabaseBlob blobData, FixedString32 name, int level = 1)
    8.             => em.SetComponentLevelRaw(entity, ref blobData, blobData.GetIdByName(name), level);
     
    Guedez likes this.
  9. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    391
    So basically, you just need TypeIndex.
    and TypeHandles are just typeIndex(int) + lastSystemVerion(uint) and safety handle
     
    Guedez likes this.
  10. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    520
    So can't be done without editing the Entities package? Whatever you are doing on your code seems like witchery to me, I'd need to read and understand the in and out of the Entities package to implement. Hopefully the devs let users do that without editing the packages eventually.
     
  11. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    391
    You don‘t need to edit Entities package. you just need to make them
    Code (CSharp):
    1. using System.Runtime.CompilerServices;
    2. [assembly: InternalsVisibleTo("your_dll_name")]
    By using a .asmref asset
     

    Attached Files:

    Guedez likes this.
  12. Guedez

    Guedez

    Joined:
    Jun 1, 2012
    Posts:
    520
    Man, I will keep this thread close to heart, as soon as I get some time to spend on optimization, I will dive head first on this
     
unityunity