Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

how to reset NativeArrays or dynamic buffers ?

Discussion in 'Entity Component System' started by Andreas88, Dec 9, 2018.

  1. Andreas88

    Andreas88

    Joined:
    Dec 22, 2015
    Posts:
    67
    hi there. sorry if this sounds like a very noob question.
    are there any efficient ways to reset the values on an dynamic buffer or at least copy an NativeArray into the buffer ?
    i tried to make a parallel job that modifies the buffer values to the reset value. it works but is very slow (about 200 ms).

    I thought i would mention that i need to reset my buffer values every frame, because i use it in a texture.
    thank you :)
     
  2. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,753
    Why you reset? Can you not simply override with new values?
     
  3. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,555
    I think it is nativeArray.GetUnsafePtr then use that with UnsafeUtility.MemClear. Requires unsafe asmdef

    He might be doing "clear camera"-like logic? (After a faster clear then you can write on only some area)
     
  4. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,753
    Maybe. But we can speculate, until OP explain, what exactly is needed for.
     
  5. Andreas88

    Andreas88

    Joined:
    Dec 22, 2015
    Posts:
    67
    hi there :)
    just to clarify what my intention is.
    i am doing some rts style player movement using flowfields. basically i have an flattened 2d array (dyn buffer) with int as value. so right now i am creating a sort of repel map, so all array indexes with an high number is a player unit and the rest has a zero value. the idea behind this logic is to create different maps that combined will drive a flow field.

    the reason i want to reset my repel map (dynamic buffer), is because every time an player unit moves i need to reset the value of the last position.

    if anyone has a better idea instead of resetting the entire array, please share your thoughts.
     
  6. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,752
    It's very expensive to loop a large array to default all values. I first ran into the need to do this when reusing arrays in pathfinding where the array contained the cost to finish. But yeah memclear is the best way. I just wrote an extension method to hide the unsafe code.
     
    DragonCoder likes this.
  7. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,753
    I am aware of the performance hit.
    But my point was, if I don't need reset, I can simply override with new data.
    But as explained above, this may not necessary be an option.
    I assume, for NativeArray dispose is fast.

    I use a lot of arrays. I don't clear them, but reuse them.
    Just having index / pointer, where last (or relevant) data have been accessed, wrote, etc.
     
  8. PeterJK

    PeterJK

    Joined:
    Dec 2, 2012
    Posts:
    35
    Anyone have any suggestions for fast clearing of Hashmaps/MultiHashMaps? With large hashmaps it can get fairly slow. There doesn't seem to be a GetUnsafePtr equivalent. Thanks for any tips.
     
  9. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,752
    NativeHashMap already has a built in Clear() method. Disposing and creating doesn't make much sense because the constructor calls Clear() anyway.

    Looking at source, it shouldn't be that slow but I can see why if you have a huge capacity it might become noticeable (as it loops the entire length.)

    Code (CSharp):
    1.         public static unsafe void Clear(NativeHashMapData* data)
    2.         {
    3.             int* buckets = (int*) data->buckets;
    4.             for (int i = 0; i <= data->bucketCapacityMask; ++i)
    5.                 buckets[i] = -1;
    6.             int* nextPtrs = (int*) data->next;
    7.             for (int i = 0; i < data->capacity; ++i)
    8.                 nextPtrs[i] = -1;
    9.             for (int tls = 0; tls < JobsUtility.MaxJobThreadCount; ++tls)
    10.                 data->firstFreeTLS[tls * NativeHashMapData.IntsPerCacheLine] = -1;
    11.             data->allocatedIndexLength = 0;
    12.         }
    Memclearing won't help because as you can see the default value of the native hash map is not 0, it's -1.

    And this is set in the constructor by calling Clear()

    Code (CSharp):
    1. public NativeHashMap(int capacity, Allocator label)
    2.         {
    3.             m_AllocatorLabel = label;
    4.             // Bucket size if bigger to reduce collisions
    5.             NativeHashMapData.AllocateHashMap<TKey, TValue>(capacity, capacity * 2, label, out m_Buffer);
    6.  
    7. #if ENABLE_UNITY_COLLECTIONS_CHECKS
    8. #if UNITY_2018_3_OR_NEWER
    9.             DisposeSentinel.Create(out m_Safety, out m_DisposeSentinel, 0, label);
    10. #else
    11.             DisposeSentinel.Create(out m_Safety, out m_DisposeSentinel, 0);
    12. #endif
    13. #endif
    14.  
    15.             Clear();
    16.         }
     
    Last edited: Dec 13, 2018
  10. PeterJK

    PeterJK

    Joined:
    Dec 2, 2012
    Posts:
    35
    Thanks tertle! I'll see if I can dig into the sourcecode to find a faster way to clear, based on some assumptions around my use case.
     
  11. AllFatherGray

    AllFatherGray

    Joined:
    Nov 29, 2014
    Posts:
    17
    Sorry to necro but may I see this implementation?
     
  12. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,677
    Since I stumbled across this thread searching the same, here how I've done it:

    Code (CSharp):
    1.         unsafe void ClearArray<T>(NativeArray<T> to_clear, int length) where T : struct
    2.         {
    3.             UnsafeUtility.MemClear(
    4.                 NativeArrayUnsafeUtility.GetUnsafeBufferPointerWithoutChecks(to_clear),
    5.                 UnsafeUtility.SizeOf<T>() * length);
    6.         }
    Note that you need to enable unsafe compilation for that assembly first.
    And naturally be absolutely sure that you provide the correct length! Ideally just with to_clear.length because "unsafe" means there is little to no protection against malicious writes that crash your software.
    Also only use this when you have write-access to the data. Not with ReadOnly containers or [ReadOnly] marked members of Burst jobs (if you are unsure, use the minimally slower method without the "WithoutChecks" part).
     
    Last edited: Apr 30, 2022
  13. AllFatherGray

    AllFatherGray

    Joined:
    Nov 29, 2014
    Posts:
    17
    Thanks! Appreciate it!
     
  14. mbaker

    mbaker

    Joined:
    Jan 9, 2013
    Posts:
    52
    Curious if you've tested this vs using the Unity.Entities.MemsetNativeArray<T> job. I have to imagine your implementation is faster but am curious what the speed up is.
     
  15. iamarugin

    iamarugin

    Joined:
    Dec 17, 2014
    Posts:
    880
    Don't understand why this method is not implemented for native array.
     
  16. mahdiii

    mahdiii

    Joined:
    Oct 30, 2014
    Posts:
    856
    Weird, yes