Clearing NativeHashMaps

I'm not sure whether this is the right place for this. If not, please just let me know.

I ran into a situation where I have rather large Native(Multi)HashMap that is only filled to a fraction of its maximum capacity, then cleared, then filled again, cleared again etc.
Clearing it however took a very significant amount of time because I allocated it so large so I figured it may be a better idea to just iterate over all the keys added and remove them individually whenever only a fraction of the capacity is used.
For reasons I'm not entirely sure about yet, removing the entries this way instead of calling Clear() made adding new items later significantly slower (roughly 10x) if the total number of items added over the hashmap's lifetime exceeded its total capacity. After staring at the source code of NativeHashMap for a while, I ended up adding the following function to be called after removing all entries which did make adding as fast as if I had called Clear().

Code (CSharp):

1.  
2.         public static unsafe void ClearNextPtrs(NativeHashMapData* data)
3.         {
4.             int* nextPtrs = (int*)data->next;
5.  
6.             for (int tls = 0; tls < JobsUtility.MaxJobThreadCount; ++tls)
7.             {
8.                 int freeIdx = data->firstFreeTLS[tls * NativeHashMapData.IntsPerCacheLine];
9.                 data->firstFreeTLS[tls * NativeHashMapData.IntsPerCacheLine] = -1;
10.                 while (freeIdx >= 0)
11.                 {
12.                     int nextIdx = nextPtrs[freeIdx];
13.                     nextPtrs[freeIdx] = -1;
14.                     freeIdx = nextIdx;
15.                 }
16.             }
17.             data->allocatedIndexLength = 0;
18.         }

Could someone explain to my why exactly adding elements would be so slow after removing elements with Remove instead of Clear? How does my function fix this and is my function safe to use or did I miss something? It seems to work well in my tests so far, but this stuff is a couple of miles outside my area of expertise​