NativeHashMap

Anyone know how to iterate over a NativeHashMap, or get Keys?

I'm trying to jobify some code and converted Dictionary<int, Dictionary<int,int>> to NativeHashMap<int, MyStruct>.

 

I could also use a ContainsKey or similar method on NativeHashMap.

TryGetValue requires I declare an 'out' variable.
I've a job loop that hits this code a lot, so would would like to avoid declaring unused out vars.

 

Yup, ContainsKey would be invaluable. Also, if an alternative job is created to iterate over a NativeMultiHashMap, I think it would be useful to have a single Execute method that passes in the key along with the value.

 

Code (CSharp):

1. var exists = map.TryGetValue(key, out _);

 

Interesting, I did not now about discards. Still, it seems like ContainsKey would be a better method to use. Also, it'd be nice to have a method that returns the number of values for a given key.

public int ValueCount(int key){}

I have a very obscure use case that could make use of such a function, just adding it here in case it's something that can easily be added. If not, no big deal.

 

I need to be able to access the key during each ExecuteNext call. I thought I could store the key as the first element (maybe as a negative number in case duplicates aren't allowed, as it's possible one of the values I am storing at each key is the same as the key), and then use the firstIndex value of the ExecuteNext call to retrieve the key, but this doesn't seem to work. Is it not possible to make the first value added to a bucket the index/first index when ExecuteFirst/ExecuteNext are called? What is the logic behind which index is the first index? Is it just completely random? That is what it seems like.

Is the IJobNativeMultiHashMapMergedSharedKeyIndices code available anywhere? I can't seem to find it.

 

Thank you, but I cannot find that file either. Are they included in the Entities package or are they in a dll (The Unity.Collections one?). The latter is my understanding. While I am able to open individual c# files from the Entities package I cannot do a Find All of the entire solution. Why that is is beyond my understanding.

In any case, I decompiled the Unity.Collections dll and found that creating a new Job that calls an Execute method that includes the key would be relatively trivial (I think), however it requires access the the m_buffer variable of the NativeMultiHashMap which is not accessible outside of the dll. So I guess I just need to wait until you guys (hopefully) add something that will suit my needs. Thanks for the help.

 

The unity package source files are located on your computer:
< 2018.3b: %localappdata%\Unity\cache\packages\packages.unity.com
>= 2018.3b: <ProjectDir>\Library\PackageCache

 

Thanks, that is good to know. Unfortunately, the m_Buffer field of NativeMultiHashMap (in addition to the NativeHashMapData struct) is internal, so not accessible by any custom code as far as I'm aware. I tried to get the fields I need via reflection, but I don't think I have the knowledge to do this properly (or maybe it is not possible), as the fields in question are all pointers.

For example, the NativeHashMapData struct has four byte* fields (values, keys, next, and buckets) that you need to access. Again, the NativeHashMapData is stored as a pointer (m_Buffer) in the NativeMultiHashMap struct.

I tried the following:

Code (CSharp):

1. var multiHashMapType = typeof(NativeMultiHashMap<int, int>);
2. var field = multiHashMapType.GetField("m_Buffer", BindingFlags.Instance | BindingFlags.NonPublic);
3. var m_Buffer = Pointer.Unbox(field.GetValue(fullData.HashMap));
4. var dataType = multiHashMapType.Module.Assembly.GetType("Unity.Collections.NativeHashMapData");
5.  
6. var valuesField = dataType.GetField("values");
7. var keysField = dataType.GetField("keys");
8. var nextField = dataType.GetField("next");
9. var bucketsField = dataType.GetField("buckets");
10.  
11. int valuesOffset = UnsafeUtility.GetFieldOffset(valuesField);
12. int keysOffset = UnsafeUtility.GetFieldOffset(keysField);
13. int nextOffset = UnsafeUtility.GetFieldOffset(nextField);
14. int bucketsOffset = UnsafeUtility.GetFieldOffset(bucketsField);
15.                
16. var buckets = (int*)Pointer.Unbox(UnsafeUtility.ReadArrayElement<object>(m_Buffer, bucketsOffset));
17. var nextPtrs = (int*)UnsafeUtilityEx.ArrayElementAsRef<IntPtr>(m_Buffer, nextOffset);
18. var keys = (byte*)UnsafeUtility.ReadArrayElement<IntPtr>(m_Buffer, keysOffset);
19. var values = (byte*)Pointer.Unbox(UnsafeUtility.ReadArrayElement<object>(m_Buffer, valuesOffset));
20.  

Unfortunately, this just returns null values. I just need a way to access the pointers in NativeHashMapData. Any ideas? Thanks!

 

Well, i can't say i've used reflection with pointers before, but try adding the binding flags (BindingFlags.Instance | BindingFlags.NonPublic) to your dataType.GetField() calls.

EDIT: Also, if you're confident that Unity will add support for this in the future, you could also just add the functionality to the source code under that package cache location.

 

Thank you for the suggestion, unfortunately that does not help.

I am not confident at all in that fact, I think my use case is rather obscure but who knows, it's possible others will call for this functionality and it will be added in the future. But this was a great idea, I will just make a copy of the code so that if it's overwritten by future changes I can re-add it. Thanks!!

 

So ECS team,

Plans for parse keys from NativeHashMap or NativeMultiHashMap ? because there are not work around for that

 

I made a makeshift NativeHashMapList a while ago, which you can iterate + use the key. But of course it has some tradeoffs since I assembled it with 3 basic native containers.

https://forum.unity.com/posts/3745042/

 

Hi,
Can we still not iterate over keys? Iterating is a must have feature

 

.GetKeyArray()

 

well I know this is an old thread, but I think my question fits this thread, the question is also asked here on stackoverflow.

when I try to get my NativeHashMap's key array, it throws an IndexOutOfRangeException

Please note that this NativeHashMap is being updated on another system, this update is changing values and not keys also nothing is added or removed from the hashmap.

Is it a bug or am I doing something wrong?

 

You should post a dedicated thread and include a code sample.