Alternative to using string

If I have an entity that I'd want to hold a string or two, what would be the best way to go about doing that in the Entity Component System?

I am aware strings and char are not blittable, but was curious if there were any soon to be replacements of these types for entities to hold as data.

 

You should be able to do this if you want to hold a string that's a fixed size:

Code (CSharp):

1.   public unsafe struct EnemyShootState : IComponentData {
2.     public float Cooldown;
3.     public fixed char StringValue[20];
4.   }

That requires you to convert the String to a Char[] array and copy it over by hand. It also requires an unsafe context to access it, and can at most hold X characters (20 in this case, as defined after the StringValue name.

Char is blittable in the runtime itself, I dont know if there's any special restrictions around the Unity ECS though.

 

char is not blittable in C# from what I am aware
https://docs.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types

But when checking:

Debug.Log(UnsafeUtility.IsBlittable<char>());

Returns 'False'

 

Ah, thats a limitation from Unitythen.. System.Char most definately is blit:able by default in .NET

 

According to MSDN, System.Char is non-blittable. Unicode characters are 1-4 bytes long so this causes problems. Also, arrays are not blittable so byte[] won't work. There is a code sample in this post that uses NativeArray<byte> to hold encoded strings. Not a pretty thing, but it works. Ideally, you wouldn't do string processing in your job due to the overhead that string garbage collection will create but it works if you need it.

 

Can a ComponentSystem access a monobehaviour?

The entityManager sees the mono thanks to GameObjectEntity.
GetComponentData fails because it's not an IComponentData.


In both hybrid and pure, I'm struggling to find a home for non-blittable data like strings and >14k arrays.
It's instance data, so icomponentshared feels wrong.
A mono is BAU, but I want access from systems.

Am I missing something?

 

I believe, it is possible to access strings out of ECS data flow. There's no need to pass strings around. It is possible to store them somewhere and systems may cache that storage on their initialization. For instance, you may have an array of strings ad you may refer them by id, passed around by int

 

I was going to say something like this. Use a static Dictionary<string> and store the hash/key.

Unless you are trying to modify the string

 

The best solution I have rationalized thus far: Store a dictionary with the Entity as the key; however, GENERALLY in the past, statics have been a little on the iffy side when attempting to do running recompiles, meaning still having the simulation running, edit some code, then recompile the code to see live changes.

However, in the current state of the ECS, live compiling is not something that has been addressed just yet, so I am sort of unsure where statics lie in terms of approaching. Singletons are typically *A* use case of a static, but I have refactored all of my code to use scriptable objects in place of static fields to allow for more modular systems.

An edge case scenario that would require more work doing this dictionary route:
EntityManager.GetComponentData<StringDataComponent>(entity) would be relatively useless in this regard; and as mentioned before I'm a little wary with the static approach. StringSystemDictionary.GetData(entity) would work as what I would see as more of a temp solution rather than the end solution.

 

Really you want to keep strings away from ECS as they have a massive memory footprint, the goal being to cram as much data into the processors caches that the whole processing streams without any snags and max through put.

What about using a hashing function to 'index' your strings for access?

I'm assuming the strings are just user information or debug data and not the core data you are processing.

 

I have a broadcast notifier system that is composed of strings so it is a little unavoidable. I am the receiver of string information from a external API.

 

We definitely need something like NativeString that can work with IComponentData nicely.

 

Just map the strings to integers and stick them in a Dictionary<int,string> when you receive them. Your classes/structs that would have had a reference to the string, now reference it via id.

Code (csharp):

1.  
2.  
3. // Somwhere
4.         public static Dictionary<int, string> Strings = new Dictionary<int, string>();
5.  
6.         // In classes/structs that need a reference
7.         private int MyStringId;
8.         public string MyString
9.         {
10.             get
11.             {
12.                 string value;
13.                 if (Strings.TryGetValue(MyStringId, out value))
14.                 {
15.                     return value;
16.                 } else
17.                 {
18.                     return null;
19.                 }
20.             }
21.         }
22.  

 

You should map the strings to tokens then you are sending less data, I'm presuming both systems have a common set of strings they work with and your not writing a dynamic system like those AI's that started developing their own language -> http://www.iflscience.com/technology/facebook-ai-develop-language-human-understand/

 

I translate a json string that I receive into a scriptable object (before ECS); my hope was to use this data to create entities with the names and stringed values so that it would render based on the data. In a NON-ecs system, this works easily.

The only known information is the topic that the data is being sent over, everything else is dynamic. This is real time data that I, again, have no control over the structure. For now, I will wait until Unity has a better solution to this as Joachim mentioned before. I already mentioned my standpoint and current solution using static dictionaries.

Also @snacktime, while that does clean a little bit up, I wouldn't need to store an extra int as I have the entity ID

Thanks though

 

It's not ideal but you could use ISharedComponentData to store string values too. There is no requirement for the data to be blittable.

 

IShared is to really be used for Non-changing and grouped data. Whilst the data, once cataloged is relatively immutable, in this case I have to assume all data will be unique which is not really ideal for the use case from what I understand. I also have to work on the assumption I don't have an end of input data.

 

Hey Joachim, I noticed in the Berlin Q&A that 'arrays and strings' are being addressed, but that wouldn't happen to be sooner rather than later would it? I'd very much rather prefer to use a look up system already inplace rather than having an extra layer dictionary lookup on my side

 

I think it's actually not that far off. He spoke about arrays in details but didn't really say anything about strings besides the title. Though I guess a string is just a character array internally anyway so it shouldn't really be that different.

I wonder what the restrictions on arrays and strings will be. Since changing the length of them would mean rearranging the whole data layout of those strings and arrays for all the Entity components of that type in memory.

 

It might be like you are allowed to store NativeArray in a component data so it is just a fixed length address, then the address will actually link to memory somewhere else which allows variable length? (Could cause complexity with serialization etc.)

 

Well, in the talk he said they would layout the data for all the arrays of the components in linear memory as well. So it can't be just pointers that point somewhere else or you'll lose all the cache optimization and basically have cache misses everywhere.

 

New to DOTS. Joachim, can you update us on what the solution is for this technical hurdle is now? Got an example?

Use case:
Will have thousands of Prefabs at application launch in Scene. We need to have MouseOver and or Click on way to populate UI elements with each Prefab>Entity's related data (e.g. color, geo location, name, in-operation time, etc.). Each Entity has their own data. This was working fine in OOP, but of course we are dealing with a different beast here. Any help would be appreciated. I've been banging my head on a wall for days now.

 

Pretty old thread.

There are several ways to work around this. One is to use ISharedComponentData which can hold managed objects like strings.
Another is to build a Singleton accessible or static Dictionary<Entity, string> as a lookup.

In systems, strings should be avoided at all costs. There's also really no reason to have them or need them there.
ECS for UI elements is also not recommended. A normal MonoBehaviour will be enough with IPointerEnterHandler, IPointerExitHandler to handle the update of texts.

 

Or simply use the NativeString types? Burst even got some basic support now in v1.3 to do debug string writing and interpolation.

 

Thanks Enzo, I'll look into these avenues.

 

I saw this option late in the evening yesterday after a lot of searching. Wondering how NativeString64 would look in an example for this use-case. I can get NS64 to Debug.Log() after testing .isBlittable() showed true. What's the best way now to associate it with the instantiation process of Prefabs (Entities)?

 

We also have FixedString. How is it different from NativeString?

 

I'd guess it lets you specify a fixed size, may help with performance/optimization

 

You can put FixedString into a struct IComponentData, you cant with NativeString since its not blittable

 

I have been using NativeString in ICDs before FixedString was a thing.

 

Why do we need NativeString then?

 

My understanding is that FixedString is essentially identical to NativeString and will serve as it’s replacement in order to be consistent with the other Fixed*** types that have been recently added.

 

Note that ECS also supports *class* component types if you must have variable length strings, so you could do something like this:

Code (CSharp):

1. class ManagedString : IComponentData
2. {
3.     public string Value;
4. }

And you would access that like a normal IComponentData via the regular API.
If you need to store strings with an upper limit on their length, such as the player name, then using FixedString32 or FixedString64 would probably be more cache friendly.

 

Note that these class component are not compatible with burst (them behing reference type), not sure they work with jobs either.

 

This is what I'm looking at now. But I'm wondering how I tie this together. I've not seen any good Unity or third-party demonstrations of this technical hurdle fixed. Surely this is needed in just about every Unity application?

 

Okay, here's where I'm at now. Anyone know what goes in where the question mark (?) goes? NativeArray<NativeString64> related.

I'm once again pulling out my hair. I'm not seeing it. Or am I way off?

I'm trying to add an array to a NativeMultiHashMap. (I want my Entities to have associated attributes or properties if you will, like color, scale/size, string label).

Here's what I have so far. I feel I'm close now.

Code (CSharp):

1. NativeArray<NativeString64> sArray = new NativeArray<NativeString64>(1, Allocator.Persistent);
2. public NativeMultiHashMap<int, NativeArray<NativeString64>> nmhm_pointTypes = new NativeMultiHashMap<int, NativeArray<NativeString64>>(100000, Allocator.Persistent);
3.  
4.                 sArray[i] = "One";
5.                 sArray[i] = "Two";
6.                 sArray[i] = "Three";
7.  
8.                 nmhm_pointTypes.Add(i, ?);  <---------- how do I code the sArray (or if you have another way?) into the position where the question mark is located?
9.  
10.                 NativeArray<NativeString64> value;
11.                 NativeMultiHashMapIterator<int> iterator;
12.                 if (nmhm_pointTypes.TryGetFirstValue(i, out value, out iterator))
13.                 {
14.                     do
15.                     {
16.                         //Debug.Log("KEY: " + 0 + " VALUE: " + value);
17.                         Debug.Log("KEY: " + i + " VALUE: " + value);
18.                     } while (nmhm_pointTypes.TryGetNextValue(out value, ref iterator));
19.                 }

 

You can't. It's not allowed to have NativeArray in native hash map, native array of native arrays etc..

 

No wonder. Thanks.

 

Consider using FixedString instead

 

What? How would that change anything in this scenario? Any blittable type is fine to use in a NativeContainer. The problem here was the fact that a NativeArray was being used. Most (if not all) NativeContainers are non-blittable. NativeStrings are not NativeContainers.

 

I know you replied to Kender here. But thanks for the clarification nontheless Soaryn.