Showcase Easy ECS Access for UI Toolkit (requires Harmony)

Update:
Now does not need to edit the packages, but uses Harmony to do the changes it previously needed to edit packages for.
Probably not gonna update this anymore.(probably)
Download::UIToolkitExtensions.zip

Everything besides not needing to edit packages is mostly unchanged, biggest exception being the

AddManipulator

class needing the namespace to be usable and it not showing up in the editor. So you need to manually add it to the UXML (

<Unity.UI.BuilderExtension.AddManipulator/>

) and then you can just use the editor to configure it.

I've been using this new version for about two months now, so it should be reasonably bug free. And I hopefully didn't forget to package anything with it either.

Old Description with explanations:

Spoiler

Data flow:
Entity -> Component -> Field(optional) -> Conversion(optional) -> Operaton
Entity -> has Component -> Conversion (optional) -> Operaton
Do not select 'None' for the conversion field if the Component output and Operation input are not of the same type.

Has Component(bool) field:
A new 'field' option is 'Has Element', rather than getting the component or a field of it, it returns whenever the Entity has or has not said Component.

Operation field:
It lets you do whatever you want with the VisualElement rather than being stuck only being able to set text. It requires two generic types, the input value and the VisualElement subtype. It is checked during runtime if the VisualElement subtype matches to what it was attached to, but it seems not possible to check during edit time. A Error will display if a UpdateOperation is not valid for the VisualElement the manipulator was attached to, remember to name your VisualElement or you will have a hard time to find which one is causing the issue.

UpdateOperation Examples:

Spoiler

Code (CSharp):

1.  
2. public class SetTextOperation : UpdateOperation<string, TextElement> {
3.     public override void PerformOperation(string value, TextElement target) {
4.         target.text = value;
5.     }
6. }
7.  
8. public class SetDisplayOperation : UpdateOperation<bool, VisualElement> {
9.     public override void PerformOperation(bool value, VisualElement target) {
10.         target.style.display = value ? DisplayStyle.Flex : DisplayStyle.None;
11.     }
12. }
13.  
14. public class SetVisibleOperation : UpdateOperation<bool, VisualElement> {
15.     public override void PerformOperation(bool value, VisualElement target) {
16.         target.visible = value;
17.     }
18. }
19.  
20. public class SetEntityData : UpdateOperation<Entity, VisualElement> {
21.     public override void PerformOperation(Entity value, VisualElement target) {
22.         Utils.UIElementUserData(target, new SourceEntity() { source = value });
23.     }
24. }

Old Old Description and more indepth explanation:

Spoiler

I've made a tool to enable you to add manipulators straight from UI Builder and even made so you can automagically take ECS data and replace Text fields' text with said values. I am sure there are more optimizations to further mitigate the whole lot of allocations and reflection used, but I am not really good at that and need to focus on other issues in my project, so those will be as is unless someone else want to keep extending this.

Video: (OBS did not capture the pop up menus, and I really don't want to remake the video) (I really wish this forum would support uploading webm)

How it works:

The Manipulator Type field will use reflection to find any manipulator on your assembly, it currently have a solution specific limitation on which assemblies it searches for.

The manipulators can declare their own 'Inspector' to be used. Unfortunately I couldn't get the inspector to both create the inspector fields and generate the manipulator to be added, since adding a manipulator have to work on the built version that lacks editor code and the 'adding of fields' require editor code, so I had to split declaring a manipulator inspector in two parts, one is declaring the inspector much alike how one declares a custon inspector, and another is making sure your manipulator class implements a interface for initializing it's fields from a string (since UXML stores data in strings).

Currently I have only one "MiniInspector" implemented, which is for the

SetTextFromEntityDataManipulator

shown above.
Data Source Type: Lets you select if the source is a

IComponentData

,

ISharedComponentData

or a

Component

Object
Data Type Filter: Is just a string field that lets you filter, because ECS projects will tend to have a whole lot of any of the above to fill your whole monitor height 3 times over. If any part of the type name contains any part of the field text, it's considered found.
Data Type: A popup field that lets you pick a type to be searched for in the entity (more on this later)
Data Field: Which field/property of the type you want to supply the conversor or .ToString() it.
Conversor: Lets you pick a class to convert your data to string, in case it needs special treatment. It will filter for matching types, so a

DataConversion<PlantData>

will only show in the list if you select

PlantData

and field as None to supply the whole

PlantData

as is (in case you need more than one field from it to properly display the information.
Currently there is no way to supply two or more components for a conversor. But the same tech used to get conversor could be used for a different

Manipulator

that supply a

Entity

and

EntityManager

to a different type of conversor.

Supplying the Entity:
Those manipulators are all and good, but there needs to be something that supplies them the Entity to take the data from. There is basically no generic way to accomplish this, so each project needs it's own way of doing it. Currently my project sets a static variable 'WorldTooltip' with whichever Entity the center of the screen is 'looking' at. This is done by this Manipulator:

Spoiler: SetEntityManipulator.cs

Code (CSharp):

1. using UnityEngine.UIElements;
2.  
3. public class SetEntityManipulator : Manipulator {
4.     public enum EntitySourceMethod {
5.         WorldTooltip
6.     }
7.     private bool Disabled;
8.     public EntitySourceMethod method;
9.  
10.     public SetEntityManipulator() {//I will eventually make a mini inspector for this when I have more than one
11.         method = EntitySourceMethod.WorldTooltip;//way of supplying a Entity to the UI Toolkit
12.     }
13.  
14.     protected override void RegisterCallbacksOnTarget() {
15.         Disabled = false;
16.         target.schedule.Execute(() => Utils.UIElementUserData(target, GetEntitySource())).Until(() => Disabled);
17.     }
18.  
19.     private SourceEntity GetEntitySource() {
20.         if (method == EntitySourceMethod.WorldTooltip) {
21.             if (UIKeyListener.WorldTooltip != null) {//Solution specific
22.                 return new SourceEntity() { source = UIKeyListener.WorldTooltip.Entity };
23.             }
24.         }
25.         return new SourceEntity();
26.     }
27.  
28.     protected override void UnregisterCallbacksFromTarget() {
29.         Disabled = true;
30.     }
31. }

What's important is a

UIElement

having a

userData

of

SourceEntity

type, so it's child can look for it and then know which Entity to use. This probably can be optimized using Events.

Utils.UIElementUserData

uses a Dictionary to let me have multiple user datas in the same object, much like a

GameObject

can have multiple

Behaviors

Getting and converting the data to string:
This class is heavily dependent on Vexe.Runtime.Extensions (https://github.com/vexe/Fast.Reflection) and uses Newtonsoft.Json to serialize the data
There is not much to explain besides displaying the code. The most important methods are

RegisterCallbackOnTarget

which starts the scheduling for updating the text field and

ConfigureManipulator

which enables this to work on the Standalone Player by taking the Manipulator configuration out of the Editor-only

MiniInspector

Spoiler: SetTextFromEntityDataManipulator.cs

Code (CSharp):

1. using Newtonsoft.Json;
2. using System;
3. using System.Collections.Generic;
4. using System.Reflection;
5. using Unity.Entities;
6. using UnityEngine;
7. using UnityEngine.UIElements;
8. using Vexe.Runtime.Extensions;
9. public class SetTextFromEntityDataManipulator : Manipulator, ConfigurableManipulator {
10.     public class SetTextFromEntityDataManipulatorData {
11.         public Type DataSourceType;
12.         public Type DataType;
13.         public MemberInfo Member;
14.         public Type Conversor;
15.         [JsonIgnore]
16.         public List<Type> DataTypeTypes = new List<Type>();
17.         [JsonIgnore]
18.         public string TypeFilter = "";
19.         [JsonIgnore]
20.         public List<MemberInfo> MemberInfoList = new List<MemberInfo>();
21.         [JsonIgnore]
22.         public List<Type> ConversorList = new List<Type>();
23.     }
24.     private bool Disabled;
25.     private bool HasEntityManager;
26.     private string OldValue = "";
27.     public Func<EntityManager, Entity, object> GetEntityDataFunction;
28.     public Func<object, string> TransformationFunction;
29.     public SetTextFromEntityDataManipulator() {
30.     }
31.  
32.     protected override void RegisterCallbacksOnTarget() {
33.         Disabled = false;
34.         HasEntityManager = false;
35.         if (typeof(TextElement).IsAssignableFrom(target.GetType())) {
36.             target.schedule.Execute(() => {
37.                 EntityManager em;//World.DefaultGameObjectInjectionWorld is not a thing when the first schedule runs
38.                 if (World.DefaultGameObjectInjectionWorld != null) {
39.                     em = World.DefaultGameObjectInjectionWorld.EntityManager;
40.                 } else {
41.                     return;
42.                 }
43.                 target.schedule.Execute(() => {
44.                     //Navigates the hierarchy until it finds a UIElement having SourceEntity, probably can be optimized with events
45.                     SourceEntity sourent = Utils.UIElementUserDataInParent<SourceEntity>(target);
46.                     if (sourent != null && sourent.source != default) {
47.                         Entity ent = sourent.source;
48.                         if (em.Exists(ent)) {
49.                             //boxing, there might be a way to optimize this, but I don't know of it
50.                             object data = GetEntityDataFunction(em, ent);
51.                             if (data != null) {
52.                                 string newval = TransformationFunction(data);
53.                                 if (OldValue != newval) {
54.                                     OldValue = newval;
55.                                     TextElement eltarg = target as TextElement;
56.                                     eltarg.text = newval;
57.                                 }
58.                             }
59.                         }
60.                     }
61.                 }).Until(() => Disabled);
62.             }).Until(() => HasEntityManager);
63.         }
64.     }
65.  
66.     protected override void UnregisterCallbacksFromTarget() {
67.         Disabled = true;
68.     }
69.  
70.     public void ConfigureManipulator(string CurrentValue) {
71.         SetTextFromEntityDataManipulatorData data = JsonConvert.DeserializeObject<SetTextFromEntityDataManipulatorData>(CurrentValue, new MemberInfoConverter());
72.  
73.         if (data.DataSourceType != null && data.DataType != null) {
74.             MethodCaller<object, object> methodCaller = null;
75.             MemberGetter<object, object> memberGetter = null;
76.             Type EMType = typeof(EntityManager);
77.             MethodCaller<EntityManager, bool> hasMethod = EMType.GetMethod("HasComponent", new Type[] { typeof(Entity) }).
78.                     MakeGenericMethod(data.DataType).DelegateForCall<EntityManager, bool>();
79.             if (data.DataSourceType == typeof(IComponentData)) {
80.                 methodCaller = EMType.GetMethod("GetComponentData").
81.                     MakeGenericMethod(data.DataType).DelegateForCall<object, object>();
82.             }
83.             if (data.DataSourceType == typeof(ISharedComponentData)) {
84.                 methodCaller = EMType.GetMethod("GetSharedComponentData", new Type[] { typeof(Entity) }).
85.                     MakeGenericMethod(data.DataType).DelegateForCall<object, object>();
86.             }
87.             if (data.DataSourceType == typeof(Component)) {
88.                 methodCaller = EMType.GetMethod("GetComponentObject").
89.                     MakeGenericMethod(data.DataType).DelegateForCall<object, object>();
90.             }
91.             Type fieldType = null;
92.             if (data.Member is FieldInfo) {
93.                 memberGetter = (data.Member as FieldInfo).DelegateForGet();
94.                 fieldType = (data.Member as FieldInfo).FieldType;
95.             } else if (data.Member is PropertyInfo) {
96.                 memberGetter = (data.Member as PropertyInfo).DelegateForGet();
97.                 fieldType = (data.Member as PropertyInfo).PropertyType;
98.             }
99.             if (data.Member == null || data.Member.GetType() == typeof(EmptyMemberInfo)) {
100.                 fieldType = data.DataType;
101.             }
102.             if (methodCaller != null) {
103.                 GetEntityDataFunction = (EM, E) => {
104.                     object[] ENT = new object[] { E };
105.                     return hasMethod(EM, ENT) ? methodCaller(EM, ENT) : null;
106.                 };
107.             }
108.             if (memberGetter != null) {
109.                 if (data.Conversor != null && data.Conversor != typeof(EmptyConversor) && fieldType != null) {
110.                     ConvertDelegate convertMethod = DataConversion.GetConvertMethod(data.Conversor, fieldType);
111.                     if (convertMethod != null) {
112.                         TransformationFunction = (O) => convertMethod(memberGetter(O));
113.                     } else {
114.                         TransformationFunction = (O) => memberGetter(O).ToString();
115.                     }
116.                 } else {
117.                     TransformationFunction = (O) => memberGetter(O).ToString();
118.                 }
119.             } else {
120.                 if (data.Conversor != null && data.Conversor != typeof(EmptyConversor) && fieldType != null) {
121.                     ConvertDelegate convertMethod = DataConversion.GetConvertMethod(data.Conversor, fieldType);
122.                     if (convertMethod != null) {
123.                         TransformationFunction = (O) => convertMethod(O);
124.                     } else {
125.                         TransformationFunction = (O) => O.ToString();
126.                     }
127.                 } else {
128.                     TransformationFunction = (O) => O.ToString();
129.                 }
130.             }
131.         }
132.     }
133. }
134.  

Custom MiniInspectors:
Create a class extending

ManipulatorMiniInspector

and with the

CustomMiniInspector

attribute much alike a

CustomEditor

. The only method you need to implement is

BindableElement CreateInspector(string CurrentValue, EventCallback<string> OnChangeCallback)

The biggest issue is that even though

SetTextFromEntityDataManipulatorMiniInspecotor

have multiple fields but can only write to one UXML attribute without a major rewrite of the

BuilderInspectorAttributes

class, and to keep changes to the packages at a minimum, I just use Json to serialize whatever I need to string.

Spoiler: SetTextFromEntityDataManipulatorMiniInspecotor.cs

Code (CSharp):

1. using System.Collections.Generic;
2. using UnityEditor.UIElements;
3. using UnityEngine.UIElements;
4. using System.Reflection;
5. using Newtonsoft.Json;
6. using Unity.Entities;
7. using System.Linq;
8. using UnityEngine;
9. using System;
10. using static SetTextFromEntityDataManipulator;
11.  
12. [CustomMiniInspector(typeof(SetTextFromEntityDataManipulator))]
13. public class SetTextFromEntityDataManipulatorMiniInspecotor : ManipulatorMiniInspector {
14.     public override BindableElement CreateInspector(string CurrentValue, EventCallback<string> OnChangeCallback) {
15.         SetTextFromEntityDataManipulatorData data = null;
16.         try { data = JsonConvert.DeserializeObject<SetTextFromEntityDataManipulatorData>(CurrentValue, new MemberInfoConverter()); } catch (Exception e) { Debug.LogError(e); };
17.         if (data == null) {
18.             data = new SetTextFromEntityDataManipulatorData() { DataSourceType = typeof(IComponentData) };
19.         }
20.         BindableElement container = new BindableElement();//
21.         Func<Type, string> func = (T) => (T.Name);
22.         Type[] values = new Type[] { typeof(IComponentData), typeof(ISharedComponentData), typeof(Component) };
23.         PopupField<Type> datasourcetype = new PopupField<Type>("Data Source Type", values.ToList(), values[0], func, func);
24.         if (values.Contains(data.DataSourceType)) {
25.             datasourcetype.value = data.DataSourceType;
26.         }
27.         container.Add(datasourcetype);
28.  
29.         TextField dataTypeFilter = new TextField("Data Type Filter");
30.         container.Add(dataTypeFilter);
31.         Type[] types = UpdateTypes(data);//
32.         PopupField<Type> datatype = new PopupField<Type>("Data Type", data.DataTypeTypes, data.DataTypeTypes[0], func, func);
33.         if (types.Contains(data.DataType)) {
34.             datatype.value = data.DataType;
35.         }
36.         container.Add(datatype);
37.  
38.         MemberInfo[] members = GetMembers(data);//
39.         Func<MemberInfo, string> func2 = (T) => (T.Name);
40.         PopupField<MemberInfo> member = new PopupField<MemberInfo>("Data Field", data.MemberInfoList, data.MemberInfoList[0], func2, func2);
41.         if (members.Contains(data.Member)) {
42.             member.value = data.Member;
43.         }
44.         member.SetEnabled(members.Length != 1);
45.         container.Add(member);
46.  
47.         Type[] conversors = GetConversors(data);//
48.         Func<Type, string> func3 = (T) => (T == typeof(EmptyConversor) ? "None" : T.Name);
49.         PopupField<Type> conve = new PopupField<Type>("Conversor", data.ConversorList, data.ConversorList[0], func3, func3);
50.         if (conversors.Contains(data.Conversor)) {
51.             conve.value = data.Conversor;
52.         }
53.         conve.SetEnabled(conversors.Length != 1);
54.         container.Add(conve);
55.  
56.         datasourcetype.RegisterValueChangedCallback((T) => {
57.             data.DataSourceType = T.newValue;
58.             UpdateTypePopupField(data, datatype);
59.             UpdateTypePopupField(data, member);
60.             UpdateConversorPopupField(data, conve);
61.             OnChangeCallback(JsonConvert.SerializeObject(data, new MemberInfoConverter()));
62.         });
63.         dataTypeFilter.RegisterValueChangedCallback((T) => {
64.             data.TypeFilter = T.newValue;
65.             UpdateTypePopupField(data, datatype);
66.             UpdateTypePopupField(data, member);
67.             UpdateConversorPopupField(data, conve);
68.         });
69.         datatype.RegisterValueChangedCallback((T) => {
70.             data.DataType = T.newValue;
71.             UpdateTypePopupField(data, member);
72.             UpdateConversorPopupField(data, conve);
73.             OnChangeCallback(JsonConvert.SerializeObject(data, new MemberInfoConverter()));
74.         });
75.         member.RegisterValueChangedCallback((T) => {
76.             data.Member = T.newValue;
77.             UpdateConversorPopupField(data, conve);
78.             OnChangeCallback(JsonConvert.SerializeObject(data, new MemberInfoConverter()));
79.         });
80.         conve.RegisterValueChangedCallback((T) => {
81.             data.Conversor = T.newValue;
82.             OnChangeCallback(JsonConvert.SerializeObject(data, new MemberInfoConverter()));
83.         });
84.         return container;
85.     }
86.     private static void UpdateTypePopupField(SetTextFromEntityDataManipulatorData data, PopupField<Type> datatype) {
87.         Type[] types2 = UpdateTypes(data);
88.         datatype.MarkDirtyRepaint();
89.         if (types2.Length > 0) {
90.             datatype.SetValueWithoutNotify(types2[0]);
91.             datatype.SetEnabled(true);
92.         } else {
93.             datatype.SetEnabled(false);
94.         }
95.     }
96.     private static void UpdateTypePopupField(SetTextFromEntityDataManipulatorData data, PopupField<MemberInfo> members) {
97.         MemberInfo[] membersInfos = GetMembers(data);//
98.         members.MarkDirtyRepaint();
99.         if (membersInfos.Length > 0) {
100.             members.SetValueWithoutNotify(membersInfos[0]);
101.         }
102.         members.SetEnabled(membersInfos.Length != 1);
103.     }
104.     private static void UpdateConversorPopupField(SetTextFromEntityDataManipulatorData data, PopupField<Type> members) {
105.         Type[] conversors = GetConversors(data);//
106.         members.MarkDirtyRepaint();
107.         if (conversors.Length > 0) {
108.             members.SetValueWithoutNotify(conversors[0]);
109.         }
110.         members.SetEnabled(conversors.Length != 1);
111.     }
112.     private static Type[] UpdateTypes(SetTextFromEntityDataManipulatorData data) {
113.         Type[] types2 = Utils.GetAllClassesImplementing(data.DataSourceType, "Unity", "Microsoft", "System", "Mono", "UMotion");
114.  
115.         Array.Sort(types2, (T1, T2) => T1.FullName.CompareTo(T2.FullName));
116.         data.DataTypeTypes.Clear();
117.         string lower = data.TypeFilter.ToLower();
118.         types2 = types2.Where(T => T.FullName.ToLower().Contains(lower)).
119.             Where(T => GetFields(T).Length > 0 || GetProperties(T).Length > 0).ToArray();
120.         data.DataTypeTypes.AddRange(types2);
121.         return types2;
122.     }
123.     private static Type[] GetConversors(SetTextFromEntityDataManipulatorData data) {
124.         Type[] types2 = new Type[0];
125.         Type fieldType = null;
126.         if (data.Member is FieldInfo) {
127.             fieldType = (data.Member as FieldInfo).FieldType;
128.         } else if (data.Member is PropertyInfo) {
129.             fieldType = (data.Member as PropertyInfo).PropertyType;
130.         }
131.         if (data.Member == null || data.Member.GetType() == typeof(EmptyMemberInfo)) {
132.             fieldType = data.DataType;
133.         }
134.         if (fieldType != null) {
135.             types2 = DataConversion.GetConversors(fieldType).Cast((T) => T.GetType()).ToArray();
136.             Array.Sort(types2, (T1, T2) => T1.GetType().Name.CompareTo(T2.GetType().Name));
137.         }
138.         data.ConversorList.Clear();
139.         IEnumerable<Type> collection = new Type[] { typeof(EmptyConversor) }.Concat(types2);
140.         data.ConversorList.AddRange(collection);
141.         return collection.ToArray();
142.     }
143.     private static MemberInfo[] GetMembers(SetTextFromEntityDataManipulatorData data) {
144.         MemberInfo[] types2 = new MemberInfo[0];
145.         if (data.DataSourceType != null && data.DataType != null) {
146.             types2 = types2.Concat(GetFields(data.DataType)).Concat(GetProperties(data.DataType)).ToArray();
147.             Array.Sort(types2, (T1, T2) => T1.Name.CompareTo(T2.Name));
148.         }
149.         data.MemberInfoList.Clear();
150.         IEnumerable<MemberInfo> collection = new MemberInfo[] { new EmptyMemberInfo() }.Concat(types2);
151.         data.MemberInfoList.AddRange(collection);
152.         return collection.ToArray();
153.     }
154.     private static PropertyInfo[] GetProperties(Type data) {
155.         return data.GetProperties();
156.     }
157.     private static FieldInfo[] GetFields(Type data) {
158.         return data.GetFields();
159.     }
160. }
161.  

Custom Conversor:
Some examples:

Code (CSharp):

1. public class PlantDataTestConversor : DataConversion<PlantData> {
2.     public override string Convert(PlantData value) {
3.         return "PlantData";
4.     }
5. }
6.  
7. public class StringTestConversor : DataConversion<string> {
8.     public override string Convert(string value) {
9.         return "" + value.Length;
10.     }
11. }

Installation:
Just drop anywhere in the project.
Requires Harmony and Newtonsoft.Json-for-Unity (insert

"jillejr.newtonsoft.json-for-unity": "https://github.com/jilleJr/Newtonsoft.Json-for-Unity.git#upm"

into your manifest.json)
Was tested against:

    "com.unity.ui": "1.0.0-preview.14",

    "com.unity.ui.builder": "1.0.0-preview.13"

and should at least compile like that.

 

Updated

 

If I understood correctly, you are modifying the packages because you need access to some internals, correct? If you create an Assembly Definition Reference for the package you want access to its internal and add a file with [InternalsVisibleTo("My.Assembly.With.Internals.Access")] you won't need to modify the packages itself anymore

 

I am directly changing the source of one of the files though, not only making some fields visible, otherwise I could just use reflection

 

Since it doesn't sound like unity is going to support DOTS+UiElements any time soon, I'd like to try your version, but the above is pretty vague- do you have a demo project of this working somewhere?

 

Sorry, I don't have the time to update/make it a proper thing ready to use. Hopefully someone can take it over

 

Updated