GUI in Pure ECS Projects

When you want to be 100% pure ECS on new projects (such as to comply with Small Things, for example, as in

), what are people doing for their GUI layer?

I would guess that most of us are just doing very bare bones "detecting clicks and touches on sprites" at the moment, but has anyone come across a true UI framework for ECS yet? Or have any ideas about where you intend to go with this? Are you writing your own input fields that handle mobile keyboards and the whole nine yards? Writing everything from square 1 seems a daunting and wasteful task, but is it unavoidable?

Thanks for any thoughts or ideas.

 

I have the exact same concern, according to this thread: https://www.gamedev.net/forums/topic/680812-ecs-gui-systems/ they think it's no go to use ECS for UI, since it would become really complex.

but i'm not sure if updating the Component value without a System, for example when a UI button clicked (eg. using a health potion or a magic spell) is allowed in Unity ECS?
or we should create an entity with certain component everytime there's an user action on UI, then process it to update the other component, then remove it after it processed (like an event queue?)

Normally without ECS, this what i do:

dataManager.UseItem(button.itemIndex);

// ^ inside the button's onclick listener



dataManager.OnInventoryChanged.AddListener(delegate{

   // update UI that need to be updated, the belt or the bag

})



// inside DataManager.UseItem method:

dataManager.OnInventoryChanged.Invoke()

 

Like @kokizzu said befor, ECS and UI isn't a good idea. It blow up everything and make fingst really complex.

if you require to interact with ecs here are some hints:

Searching for entities won't work outside of an system as far as I know, because we can't get component groups through the entity manager. So the best thing would be to create an GameObject with the GameObjectEntity attached to it and some settings components if you like.
Or use a static class that provides the settings for all systems.

If you require some event like behaviour on your ECS, than create entities with an component as flag and the required data. Then you can implement Systems which will act on that specific components and will destroy them after handling!

Creating and destroying entities and components in MonoBehaviour can be done without any risk to invalidate systems!
So you can just use the follwing inside your MonoBehvaiour UI classes...

Code (CSharp):

1.  
2. EntiyManager entityManager = World.Active.GetExistingManager<EntityManager>();
3. entityManager.SetComponentData(e, new MyData{});
4. entityManager.AddComponentData(e, new MyData());
5. entityManager.RemoveComponent<MyData>(e);
6. entityManager.DestroyEntity(e);
7. ...
8.  

 

I do it in similar fashion as SpyShifty suggested - I have CommandEntity archetype where I have a CommandType and some additional payload.

UI callbacks just creates the command entity with appropriate information (like: build-this, move-there). I react on those command entities in systems, and have a cleanup system at the end of the frame that cleans the issue commands, if they have been "consumed". This is one way system tho, so if you want to have some UI feedback you need to work around it somehow.

I don't use ECS for implementing UI flow, it's not meant for that - just use whatever UI framework you have.

For example if you have a button to issue "Stop" command, don't create a MouseDownEntity and try to workaround in your systems what was clicked. Handle the click event in UI, and create a StopEntity and react on it in your systems.

 

As I stated, though, the requirement for the "small things" version of unity is 100% pure ecs. No "classic" MonoBehaviour stuff at all. That's why I opened this topic.

 

You should keep in mind that ecs isn't the panacea of all the problems in the programming world...
You shouldn't solve problems that can be done in a simpler way in the "traditional" oop version!
ECS is good for simultion stuff but won't work well for UI stuff for excample...

You also won't profit from performance by jobifing UI stuff at all.
So keep it simple as possible and don't try to squeeze all to just one paradigm like ecs. The world isn't just black and white, its full of colors!

I don't write that just for you.. I want to write that for all who think that all have to do in ecs and everything else is just bad and shouldn't be used until know

Just keep this in mind.

Best practice is sometimes something that just makes you feel comfortible with.

 

Please don't advise people to avoid ECS for any reason as Unity needs ECS to be suitable for everything including UI/Editor.

The point of this forum is to provide feedback, especially if you cannot do something. If it turns out that something cannot be done easily, this is good data and what Unity needs to figure out how to solve these problems

I think ECS can be made to work better, not worse with UI, but it will require extra coupling or functionality and that's what Unity needs to get back to us with.

 

How exactly would ECS NOT be a good fit for UI.
So more accurately, what part of your UI is not data? Personally I think the ECS is a PERFECT fit for UI for both design and implementation.

 

Thanks for those thoughts, @Spy-Shifty , and I'm sorry to have to repeat this yet again, but if you will look above I've explained why this question is about 100% pure ECS, not out of some arbitrary choice of mine, but because the "small things" build of Unity makes it a requirement. You keep telling me how it's better to not use ECS. That is simply not an option. I hope we can move the discussion past that.

 

being accurate - everything is some sort of data for ECS is how you process of that data that maters

 

@Claytonious As the ECS framework matures, I'm pretty sure Unity will provide us with a GUI framework for pure ECS. Things are too early at this point.

 

When will ECS be suitable for UI?

I'm familiarizing myself with ECS now and as my primary job is handling UI/screens it'd nice to know when I'll be able to truly move to pure ECS. I find it quite hard to only partially adopt ECS. It just seems inconvenient to me to operate in two really disctinct mindset... If I wait for ECS to be completely integrated in Unity, I might just wait forever

I really like how the Tiny package works, displaying entities in the hierarchy instead of on the backend / invisible side of Unity. I

 

Joachim stated at the last Unite that ECS 1.0 will release sometime in 2019.

No telling what features will be officially supported in that 1.0 release, but we can bet it will be explicitly stated.

In that same talk, Joachim also mentioned UI as one a handful of specific targets the team is considering.

 

Was it ECS 1.0 .... or Burst 1.0? I was under the impression it was the latter according to the unite LA roadmap slides

 

I really wonder what ECS 1.0 means as it currently has no "pure" support for raycasting, animations or whatsoever.

It actually only works with rendering mostly non-interactive stuff, and currently don't show up anywhere in editor windows (except for Project Tiny).

It's like they are rewriting and rewiring the whole engine and that will take *a long long time* to achieve.

 

What you mean no pure ECS support for raycasting? I use it without a problem. Or do you mean specifically for UI?

 

Uh oh, do I have that wrong?

 

I meant in a complex game scenerio, there are a bunch of things not working in pure ECS in regards to rigid bodies and collisions.

There are ways to achieve similar things, but none are built in at the moment.

 

Eh, yes that is true unfortunately. But that is expected for beta.
But we can look at this from other point of view, ... we were simply spoiled with things, offered by Unity Classic OOP and editor

 

Hey, i am guessing we would need to write our own text mesh component data, then a system for it. Converting the text to meshes and uv. For a simply health bar I can just do this with some quads I guess. I want 4k + agents, and each with their own health bars, so this will be annoying haha.

 

I think right now that it is the only way to go.
I tried having hydrid components with UI text but it kills performance after a couple thousand elements, not to mention that we can't use it with Unity for Small Things.
It would be nice to have at least a base system/components for rendering Text in ECS.

 

Was there any rumors about UI in ECS during GDC2019?

 

Not that I am aware of.

 

There is data oriented GUI system made by CoherentLabs which use HTML5 code and render it on top of another engine.

It is possible to make UI in ECS. Still I dont think anybody will do it anytime soon.

 

Of course it's possible to do an UI using ECS. This guy did something like that:

 

Scene0:
Script JobSystem: Create Entities:
- Camera Entity
- 2D sprite Entity for the background
- 2D sprites Entities for the GUI elements

Script TagComponent: Need-to-be-placed
Give tag to camera, background and GUI-sprites

Script Job: Place Entities that need-to-be-placed
- Place Camera
- Place Background
- Place GUI-Sprites at coordinate on Background
Would have to adjust for resolution...

Script Job SetUpCamera:
Set Camera settings (distance, angle, etc...)

Script TagComponent: Clickable
- Set Entity GUI-Sprite as Clickable

Script SystemComponent: Click
check mouse position
check for mouse input: click
if mouse position is on clickable entity & input is click then schedule a job.

Would that work for pure ECS/JobSystem?

 

I think the solution to most of the objections here is simply to create good OOP style UI authoring tools for DOTS to give us the best of both worlds. I believe the reason OOP is consider better than ECS here is simply because it's just easier to author such elements that way. But that doesn't mean you also have to orgianze your data and logic that way.

Well according to the roadmap presentation at GDC 2019: Jobs 1.0 release for 2019.1 and DOTS "Core API" 1.0 release for 2019.2. I'm guessing by "Core API" they mean the Entities package and it's dependencies, but they only released 0.1.0 So was this some kind of out of season April fools joke? Not funny Joachim, not funny.

 

I think the best current approach to this is Tiny Unity.
Much of what ECS is supposed to look and feel like is only found in tiny. Namely building hierarchies with entities.

Not sure what's the holdup.

 

I suppose they want to polish this feature in Project Tiny before moving it into ECS. Otherwise, they may have two different animals to take care of, on top of whole barn. Just my little guess

 

There is still time before the 2019.3 release and the Core API 1.0 release for 2019.2

 

@runner78 I've thought that could be a very remote possibility too, but I wouldn't hold my breath! I"m guessing it was delayed so they just released this "0.1.0" version hoping nobody would notice. Ha!

 

I believe the Tiny Unity stuff is what Unity intends the Editor to look like in the future, think of it like an "Alpha". They are changing the editor to be more ECS friendly, when the time is right.

 

It's not as easy as it seems to be. Tiny has still very poor graphics API. No shaders, meshes or command buffers. To write complex and performant UI, you need full control over vertex layout, shaders and draw calls execution. Preferably you want to batch as many controls as possible in one draw call or even cache static UI elements in RenderTexture. UI Development is a tradeoff hell. You can try to save CPU, GPU bandwidth, or GPU RAM, but you can't save them all. Depending on your project, expectations may differ and many UI frameworks will struggle with edge cases. For dynamic UI with different animated elements, batching is just another heavy task for CPU and GPU memory bandwidth, that won't optimize performance (in fact it makes things even worse). For static UI, you want to build hierarchy once and keep it in the very tight command buffer, bake into texture or even stop rendering loop (render on-demand only).

The question is: can you create a generic solution for all of these cases? I think it's possible... with ECS! Entity Component System is very flexible and makes it possible to share the same data and behavior for one part of the framework and use completely different systems in edge cases. Just change one component type, and another system will pick the new archetype. It's like with current UnityEngine.Canvas render mode. You can render UI in screen space or camera space, but it's still one bloated Component with branched logic. In ECS these "modes" can be defined as IComponentData (e.g. RenderToCamera, RenderToScreen, RenderToTexture, RenderOnDemand etc.). Then systems use queries to filter right canvases and render in the most optimal way for your needs. Code is clean and extensible.

These are the reasons why I started my own open-source ECS UI framework. I hope one day I (or someone else) will implement all of the optimal rendering paths for different needs. At the moment it's more like a research and learning project.

I bet Tiny UI won't be the main UI framework for "Big DOTS". Check this blog post and comments from Damian Campeanu:

That clearly tells us which direction Unity is going to follow. They want UIElements to be the only proper choice for UI Development for "Big DOTS", GameObjects and Editor. Tiny will probably stay with tiny UI since it needs a smaller code size. And that's ok. I still wonder how they are going to expose UIElements in DOTS.

 

They told, they changing tiny editor to DOTS conversion workflow instead, current tiny editor it’s not future, it was just try

 

@supron Yes, UIElements is how Unity plans to implement UI for DOTS in both the editor and runtime. Witch is very interesting and exciting. Best of luck with your research on the subject too.

 

I would be very interested if you could make a video or a page explaining what you said above:
- Why is performant UI needed?
- Why & how to batch controls in one draw call?
- Why & how to cache static UI in Render Texture?
- How come the UI struggles with Edge case and how to fix it?
- Why & How to render on demand?
- Batching for dynamic UI?

 

I don't know the answers to other questions, but to this:

Because some games are UI heavy. And current UI implementation is very inefficient. Every time you use any animation in UI you hit some sort of performance problem.
Most of these problems can be solved with multithreading. Unity devs say that you need to split your canvases to avoid rebuilding mesh for elements that have not changed. But even when we split them, they're still on the main thread. If all of the UI changes, Every canvas will rebuild its layout on mainthread.
But if you have ECS implementation, You can execute layout generation of each canvas on a separate worker thread. And you suddenly get a performance boost.

 

Hi everyone,

Two years ago we published a research article on engineering an ECS-based UI framework : https://hal.inria.fr/hal-02147180/document. The accompanying (very rough) proof of concept is at https://gitlab.inria.fr/Loki/PolyphonyECS/

TLDR:

• a basic vector-drawing application with single buttons, toggle groups, text input, and drag&drop
• input/output devices are represented as Entities -> no more global event queue (and no global storage apart from Entities/Components in our application), straightforward support for multiple mice/keyboards/etc, straightforward support for hotplug.
• Systems signal events by putting "temporary Components" on Entities, that are deleted at the end of every chain of Systems.
• Systems are Entities themselves, their running order is a Component, and a MetaSystem gathers all Systems to run them periodically -> in hindsight not so useful but we experimented...
• the ordered list of Systems that were designed for the sample UI is presented in figure 4
• the set of Components that were designed to represent the UI is presented in table 1
• metaprogramming is used in JS to get a more concise and readable syntax

After working for 3 years on this during a PhD I am now confident that a UI based on ECS makes perfect sense, even though there are still some pain points to resolve. I hope that you'll find the article interesting though, and would gladly discuss the topic here There is also an ongoing discussion at https://github.com/traffaillac/traffaillac.github.io/issues/1

 

After (2 years ago) my last post on here, I ended up implementing UI for my pure ECS game. I simply stored my characters as bytes, inside components. I made my own Text (or string) struct. Then I used that everywhere inside ECS. The bytes are then converted into meshes with appropriate font data in other UI systems. If you are using the same font looks, you can further optimize it, but i wanted everything to look unique. My game is UI heavy, and even with unique meshes and textures everywhere, it's extremely performant. ECS is very scalable.
@ traffaillac yeah basically everything can be ECS and works better that way. Because it's engineered for computers. But its also easier to manage because everything has an exact purpose. So it scales well for projects too.