ECS System Order Editor

Update groups and fine tunning the execution order is terrible right now. So i made this:

I'm gonna test it on our live project and will rls the source file for free soon(ish).

 

Ability to set execution order of systems from code is one of the biggest benefits of ECS. Someone may like this tool, but for me it is a step back.

 

I see put effort there.
However, if the tool means to change [UpdateBefore] [UpdateAfter] in the scripts itself, then asset could be ok.
Yet ensuring that ECS project won't get screwed without asset, or when uninstalled, if order is stored in some additional file.
But what I have noticed on very end of the video, even I just skimmed through it, you use MonoBehaviour FixedUpdate, LateUpdate etc. This misses the point of decoupling from Unity OOP. Or I missed something?

Depending how system is suppose to be done, but I see potential issue, when adding new systems, with order header, via pure scripting.

 

loosing the asset doesn't matter because it generates code. you can set the execution order to match whatever is currently in the unityengine.experimental.playerloop , there was a mono notasystem.cs put it the systems folder to show that it only detects files that inherit the ComponentSystemBase. So maybe that's what you saw.

Cant agree. You clearly didn't run (yet) into any problems when you have lots of systems and you need to ensure that they are all updated in the exact order you want. Groups are terrible and sometime even if a system is in a group it will go out of fixed update to update etc. Inserting new systems is also a chore. This solves this all (it does the writing for you)

 

Why would I need them to be updated in the exact order? All I need is to have them updated after the systems they depend on.

 

I have over 70 systems at the moment and ordering was becoming a huge nightmare. Instead of trying to find a better way to order them I took a step back and had a think about if I actually needed to.

In the end I decided it was a lot better and maintainable to not have system order dependencies. I set a few guidelines for myself and managed to get get my UpdateBefore/UpdateAfter count down to 2 which I hope to remove at some point, both for a 3 stage AI utility system

My guidelines have also significantly reduced the amount of bugs and errors I run into as well.
The most important thing I've found is ensuring that when setting a tag component, the system that sets it and removes it has to be the same.

 

When I was doing my project in OOP, with many subsystems, for me most important was the order of initialization. Later order wasn't as that critical as such, since all was driven by logic. Something as perhaps equivalent. what tertle stated. I will try to implement similar approach in ECS, if I will be able to, avoiding system dependencies ordering. I believe that tagging is good approach, if I understood you correctly.

I think with ordering otherwise, it become pain in backside, when adding new systems, and trying fit in between already ordered once. I suspect, it can be very bug prone.

 

And this tool does the exact same thing! It generates code using UpdateAfter and UpdateBefore.

I don't see why would you prefer to manually open 70+ files, order systems in your head, and than manually edit and check each of them just to paste 1-2 lines of code.
Having all of them in one slick reorderable list can be a huge benefit to your mental health and stability.

 

What tertle means, is that he don't need create dependencies between every system. Only for these, which does require ordering. So for example, from 70 systems, maybe only 10 need dependencies. And these dependencies pairs/groups, may be independent from other pairs/groups.

 

There must be some huge misuderstanding, because I really don't get why I should open those 70+ files and change something in them, why would I do that? Could you please explain it to me?

 

@mikii123 it is not the same since the tool order system in linear ordering. For example we have A B C D E in order. I am assuming how you achieve this is by putting a chain of UpdateAfter/Before one by one on each system to guarantee this order. Probably requiring only one of these attribute if you put them all in chain neatly without skipping. And so you have an idea of having to edit all 70+ files to guarantee a new ordering if you want to insert a small system in an exact place. This is correct if you use [UpdateBefore/After] in this chaining way because the chain would break if you are not careful and the tool will help.

But what @Fido789 meant by "depend on" means it can be at any point after the dependency, and so not forcing the hard order. For example D needs data (depends on) from A but not really related to B and C. And then E requiring data from A C and D. The order can end up the same way as A B C D E like the tool produce, but an order like B C A D E is also valid. Using [UpdateBefore/After] this way will requires multiple attributes per system but the intent is different, it is not for specifying the exact place in the chain but any place that guarantee the correct behaviour. If we use the attribute in this intent, there is no need to open any other file to edit because they will surely still land at the correct place. We then need only to specify enough [UpdateBefore/After] on this new file to make it land on the correct place automatically too. (And so what @Fido789 said he think there are some misunderstandings)

To translate ordering based visualization to dependency based, the tool's interface implies that B depends on A, then C depends on "both" A B, then D depends on "all" A B C, and so forth. If all of your later running system really do need all prior system to be run to function properly then the tool is great.

But usually this is not the case as system running later in the chain ended up there not because they need all the former but only some. In script based editing we can specify this intent by putting as many [UpdateAfter] as we want to a given system without caring about their eventual linear ordering resulting from the work of automatic attribute analyzer. And so this is the point of someone who do not want the tool because linear list is not the correct interface for specifying dependencies. And honestly I can't think of other good way to do dependency spec than editing the attribute in the script since we get auto complete there too and scripts are already nicely segmented, IDE can jump, and IDE can refactor in one go

 

@mmankt Do you still maintain this tool?
I want this, because I spend half an hour trying to tune my few systems (force calculation, movement, rendering).
Even a little number of systems can lead to a headache.

 

it's the pillar of our work with the game. we have 128 system s right now and it would be immpossible to keep it deterministic and acting as we want without it. i'll see about the source.

 

Hi, Could you open source your ECS system order editor ? thanks

 

@quanquan123666 since 2 years, as DOTS evolved by much, there are few ways to organise you systems.

Probably one of most suitable now, is systems group. Besides, there always UpdateAfter / Before keywords. And manual system updates.

 

I created the same kind of thing: https://github.com/tbg10101/DOTS-Hybrid-Simulation-Worlds

 

Genuinely, I’m curious to hear what problems people have been having using the system order attributes.

even with a lot of systems, it seems like there’s always a way to organize things the way you want.

The two rules I usually follow are:

1. only use of of the order attributes, rather than mixing them (UpdateBefore or UpdateAfter).

2. Take advantage of nested groups, when you need them.

 

1) Using systems in multiple projects without introducing compiler errors because the new project doesn't have a system referenced by an attribute.
2) Updating a system in multiple groups in which there are different ordering requirements in each group. (This might work now, but didn't when I built my alternative solution.)
3) Reordering systems whose archetypes don't depend on each other but their component types alias in ways that create thread occupancy issues.
4) Inserting debug systems which capture the state directly before and after a particular system.

 

Thank you for all of those answers.

Thoughts on some of these, fwiw. Not saying these are right, or right for everyone:

We avoid this by using a shared framework between projects. There end up being only a few major groups that systems need to live inside of, and most of the system order dependencies are handled on the group level - not the individual systems.

It’s worked for us so far, but I’m not saying it’s a panacea at all.

I’m having a hard time visualizing this one. Is there a different way to explain it?

Same here...How would you explain it to a child?

It seems like you could solve this one by putting both the target system and the following debug system in their own sub-group. The group would contain only those two systems, and would be strictly ordered. Both of the systems and the wrapper group might be defined in the same file, as well.

I personally wouldn’t consider this type of thing too cumbersome, but I can understand how some might.

Thank you again.

 

Have you ever executed a system group more than once in a frame? Have you ever noticed that the TransformSystemGroup has its own sync point? If you answer "no" to both of these, I unfortunately cannot explain it to you now.

I probably wouldn't. But if my hand was forced, I would show them a picture of the worker threads with a lot of idle gaps between jobs, and then another picture without those gaps where the frame is 2-4 times shorter. I will have those pictures when I release v0.3 of my framework. I'm still working on filling one more idle gap.

The group would contain three systems, and now everything that referenced the real system now has to reference the group instead. Also, every time I do this for a system, I am creating a new ComponentSystemGroup that I have to pay for at runtime, even if I am not debugging that system.

 

The answer is actually yes to both! So please do, if you’d like.

I was mainly confused about what you meant by ‘alias’ there.

I’m sure you’ve thought of this, but if the cost of updating extra systems is a concern, could you eliminate the debug system altogether? Could that debug code be embedded at the end of the target system’s Update function?

 

So I was originally going to discuss a use case where TransformSystemGroup gets updated in multiple places during the frame and how another system which also updates in multiple places during the frame might have problems due to OrderFirst and OrderLast. But then I remembered a dirty hack exists where you write a wrapper system that calls World.GetExistingSystem<>().Update() to bypass the attributes. And I think this functionality can now be replicated with multiple ComponentSystemGroups which only contain the system in question (because a system can be injected into multiple groups now I think).

Got it. That I can clarify. I have two systems with a job each. ShipsSystem reads ships transforms in parallel. BulletSystem writes bullets transforms single-threaded. A ship will never be a bullet and a bullet will never be a ship. These systems are totally independent. But if I don't run ShipsSystem before BulletsSystem, then the ships job will get stuck waiting for the single-threaded bullets job to finish, and the rest of my worker threads go temporarily idle. Whose responsibility is it to resolve this? ShipsSystem? BulletsSystem? Or some higher-level manager that can explicitly order and resolve these issues without ShipsSystem and BulletsSystem having to reference it or each other? (I'm sure you can guess my preference. )

Debug systems require more than just the OnUpdate callback. The debugging systems chew up a lot of RAM so I try to keep the number of instances down and instead move them around in the player loop. That's really easy to do with explicit ordering. It is borderline impossible to do with injection.

 

This is the one case where I think a node graph would be better, to also Link the dependencies.