ECS too verbose

The ECS code I'm reading or tried is far too verbose, tons look like boiler plate so I am giving up for now, since these are the very early days I think it will get better.

What is the plan to make ECS code more readable and accessible?

 

I think there is plan to integrate it to unity so then you will not be able to read it.

 

I wouldn't call it verbose exactly, but it does require a great deal of effort at the moment and is not very intuitive or streamlined, at least not for those coming from a Unity / OOP mindset. The pure ECS approach also currently requires you to essentially rewrite every system yourself, which kind of defeats the whole purpose of using an engine like Unity in the first place!

So I think at this moment the hybrid approach is ideal for beginners and is how I plan to start using this in projects. Since this allows you to gradually transition to this new way of doing things without having to relearn everything all at once, which can be overwhelming.

In the future entities and entity components will be tightly integrated into the editor just like good old classic components. However, if you're more of a visual design kind of person (like myself) then there seems to be some very exciting editor features in the works. The video below seems to be our very first glimpse of what they're working on, and I like what I'm seeing so far.

It would also be fantastic if someone from Unity could provide additional information about these new visual editor features they're working on. (hint hint)

 

The API does seem pretty heavy and geared towards low-level concerns, but I suppose if the animating concern for switching to ECS was performance that makes sense as a starting point.

I'm with you though in hoping that they don't stop there. Most ECS implementations see it as an extension of data oriented design - which it has an obvious affinity with - but it has a lot to offer as a general paradigm: it naturally pushes you towards small modular code; greatly simplifies dependency management; is easily testable; etc. It would be a real missed opportunity if those benefits were locked behind a wall of complexity that made them only accessible to the "engine" programmers on the team.

Side Note: For me at least, having a low-level API for "programmers" and an editor-centric visual interface for "designers" would be great but not sufficient. On small or one-man "indie" teams (which I suspect are well represented in Unity's user base) those roles blur a lot and it would be great to have a high-level API that allows "programmer/designers" to express and iterate on designs in code without being forced back into MonoBehaviours and having to define things in a somewhat arbitrary mix of code and editor based assets.

EDIT: Actually I take some of the above back. It's fine if there's only a low-level API as long as it exposes a lot of the engine and gives us the flexibility to create our own fully featured abstractions on top of it. What I would want to avoid is something along the lines of how now if you want to get access to engine events (e.g. Update, etc.) you have to create an object in the scene and attach a script to forward those events. If we can access everything without having to create objects in the scene, we can take care of the rest.

 

Oooh that's a tough one... how do I narrow down a general feeling of 5x character count into one example.

Let's try this one:

Code (CSharp):

1.             entityManager.SetComponentData(player, new Position2D {Value = new float2(0.0f, 0.0f)});
2.  

And this

Code (CSharp):

1.         public static void SetupComponentData(EntityManager entityManager)
2.         {
3.             var arch = entityManager.CreateArchetype(typeof(EnemySpawnCooldown), typeof(EnemySpawnSystemState));
4.             var stateEntity = entityManager.CreateEntity(arch);
5.             var oldState = Random.state;
6.             Random.InitState(0xaf77);
7.             entityManager.SetComponentData(stateEntity, new EnemySpawnCooldown { Value = 0.0f });
8.             entityManager.SetComponentData(stateEntity, new EnemySpawnSystemState
9.             {
10.                 SpawnedEnemyCount = 0,
11.                 RandomState = Random.state
12.             });
13.             Random.state = oldState;
14.         }

and that

Code (CSharp):

1.             PostUpdateCommands.CreateEntity(TwoStickBootstrap.BasicEnemyArchetype);
2.             PostUpdateCommands.SetComponent(new Position2D { Value = spawnPosition });
3.             PostUpdateCommands.SetComponent(new Heading2D { Value = new float2(0.0f, -1.0f) });
4.             PostUpdateCommands.SetComponent(default(Enemy));
5.             PostUpdateCommands.SetComponent(new Health { Value = TwoStickBootstrap.Settings.enemyInitialHealth });
6.             PostUpdateCommands.SetComponent(new EnemyShootState { Cooldown = 0.5f });
7.             PostUpdateCommands.SetComponent(new MoveSpeed { speed = TwoStickBootstrap.Settings.enemySpeed });
8.             PostUpdateCommands.AddSharedComponent(TwoStickBootstrap.EnemyLook);

By the way, why is it math.normalize and not Mathematics.Normalize?

 

Another good example is TransformSystem. Haven't done a "scientific" count yet but the boilerplate/ code ratio feels like 6:1. That's without scaling and "proper" hierarchy support.

To be fair, getting the clowns out of the car still leaves you with a bunch of clowns. However, imho, it's better for the clowns to be working the factory ( boilerplate ) than slowing down your car.

At the end of the day, how many clowns we get out of the car is still up to us. As in, instead of implementing the best performance case ( separating transform components to ?fit within cache lines? and only update what is used ), you could just roll them into one component/ job and be done with it.

I prefer the clowns to be out of sight of consumers. My only worry is how the regular unity devs ( hobbysts, one man indie shows, students, jammers, which prefer human-understandable code and super fast iteration ) will react to the final ECS system. If adoption is low ....

 

Am I misunderstanding you because how is what you showed any more verbose then the a similar method with the old component system? Its 12 more characters and once you start adjusting multiple properties per component and adding multiple components you wind up typing up a lot more code the old way.

Code (csharp):

1.  
2. var someVar = playerGo.AddComponent(SomeType);
3. someVar.Value = Vector2.zero;
4.  

Personally I load most of my assets from bundles then attach the scripts at runtime and I feel like I have been overall typing up way less code with the ECS than with the old system.

 

Interesting !

Personal opinion (from an ECS beginner )

• My expectation would be that the API get's as simple as possible but not any simpler - in the end it's C# and it's OK to type some code (example: while I like the injection, I would also be OK to just type it out with get component groups. I prefer if there is only 1 way of doing the same thing and this way is properly documented)
• I think the ECS lends itself to some way of a visual user interface with code generation since it consists of fairly standardized blocks (I never used a visual coding tool, but I could see it totally for ECS) - for entities, components, systems and world management

 

This pops into my head whenever I try to imagine the medium-long term future of ECS & Unity. I hope it ends up making sense and is the way Unity go.

 

@Joachim_Ante, love the generic and ref return (it's that "=" I guess), pays off to have upgraded c#
While you have the hedge trimmer, CreateEntity could also go generic. Same for everything that takes type as argument.

Code (CSharp):

1. CreateEntity<EnemySpawnCooldown, EnemySpawnSystemState>();

 

That's an important discussion to have early because jobs confuse the hell outta me when people don't put read/write attributes, what's the default in this case, read only? like when you don't put public in front of a class' variable? One way to do things and that way is super concise and at the same time clear of intention.

I'm more a type less, read less sorta guy and since I've only learned c# from the version before unity 2017, c# always felt too verbose. But with the transition to c# 6, we're at the cusp of a change similar to when unity API got its generics overloads (you guys & gals remember when you had to pass a type? ) less code, clear intention

 

You're right, going with = in this case is more chatty, I don't know why I gave this example, must be a temporary allergy to constructors.
Regarding the instantiation and ECS, that's what I noticed: those like you who build their assets by code have cleaner tighter code with ECS. And reading through the 100K soldier example is crystal clear because the points of entry to the ECS systems are very rigid. It's just a long process at the moment.

The funny thing with reading the that example is I'd never bother if it were OOP because everybody got their own pasta flavor and following the meandering trail of calls isn't fun.

Story time: I'm programming microcontrollers nowadays, all in C, it's all arrays of data, very linear, when you need a thread you call it by hand, very concise functions, it's relaxing.

 

What does a GameObject-less prefab look like? That implies some kind of container that can have components (like IComponentData) attached to it, but doesn't have a transform.
Unity already supports putting ScriptableObjects as a part of a scene file. You have to create custom tools to edit that stuff, though. Is that the route you're going down, or are you planning on introducing a new concept entirely? Or is this not decided yet?


The important part about Unity that needs to not get lost in all of this is the inspector. No matter how fast the code runs, if I can't make things my designers and artists can click on and fiddle with the values on in the inspector, it's not really usable. That's what the pure approach looks like right now; the demo has a bunch of spaceships on the screen, and the non-programmers on the team doesn't seem to have any good way to interact with them at runtime, which means that I suddenly become a bottleneck on everything.

 

I agree completely. For the prefab question, they could flesh out the Archetypes concept?

 

So in a typical ECS system you start with entities and components. The components on an entity are basically what drives the behavior. Even though it's systems that perform the work. This is because data driven in the ECS context means behavior is data driven. It was really never intended to describe the benefits you can get in performance related to cpu cachelines and such.

It's confusing to mix up those two concepts, you need separate terminology. I see a lot of newcomers missing one of the whole key points of ECS because of this.

It's not at all clear from where the abstractions are that you have two distinct and separate parts of an ECS system. You have your components and how you model them, and the data driven part here is that components drive behavior of entities via which components you attach to an entity.

Which is related to but distinctly different from the systems that use components to implement the behavior functionality.

Judging by the posts people are making, if they were not already familiar with ECS the above is just completely lost on them.

For example, you should be manipulating which components go on which entities via the entities themselves. Not the entity manager. Abstraction is in the wrong place.

It's little things like that where the abstractions cleanly map to the domain model that really make the difference between an elegant easy to understand api, and well what we have now.

Like the shared components. That's almost an anti pattern in the context of ECS.

Systems overall seem ok. If anything the needs of systems are leaking out into places they don't belong.

It does seem that within a system jobs should just be abstracted away. So that maybe a system runs in a job context, but you aren't creating jobs within systems.

 

Agree with the terminology part. Say now Component is the traditional component, ComponentData is Component in ECS, ComponentSystem is System but with a "Component" prefix. Please, make it less confusing.

 

I get how ECS works, been using/implementing them for almost a decade now. The only reason I bring that up is so people stop going over the basics so we can move beyond.

You need the concept of components attach to entities, or you have no good way to reason about the composition of logical entities.

That's why most ECS's are designed so that components attach to entities. The systems don't even use that information for the most part, but it's a useful and I would say necessary abstraction.

Which is different from how you define components.

Where I see the design going wrong, is a tendency to conflate implementation with design. It doesn't matter if the implementation is components don't actually attach to entities at the C# level, that's implementation after all. The api should follow the design not implementation.

 

I've been a fan of Entitas' approach to Entity-Component-System architecture. It utilizes heavy code generation to take care of boilerplate on the back-end, but the results are incredibly concise and clearly present their intent:
https://github.com/sschmid/Entitas-CSharp

 

Definitely the way ahead IMHO...

Code (CSharp):

1. var entities = context.GetEntities(Matcher<GameEntity>.AllOf(GameMatcher.Position, GameMatcher.Velocity));
2. foreach(var e in entities) {
3.     var pos = e.position;
4.     var vel = e.velocity;
5.     e.ReplacePosition(pos.value + vel.value);
6. }

vs

Code (CSharp):

1. using Unity.Entities;
2. //using Unity.Transforms2D;
3. using Unity.Transforms;
4. using Unity.Mathematics;
5.  
6. using UnityEngine;
7.  
8. namespace ECSSpaceInvaders
9. {
10.  
11.     public class EnemyMoveSystem : ComponentSystem {
12.  
13.         public struct Data
14.         {
15.             public int Length;
16.             public ComponentDataArray<Position> Position;        
17.             public ComponentDataArray<Heading> Heading;
18.             public ComponentDataArray<MoveSpeed> MoveSpeed;
19.             public ComponentDataArray<Enemy> Enemy;
20.         }
21.  
22.         [Inject] private Data m_Data;
23.  
24.         protected override void OnUpdate()
25.         {
26.             var settings = SpaceInvadersBootstrap.Settings;
27.             Rect r = settings.playfield;
28.  
29.             r.xMin *= 0.95f;
30.             r.xMax *= 0.95f;
31.  
32.             float dt = Time.deltaTime;
33.  
34.             float3 move = dt * settings.invaderMoveDirection * settings.enemySpeed;
35.             float3 position;
36.  
37.             for (int index = 0; index < m_Data.Length; ++index)
38.             {
39.                 position = m_Data.Position[index].Value;            
40.  
41.                 position += move;
42.  
43.                 if (position.x > r.xMax)
44.                 {
45.                     position.x = r.xMax;
46.                     settings.invaderMoveDirection.x = -1f;
47.                 }
48.  
49.                 if (position.x < r.xMin)
50.                 {
51.                     position.x = r.xMin;
52.                     settings.invaderMoveDirection.x = 1f;
53.                 }
54.  
55.                 m_Data.Position[index] = new Position{ Value = position };                            
56.             }
57.  
58.             SpaceInvadersBootstrap.Settings.invaderMoveDirection = settings.invaderMoveDirection;
59.         }
60.     }
61. }

Compared to setting up the data used then writing an Execute function and a JobHandle wrapper and all those [Inject] and [ReadOnly] and [NativeDisableParallelForRestriction] this is definitely the way Unity ECS should go.

 

As I pointed out in the other thread, the examples above are really not a fair comparison.

While I do agree that there is too much overhead and that more code generation can help I did have problems with Entitas code generation breaking on me in the past when trying to do more low level work and therefore losing control.

 

Your Entitas snippet corresponds to the following in ECS:

Code (CSharp):

1. // This recreates the group and is a bad idea both in UnityECS and in Entitas.
2. var data = GetComponentGroup(typeof(Position), typeof(Velocity));
3. var pos = data.GetComponentDataArray<Position>();
4. var vel = data.GetComponentDataArray<Velocity>();
5. for (int i = 0; i < data.Length; i++) {
6.     pos[i] = new Position(pos[i] + vel[i]);
7. }

which is about the same in verbosity. Same for the version where you store the group (line 2. moves to constructor(Entitas) or OnCreateManager(ECS)). [Inject] API is in fact more verbose, but is afaik a bit faster than GetGroup and has no direct equivalent in Entitas.

My problem with code generation is that you get a little less verbose code at the cost of workflow issues. Ever tried renaming a component in Entitas? Rename -> Regenerate -> Recompile -> Fix compile errors the regeneration introduced when it deleted the methods with old name (no refactoring tool will help here) -> Recompile. It's really hard to avoid this problem for a codegen tool that generates code visible to the user.

Not saying the current ECS API is perfect, I for one would love to be able to just write:

Code (CSharp):

1. class MySystem : ComponentSystem
2. {
3.     public void Update(int Length,
4.                        ComponentDataArray<Position> pos,
5.                        ComponentDataArray<Velocity> vel)
6.     {/* ... */}
7. }

and have all the args injected, but I'm fairly sure this would require code generation (or reflection, but that's just as evil).

Would be nice to have some reasonable constructors for the built-in components, so I can say new Position(x, y, z), and not new Position() {Value = new float3(x, y, z)}.

Most other verbosity problems I've found will either disappear with C#7, or can be solved with convenience shortcut extension methods:

Code (csharp):

1. // Saves you from typing PostUpdateCommands all the time
2. static void AddComponentDeferred<T>(this G1.DrawCardsSystem s, Entity e, T c)
3.         where T: struct, IComponentData
4.         => s.PostUpdateCommands.AddComponent(e, c);

I'm fine with those being defined on the user side if someone wants them.

 

@Arowx Comparing a snippet of code to a complete script that handles a completely different set of components and function seems very deceitful here in my opinion. A much fairer comparison would be something like this:

Entitas

Code (CSharp):

1. using Entitas;
2.  
3. public sealed class MoveSystem : IExecuteSystem {
4.  
5.     readonly IGroup<GameEntity> _group;
6.  
7.     public MoveSystem(Contexts contexts) {
8.         _group = contexts.game.GetGroup(Matcher<GameEntity>.AllOf(GameMatcher.Position, GameMatcher.Velocity));
9.     }
10.  
11.     public void Execute() {
12.         foreach (var e in _group.GetEntities()) {
13.             var vel = e.Velocity;
14.             var pos = e.position;
15.             e.ReplacePosition(pos.x, pos.y + vel.value, pos.z);
16.         }
17.     }
18. }

With Entitas you also need to execute the systems yourself:

Spoiler

Code (CSharp):

1. using UnityEngine;
2. using Entitas;
3.  
4. public class GameController : MonoBehaviour {
5.  
6.     Systems _systems;
7.  
8.     void Start() {
9.         var contexts = Contexts.sharedInstance;
10.         contexts.SetAllContexts();
11.  
12.         _systems = createSystems(contexts);
13.         _systems.Initialize();
14.     }
15.  
16.     void Update() {
17.         _systems.Execute();
18.     }
19.  
20.     Systems createSystems(Contexts contexts) {
21.         Add(new MoveSystem(contexts));
22.     }
23. }

Unity ECS

Code (CSharp):

1. using Unity.Burst;
2. using Unity.Collections;
3. using Unity.Entities;
4. using Unity.Jobs;
5. using Unity.Transforms;
6. using UnityEngine;
7.  
8. public class MoveSystem : JobComponentSystem {
9.  
10.     [BurstCompile]
11.     struct Move : IJobProcessComponentData<Position, Velocity> {
12.         public float dt;
13.         public void Execute (ref Position pos, [ReadOnly] ref Velocity vel) {
14.             pos.value += dt * vel.value;
15.         }
16.     }
17.  
18.     protected override JobHandle OnUpdate (JobHandle inputDeps) {
19.         var job = new Move () { dt = Time.deltaTime };
20.         return job.Schedule (this, 64, inputDeps);
21.     }
22. }