ECS - Rebuild of Unity's upcoming - Entity Component System - FREE

Follow us on git!
https://github.com/Spy-Shifty/BrokenBricksECS

Hello everyone!

I want to do something good to all of you!

I like the new ECS approach of unitys tech-team! But I cant wait for it... So I've build my own one, that's similar to Unity's upcoming.

And because of that....
It's purposeless to sell it on the store... So I decided to publish it for FREE!
(attachment)


Whats the benefit...

• Huge speedup in relation to the (legacy) Unity System
• Easy maintenance and extensibility
• System debugging like Entitas ECS
• Inversion of control (dependency injection)
• No Garbage Collecting (only if you do live debugging with components on gameobjects, but you can turn on and off in playtime)
• TupleInjection for multiple components in systems (only entities that matches all tuples)
• Easy to use components, supports structs and classes
• Easy to extend for multithreaded Systems (but not yet implemented)
• Fast system and component creation with templates
• Unity monobehaviour integration
• Supports data prefetching (fetches data before it is needed)
• Creating entities from Prefabs
• Supports different contextes for entity (just by creating new classes of RootSystems and EntityManagers)
• No Object pooling required

Here is an short example of usage:
Controller Executes all the systems

Code (CSharp):

1.  
2. namespace ECS {
3.  
4.     // Use this class to control the ECS System
5.     public class GameController : ECSController<UnityStandardSystemRoot, EntityManager> {
6.  
7.         // Use this for initialization
8.         protected override void Initialize() {
9.             AddSystem<MySystem>();
10.  
11.             // or you can use this for more controll over compositions of systems
12.             //AddSystem(new MySystem());
13.         }
14.  
15.         // Override this function if you want to controll what Scene Entity should be load in this context
16.         // The base implementation will add all Scene Entities to the context
17.         //protected override void AddSceneEntitiesToSystem() {
18.  
19.         //}
20.     }
21. }

It creates the system and the entity manager by dependency injection for you
With the AddSystem methods you can assign your systems to the ECSSystemRoot.
The ECSController also executes the Start, Update, and FixedUpdate Routines of the Systems

An ECS Component

Code (CSharp):

1. using ECS;
2. using System;
3.  
4. namespace ECSExample {
5.     [Serializable]
6.     public struct FloatComponent : IComponent {
7.         public float value;
8.  
9.         public FloatComponent(float value) {
10.             this.value = value;
11.         }
12.     }
13.  
14.     public class FloatDataComponent : ComponentDataWrapper<FloatComponent> { }
15. }
16.  

Initialize Entity System

Code (CSharp):

1. using ECS;
2. using ECS.VisualDebugging;
3. using UnityEngine;
4.  
5. namespace ECSExample {
6.     [DebugSystemGroup("Init")]
7.     class InitEntitiesSystem : ComponentSystem {
8.  
9.         private GameObject _gameObject;
10.  
11.         [InjectDependency]
12.         private EntityManager entityManager;
13.  
14.         public override void OnStart() {
15.             _gameObject = new GameObject("Entities");
16.             for (int i = 0; i < 1000; i++) {
17.                 Entity entity = entityManager.CreateEntity();
18.  
19.                 GameObject gameObject = new GameObject("Entity-" + i);
20.                 gameObject.transform.SetParent(_gameObject.transform);
21.  
22.                 GameObjectEntity goEntity = gameObject.AddComponent<GameObjectEntity>();
23.                 goEntity.SetEntity(entity, entityManager);
24.                 entityManager.AddComponent(entity, new FloatComponent(1f));
25.             }
26.         }
27.     }
28. }
29.  

Update float component of all entities

Code (CSharp):

1. using ECS;
2. namespace ECSExample {
3.     [DebugSystemGroup("Update")]
4.     class UpdateFloatSystem : ComponentSystem {
5.  
6.         [InjectDependency]
7.         private EntityManager entityManager;
8.  
9.         [InjectTuple]
10.         private ComponentArray<FloatComponent> floats;
11.         public override void OnUpdate() {
12.             float sum = 0;
13.             for (int i = 0; i < floats.Length; i++) {
14.                 entityManager.SetComponent(floats.GetEntity(i), new FloatComponent(floats[i].value + 1));
15.             }
16.         }
17.     }
18. }
19.  

Accessing Components in Systems

Code (CSharp):

1. //This class see only Enities with ComponentA and B attached to it
2. class MySystem : ComponentSystem {
3.     [InjectTuples]
4.     ComponentArray<ComponentA> componentA;
5.  
6.     [InjectTuples]
7.     ComponentArray<ComponentB> componentB;
8. }
9.  
10. // if you want to manualy will filter components use the following:
11.  
12. ComponentGroup group = m_EntityManager.GetComponentGroup(typeof(ComponentA), typeof(ComponentB),...)
13.  
14. ComponentArray<ComponentA> compA = group.GetComponentArray<ComponentA>();
15. ComponentArray<ComponentA> compB = group.GetComponentArray<ComponentB>();

Instantiate from Prefab without instantiating the GameObject:

Code (CSharp):

1.  struct Position : IComponent {
2.     public Vector3 position;
3.     public Quaternion rotation;
4. }
5.  
6. PositionComponent : ComponentDataWrapper<Position>{ }
7.  
8. class Spawner : Monobehaviour {
9.     public GameObject prefab;
10.  
11.     [InjectDependency]
12.     EntityManager _entityManager;
13.  
14.     void Awake() {
15.         InjectionManager.ResolveObject(this);
16.     }
17.  
18.    void Start() {
19.         // Instantiate the prefab with all its components attached to it
20.         Entity entity = _entityManager.Instantiate(prefab);
21.  
22.         // just update the position component
23.         var position = new Position(Vector3.zero, Quaternion.identity);
24.         entityManager.SetComponent(entity, position);
25.    }
26.  
27. }

Instantiate Entities from Prefab with instantiating the GameObject

Code (CSharp):

1.  public class PrefabSpawner : ScriptBehaviour {
2.         public GameObject _prefab;
3.  
4.         [InjectDependency]
5.         private EntityManager _entityManager;
6.  
7.         // Use this for initialization
8.         void Start() {
9.             GameObject gameObject = _entityManager.InstantiateWithGameObject(_prefab, transform.position, transform.rotation);
10.         }
11.     }

Components that supports Unity Component

Code (CSharp):

1.  
2.     [Serializable]
3.     public class ECSTransform : IComponent, ICloneable {
4.         public Transform transform;
5.  
6.         public object Clone() {
7.             return MemberwiseClone();
8.         }
9.     }
10.  
11.     [HideInInspector] // dont show inner component in Inspector
12.     public class ECSTransformComponent : ComponentWrapper<ECSTransform> {
13.         // This will assign the Unity Component to the ECS Component
14.         public override void Initialize() {
15.             TypedComponent.transform = gameObject.transform;
16.         }
17.     }

Creating of different Contexts

Code (CSharp):

1. //context for Cars
2. class CarEntityManager : EntityManager{}
3. class CarRootSystem :  UnityRootSystem<CarEntityManager>{}
4.  
5.  
6. //context for Humans
7. class HumanEntityManager : EntityManager{}
8. class  HumanRootSystem :  UnityRootSystem< HumanEntityManager>{}
9.  
10.  
11. //usage
12. class Controllers : Monobehaviour {
13.        CarRootSystem carSystem;
14.        HumanRootSystem humanSystem;
15.  
16.        void Awake() {
17.             carSystem = new CarRootSystem ();
18.             humanSystem= new HumanRootSystem  ();
19.  
20.             ... add systems to the rootsystems
21.       }
22.  
23.       void Start() {
24.             carSystem .Start();
25.             humanSystem .Start();
26.      }
27.  
28.       void Update() {
29.             carSystem .Update();
30.             humanSystem .Update();
31.      }
32.  
33.  
34.       void FixedUpdate() {
35.             carSystem .FixedUpdate();
36.             humanSystem .FixedUpdate();
37.      }
38. }
39.  

This will enforce that you will separate enitities, components and systems
Systems of context Car only knows entities of that system and so on.
If you want to communicate with the other Context use the EntityManager of that context

Its almost identical to the video of Joachim Ante from Unite Austin 2017

Great thanks to him and his team!!!



It took me a week of afternoons and a whole weekend to write it that performant!
So please enjoy it and fell free to use it.

It's not fully completed yet and tested.
I also want to implement some basic components: like transform and so on.

Here are some pics for you:
Entitas like system debugging


Enitity Debugging live in GameObjects


Easy Setup for template script integration (be aware its global)


Template Scripts

The GC pike in the image comes of enable debug mode of an entity (for confusion)

IOC:
it works with any class you want.
Just call InjectionManager.Resolve<myClass>() or if the object already exist use InjectionManager.ResolveDependency(my object)

A injectable class should have the [InjectableDependency] Attribute on it
And to inject some object to your Fields and Props use [InjectDependency] Attribute
The system will automatically recognize dependencies of constructors

Example

Code (CSharp):

1. [InjectableDependency(LifeTime.PerInstance)]
2. class MyClass {
3. ... some code
4. }
5.  
6. class SecondClass {
7.     [InjectDependency]
8.     private MyClass myClassInstance;
9. }
10.  
11. class ThirdClass {
12.     public ThirdClass(MyClass myClassInstance) {
13.         ... some code ....
14.    }
15. }
16.  
17. ... somewhere in the programm
18.  
19. InjectionManager.Resolve<SecondClass>();
20. InjectionManager.Resolve<Thirdclass>();
21.  
22. // or
23. SecondClass mySecClassInstance = new SecondClass();
24. InjectionManager.ResolveDependency(mySecClassInstance);
25.  

If you want to use Dependency Injection in a Monobehaviour class,
just use ScriptBehaviour instead of Monobehaviour.

Code (CSharp):

1.     public class SomeMonoBehaviourClass: ScriptBehaviour {
2.  
3.         [InjectDependency]
4.         private EntityManager _entityManager;
5.  
6. }

If you have some questions, tips or hints, feel free to contact me!
Again, enjoy it!

Kind regards,
Spy-Shifty

 

I've made some Updates to the documentation above like usage of different contexts, component access and so on!

It would be nice if you can give me some feedback



EDIT:
I updated the zip! I found some bugs that i solved now!
There is one more with class comonent types if you instantiate them from prefab.
I'll look how to solve that and submit a new Version. Maybe tomorrow.
I'll also investigate time on how to integrate Unity basic Monobehaviour classes into the component system

I'll let you know!

 

Ok I've found a solution for Unity Components and the problem with instantiating components of class type.
I also fixed some more bugs with the system debugger. I've added one more example for spawning GameObjects that using ECS and which are interacting with the Transform component.

And one more: I added some more lines of code for documantation at the top post in this thread.



Something to class and struct components:
class components are only introduced to support Unity Components
they should not be used in other way!

structs are much faster! Thats because of structs are valuetypes and will be stored right after each other in memory

If you use struct data, they are tightly packed right after another. The Dataprefatching will happend and that gives us a hughe performance boost.

If you use classes just the pointer to that class are packed right after another.
The instance of the class itself is far away. So to load a class somewhere in the RAM, it will takes ~100 to ~300 cycles. That is bad for performance and the reason to use structs instead.

 

@Spy-Shifty I am impressed.

1) How do you accomplished this in a week? Did you use any kind of framework or are you crazy?

2) Do you see any possible problem to use this in the wide range of deployment targets, that unity provides (Mobile, Desktop, TV)? Does it work for Unity 5.4.5 and higher?

3) You and Andy described, too use structs and fields with no reference types to keep the data inline in the memory. I am working at a moment on a solution to filter a array with a bunch of objects by values. One of the properties contains a string (for text search). But If I can not use reference types (which a string is), how would you solve this?

 

Hi and thank you.

1)Well I've been with it for a longer time. I worked with different kind of ECS Systems like Entitas and SveltoECS.
I've only analysed what Joachim Ante said at the Unite Austin 2017. And because I cant wait for it,... I started to write my own. So its not out of the box my ideer. I've only implemented what Joachim said.
Yes maybe I'm crazy and no I don't use any framework. It's all self implemented. (Except the visual system debugger, its from Entitas. Thanks to Simon Schmid from Wooga)

2) Yes for sure you can use it on any device/system! I would recommend to use it. Because of the performance it saves you energie (mobile platform). You can use it even without unity. The unity part is just build on top of it. And yes it works for any version of unity.

3) Well I dont have a right answer for this right now. I would sugest no. Any typ of array is indeed an reference type in C#. So a workaround could be using hashes of that string. But this will only work if you search the hole string. And we sould not forget... Unity has its own compiler. Maybe they handle things slightly different in memory.

 

Hi Spy-Shifty. Looks very good. Thanks.

I was looking at Entitas and Svelto recently.

Your approach looks more simple to me then the other two. How production ready would you describe your system?

 

Well, the software right now is more a beta. I start to test it in live production right now. And I still find some bugs and things to improve. But its more the Unity part of the software. The ecs system it self works till now with no problems.

As soon as I can improve or fix a bug I'll release a newer version of it. So that you can stay up to date. And I also hope that the community can help me to find bugs and send me feedback to improve the software too.

Later in the day, I will release an update. I added a more easy to use controller for the root system with the ability to add entities of sceneobjects automaticly to the entitymanager. And also fixes for some issues.

On the whole I believe that the software is soon in a state where it is really stable.

There are some features I'll implement too. For example an entity inspector. A graphic window there you can inspect entities and it's components (not every entity is a gameobject...).

 

That's great.

I will try to play with it also. I'll let you know if I stumble on anything.

 

New Version is out now!

Fixes:
Inspector issues with ComponentWrapper
Inspector issues with Visual Debugger
Updated Templates
Issues with the ComponentGroup (Components which were created bevor a specific ComponentGroup will now also be added to that group)

I introduce the ECSController class thats the base class for the main controller there you add and execute your systems

Code (CSharp):

1. using UnityEngine;
2.  
3. namespace ECS {
4.  
5.     // Use this class to control the ECS System
6.     public class GameController : ECSController<UnityStandardSystemRoot, EntityManager> {
7.  
8.         // Use this for initialization
9.         protected override void Initialize() {
10.             AddSystem<MySystem>();
11.  
12.             // or you can use this for more controll over compositions of systems
13.             //AddSystem(new MySystem());
14.         }
15.  
16.         // Override this function if you want to controll what Scene Entity should be load in this context
17.         // The base implementation will add all Scene Entities to the context
18.         //protected override void AddSceneEntitiesToSystem() {
19.    
20.         //}
21.     }
22. }

It creates the system and the entity manager by dependency injection for you
With the AddSystem methods you can assign your system to the SystemRoot
The ECSController also executes the Start, Update, and FixedUpdate Routines of the Systems

Thats it for today!
Have fun with it





*************
EDIT
*************
Ah sorry I found one more bug in the visual update system, so that not all the systems wouldn't called.
It's now fixed


*************
EDIT
*************

I also forgot to say that I've added two Basic Components for Transform and Animator
look at ECSTransform and ECSAnimator

 

I am getting there errors:

Assets/Example/GameObjectExample/ECSTransformComponent.cs(11,18): error CS0101: The namespace `ECS' already contains a definition for `ECSTransform'

Assets/Example/GameObjectExample/ECSTransformComponent.cs(21,18): error CS0101: The namespace `ECS' already contains a definition for `ECSTransformComponent'

 

Fixed I forgot to update the example...

I've also added some text to the example scenes

 

Its ok now.

Some thoughts:

If you want to compete against Entitas and Svelto I would advice to write small and readable examples where the user can get the grip of it.

Svelto made the survival demo but it was really hard to get what is going on.

 

I have to say, when profiling this is much much slower than entitas and svelto.

I like your code style, but the performance loss is too much to compare it to the others right now

 

Can you go in detail please?

 

No need , I accidentally opened 3 instances of Unity.

 

Oh ok

 

Wow, dude, that looks promising...
I've been working on a lightweight ECS as well, but it's like half done. Glad you made this and share it to the public.

And when I was implementing ECS in C#, one thing bothers me is that: using array over List, I mean, using List[index] is an indexer, which is a method call, which cause cpu to jmp to another address, that kinda break the DOD's purpose.

 

Oh, man, could you put it on the github? or assetstore maybe? That would be easier to follow.

 

Thank you!
Well, I use an generic array. It's still in a class but that should be ok. See ComponentArray and ComponentArray<TComponent>. It's in the ComponentGroup.cs sorry I have to clean it up...

It resize its memory usage by the factor of 2. It only grow at the moment. So I have to change. It's also not power of 2. I beleve I use a start capacity of 10. I also have to change this.

I'll do soon

 

Git goes live! https://github.com/Spy-Shifty/BrokenBricksECS
I do not have much time for beauty, sorry for that.

I'll expand it later on

 

Hi all,

I've commited an update to the git repos! Check it out on https://github.com/Spy-Shifty/BrokenBricksECS

The changelist:

Bugfixes and Preview of Entity Manager Debugger

Bug fixes:
- issues with the hash of GroupMatcher (hashes of that struct were not unique)
- EntityManager events will now be called at ones, after OnStart, OnUpdate and OnFixedUpdate.
- Some bugs with the Systemdebugger

Added feature:
- EntityManagerDebugger (preview): shows all entities of that manager with its component. You can also filter by components. The visual debugger will create a gameobject for each EntityManager with a DebugEntityManagerBehaviour on it.



Note: Replace EntityManager with UnityEntityManager in all your scripts. The introduction of this is necessary because of the debugger.
If you use the EntityManager with [InjectDependency], it will create a new Manager of that base type and not of the used UnityEntityManager! So check you scripts and change it!

 

I did a benchmark test and it seems to consume an enormous amount of memory when spawning 10.000 cubes.

It takes 6.7 GB of ram and takes quite a while to do so.

The Unity way consumes only 25 mb.

 

Can you go further in detail pls? What components do you use and so one

Maybe there is a data lag

 

I just raised the number in the demo from 3 to 10000 prefabs and compared to a unity only demo.

 

Hi, well I solved the problem...

It wasn't one! Try the following: Close the profiler window.
The profiler it self creates this amount of memory and not the ecs system!

10.000 cubes spawned takes only 1.695 Mb on my machine (UnityEditor in the Taskmanager)

 

Ok did that and it didn't reached 6.7 GB.

But it took a significant amount of time to finish the spawns. (i7 6700k @ 4.2 ghz)

And the memory went from 327MB to 530MB.

 

Man, great job on that system! I'm using in a project that i'm working with right now.

The system itself works great, but I am having some troubles with the unity editor part.

When I click on install I get a error saying that the 91-ECS__Wrapped Component Class-NewComponent.cs is denied.

And every time I enable or disable the visual debugging I get the following error:
PlayerSettings Validation: Requested build target group (16) doesn't exist; #define symbols for scripting won't be added.
UnityEditor.PlayerSettings:SetScriptingDefineSymbolsForGroup(BuildTargetGroup, String)
ECSSetup: DisableVisualDebugging() (at Assets/ECS/Unity/Editor/ECSMenus.cs:85)


Other than that the system is just beautiful. I let you know if I stumble across some bugs.

 

Thank you!

Well you need access privileges to the unity install folder. Try to start unity as admin

or

Go to: <Unity Install dir>\Editor\Data\Resources\ScriptTemplates
and copy the files from
<ECS folder>\Unity\Editor
manually

Don't worry about the exceptions
The script just go'es through all possible buildings in the playersettings and tries to add or remove a Scripting Defined Symbol.

It works even with this exceptions. Just ignore that

Again thank you. Let me know if you have some!

 

I'll investigate some research on that.

 

Thanks man! Worked like a charm.

 

I just profiled your ECS beginner example and I'm getting 44ms on this part (desktop i7 3.5ghz):

Code (CSharp):

1.  Stopwatch sw = new Stopwatch();
2.             sw.Start();
3.  
4.             if (CompoentChanged != null) {
5.                 for (int i = 0; i < _compoentChangedList.Count; i++) {
6.                     CompoentChanged.Invoke(_compoentChangedList[i].Key, _compoentChangedList[i].Value);
7.                 }
8.             }
9.  
10.             sw.Stop();
11.             UnityEngine.Debug.Log("Time: " + sw.Elapsed.TotalMilliseconds);

Deep profiling also shows this part. The invoke is quite costly and has 0.54ms self.
I also don't get some of the ECS use-cases. In UpdateFloatSystem you set the data. From an architectural design this is nice, but setting the data, that is just deferred to the CompoentChanged loop, which is not faster, is not.

I also tried an init with 1.000.000 objects and Unity froze.

Still, thank you very much for writing this so close to the Unity ECS, switching will be much easier.

I'm currently using SveltoECS and my test program moves 1 mill objects in 7-8ms as a reference, 12ms without SIMD optimization. (26ms in-editor)

Edit: There's also:

Code (CSharp):

1. floats.Update(floats.GetEntity(i), new FloatComponent(floats[i].value + 1));

which is fast as expected. Why not use this? What's the difference in your ECS to this? Just for triggering subscribed observers?

Code (CSharp):

1. entityManager.SetComponent(floats.GetEntity(i), new FloatComponent(floats[i].value + 1));

Edit2: Ok, got it. What's so slow is piping the data back to Unity. Maybe it'll be usable when the job system hits but right now, this is the fastest way to break performance so maybe you revisit the example to not get the wrong coding-basics. Use SetComponents only if you need your data back in a unity behaviour. If you only need the value updated in the system it's safe to just use update on the ComponentArray. Is this correct?

Edit3: I think you need to rewrite the public event Action<Entity, Type> CompoentChanged; approach of subscribing every entity to it and then invoking every entity when changes occur. It's a O(logn) right now with even multiple hits (unsure why) in the same entity. (GameObjectEntity.EntityManager_ComponentChanged) that only get ignored because of
if (!_entity.Equals(entity)) { return; }

 

Thank you for your detailed report.
I want you to let you know that I'm working on this issue.

Approach:
No global event fireing. If you change a component on an entity than only the subscriptor on this entity with the specific componenttype will be called. And it'll be called without reflection.
This should be super fast I beleave. But it's still in development!


EDIT

don't call

Code (CSharp):

1. floats.SetComponent(floats.GetEntity(i), new FloatComponent(floats[i].value + 1));

this only updates the component on that componentArray

If you want to update a component to all ComponentArrays of type "FloatComponent" (global), than use:

Code (CSharp):

1. entityManager.SetComponent(floats.GetEntity(i), new FloatComponent(floats[i].value + 1));

 

Very cool, good to hear!

I got some more findings yesterday that I want to report. I'm nearly on the same speed of SveltoECS, some decisions in design are preventing it of fully reaching it.

So first of, when working with struct components, data access is much faster if you can use something like

Code (CSharp):

1. _sleepComponents._components[i].sleep

_sleepComponents is of type ComponentArray

You wrote a this[int index] getter for the ComponentArray and with structs, the return has the side effect to return a new instance.
I haven't found a good place for it, but I put this line of code in the beginning of OnFixedUpdate to have a local struct array which I can iterate upon.

Code (CSharp):

1. _sleepArray = _sleepComponents._components;

Of course this shouldn't be called in every OnFixedUpdate. I was unsure what this really does, but from the allocation and performance it gets the pointer to the internal struct array. Still, there are better places and this leads me to the next point.

Caching. What are your plans/thoughts?
The cpu has to work on local struct arrays to get nice scaleable code that we can call "high performance code".
So there's a huge difference of data handling when it's just local system data or system-wide data.

With local system data you could update the locally cached struct array only if entities are removed/added. System-wide data doesn't really need (much) change from your implementation.
So maybe an additional InjectTuple parameter (Local,System-Wide) to reflect this behaviour or an InjectTupleLocalStruct? Just throwing ideas around.

Maybe I'm misunderstanding the problem and I'm supposed to use method calls over directly setting data. It's just, why pay the overhead for the call at all, right?

Setting data in ECS weirds my out in general because of this generic behaviour of system-wide data.
If I have the local struct array I can perfectly use:

Code (CSharp):

1. _sleepArray[i].sleep = true;

but most ECS design I've seen, including yours, prevents this. Not sure why, maybe you can elaborate?

Something that puzzled me. I've 2 entites. One with Comp A one with Comp B and both have a sleep Component. When working on 2 different systems I get 1 CompA, in the other system Comp B but 2 sleep components even though they should be split. Iterating upon this is senseless and I need a costly dictionary lookup to get the index from the entity id.

Another thing I found was that a system OnEntityAdded(entity) is called for every component but I don't know from which component the call came so I can't really init the entity once.

Can you point my to the part where the data prefetching is handled? Is it a side-effect?

That got longer than expected.
I have hopes for this project and I strongly believe that any good ECS can stand side-by-side even if Unity releases its ECS. There's nothing holding you back to use the same fast-path routines they are using in their ECS and design-wise, Unity will never be as agile as a small team.

 

First of all thank you for this great discussion and your inspiration! This will really improve the ecs system! Please go on!

Next to say: my new event system is almost done! I'll do some more tests on it...
Stay tuned

You're right that accessing the inner array would be faster!
I tested it on a float struct with 100,000,000 entries: (i7 cpu)
through this[int index]: 20151736 ticks (2015 ms)
direct access: 8577911 ticks (857ms)

The code:

Code (CSharp):

1. public class Test : MonoBehaviour {
2.     public int length = 100000000;
3.     // Use this for initialization
4.     void Start () {
5.         float[] values = new float[length];    
6.         for (int i = 0; i < length; i++) {
7.             values[i] = Random.value;
8.         }
9.  
10.         MyArray array = new MyArray(values);
11.         Stopwatch stopwatch = new Stopwatch();
12.         stopwatch.Start();
13.         for (int i = 0; i < length; i++) {
14.             var value = array[i];
15.         }
16.         stopwatch.Stop();
17.         Debug.Log(stopwatch.ElapsedTicks + " " + stopwatch.ElapsedMilliseconds);
18.  
19.         stopwatch.Reset();
20.         stopwatch.Start();
21.         for (int i = 0; i < length; i++) {
22.             var value = array.array[i];
23.         }
24.         stopwatch.Stop();
25.         Debug.Log(stopwatch.ElapsedTicks + " " + stopwatch.ElapsedMilliseconds);
26.  
27.     }
28. }
29.  
30. class MyArray {
31.     public MyStruct[] array;
32.     public MyStruct this[int index] { get { return array[index]; } }
33.  
34.     public MyArray(params float[] values) {
35.         array = new MyStruct[values.Length];
36.         for (int i = 0; i < values.Length; i++) {
37.             array[i] = new MyStruct(values[i]);
38.         }
39.     }
40. }
41. struct MyStruct {
42.     public readonly float Value;
43.  
44.     public MyStruct(float value) {
45.         Value = value;
46.     }
47. }

But you can't access the inner array directly!!! And please don't do this!

the inner array isn't the real size of subscripted entities.
It's larger! If it would be the same size we had to resize the array everytime an component would be added or removed and this would lead to garbage collection (arrays are ref types)

And to say: "Ok use the inner array but take the length of the class that holds the array" isn't error proofed to me!

Well in c# 7 we could do something like:

Code (CSharp):

1.  
2. public ref MyStruct this[int index] { get { return array[index]; } }
3.  

Well an alternative could be:

Code (CSharp):

1.  
2.     public void Get(int index, out MyStruct value) {
3.         value = array[index];
4.     }
5.  

but this takes "only" 17516585 ticks (1751 ms)
but this for few ms? I don't like that...

I'll answer you to the rest of your post later... I have to think about it first!

EDIT

I also plan to improve the inner system calls with ref Entity and red Component
this would lead to less copies

 

I don't really know what do you mean with that...
But I think there is an missunderstanding and you effectively mean the dataprefetching effect it self.
But let me answer your next question first!

Data prefetching isn't a command that you call "Please do data prefetching now"!
It's the way you implement something, so that you benefit from data prefatching.
Data prefatching is a native hardware thing. The hardware preloads the next address from RAM into the CPU cache depending on the current address. And this will happens without a instruction of the cpu. So the cpu doesn't have to wait for the next data (because it's already loaded.)

We support this prefetching by using the array structur an process one by another.
So in the system we load the current componentarray class from somewhere.
Because of the use of struct arrays the data in the componentarray class is tightly packed one after another, so that the hardware can prefatch the right data into the cpu cache.

And I beleave you mean with "local data" that you use the stack instead of the heap to store data. You can't do this because arrays are always in the heap in c#. And also we already have this "high performance code" because of ComponentArrays and the use of struct arrays in that.

I hope this is understandable.. .Take a look at the image of my second post.

This is answered in my last post (...resize a array will lead to gc...)

You can't assign a member of a struct in a array in C#! This isn't possible

Code (CSharp):

1. _sleepArray[i].sleep = true;

You only can set a new instance of that struct to the array

Code (CSharp):

1. _sleepArray[i]= new SleepStruct(true);

EDIT
If you use classes instead of structs, it's ok and you can do this!
But with structs you have to go through the EntityManager to assign the new value to all ComponentArrays

Can you go further into detail, to give you a better answer.

In the new event system you'll have 2 options!
Subscripe to OnEntityAdded event of the EntityManager. This will fired each time an entity will created

Or
Subscripe to OnComponentAddedToEntity event of the EntityManager, This will fired each time an Component will add to a specific entity. But you subscripe only for a specific entity.

I'll descripe it more later

Well and my answer is even longer

Thank you! And with the help of you and the community, we'll create a really good tool!


Please, also look at my last post!

 

I updated the source! Check the git repos.
https://github.com/Spy-Shifty/BrokenBricksECS


Whats new:
Events called only for specific subscribers
I decided to use the observer pattern for this implementation because delegates and events produce garbage. Well the current event system also produce some, but we can improve it. For that some more tests are required.

If you wanne listen to component changed events for a specific component on a specific entity use:

Code (CSharp):

1. class MyClass : IComponentChangedEventListener {  
2.  
3.     entityManager.SubscripeComponentChanged(entity, this);
4.  
5.     void OnComponentChanged(Entity entity, TComponent component) {
6.  
7.     }
8. }

If you wanne listen to component added to a specific entity use:

Code (CSharp):

1. class MyClass : IComponentAddedToEntityEventListener{  
2.  
3.     entityManager.SubscripeOnComponentAddedToEntity(entity, this);
4.  
5.     void OnComponentAddedToEntity(Entity entity, Type componentType) {
6.  
7.     }
8. }

Where are some more, check out the ComponentGroup, ComponentArray and the EntityManager classes for more information.


Have Fun with it

 

A practical issue with this is that to take advantage of the performance improvements and multiple core support you have to use Unity's ECS. It's their specific data structures and api's that allow the batching to happen at lower levels.

 

@snacktime The Unity ECS is pure C#, all implementations of the interfaces and classes will be available to us as well as the calls to the unity engine that are needed to get batched calls. In the talk I already saw a new low level batched process for RayCasts. All these calls will be exposed to us. The only time this will get complicated is when Unity builds in the ECS into their C++ code but lets talk about this when we're there. I guess we're surely 2-3 years away from this.

@Spy-Shifty Regarding the struct set issue, I wrote this and profiled in VS. It's Console C# application and not Unity but works the same.


Setting a new struct of any kind is costly because you have to go through the struct constructor and the setter(s), creating new stack values that finally end in an assembly MOV on the struct address that was already pre-allocated.

And for some things it's better to look at the assembly code. Even if it's mumbo jumbo. More arithmetics = bad is the gist of it.
Here is a pastebin of it: https://pastebin.com/0jyJh7Ei

Looking at it, setting a new struct requires a lot more operations. From another program I can even generalize the statement, that calling a struct constructor with parameters is more expensive.


> And I beleave you mean with "local data" that you use the stack instead of the heap to store data.

No, with local data I mean blocks of memory that belong to a certain system.
System wide data are blocks of memory where more than one system operates on, so systems have to be updated when changes occur.

Systems, unless multi-threaded, should work on the same block of data when they are in a group. There's hardly a case where you want to operate on maybe 3 different blocks of the same memory in 3 systems just for a speed up. I don't think there even would be any, really.

I rewatched the talk from Joachim Ante the 5-th or so time now and the ECS seems to handle classes with ComponentArray and structs with ComponentDataArray. And in his talk you can also see that they are directly setting values on public struct variables in an array.

I'll take a look at the new version in the following days! Thanks for the update!

 

Ah, ok I got it! Well yes this is a nice idea, but this could lead to "side effects" if you don't know what you're doing. Even if you know it.

Scenario: You deside that component XY is a system local one, because its only used by this system. Now you want to add new features a few days later (or maybe a different developer) and you're using your component XY again in another system, but you've forgotten that this component is treated locally... And now you have an unexpected behaviour...

You hear what I'm getting at?

I beleave performance is a really important thing, but it have to be save as well!

Can you show me that point in the video?I can't find it.

I only see that he is updating a class component. Well through a property you can't update the struct array this way:

Code (CSharp):

1.             var test = new TestArray();
2.             test[1].i = 5; // Compiler Error->the return value isn't a variable and can't be updated
3.  
4.         class TestArray {
5.             Test[] test = new Test[100];
6.  
7.             public Test this[int index] { get { return test[index]; } set { test[index] = value; } }
8.         }
9.         struct Test {
10.             public int i;
11.         }
12.    

and as I mentioned bevor, you can't access the real array inside the class (well you can but it's not save)

 

> I beleave performance is a really important thing, but it have to be save as well!

Well, Unity has a nice philosophy about it. We give you the freedom to make mistakes but at least we warn you about it. Frameworks for high performance code shouldn't have checks in release mode, it defeats the purpose.

> Can you show me that point in the video?I can't find it.
Sure:

> I only see that he is updating a class component. Well through a property you can't update the struct array this way:
Hm, good question on their implementation.I'll think about it.

edit: Ok, in the timestamp he's setting a class, so this is why it works.
At 58:28 he gets the position struct form the ComponentDataArray, modifies the internal position vector and sets the struct back.
I'm kind of dissapointed now that this doesn't work. This was always a problem in C# for me, coming from C(++) and it still seems like a brick wall. Setting data with generating more data is just weird and forces you to split components in single values which prevents proper read-access and cache linearity, when operating on multiple components. Maybe some unsafe magic can help.

Something to think about over the weekend.

 

Hi I've added one more feature!

It's now possible to inject different entity groups per system.
How:
You can group ComponentArrays by adding a group name or id to the InjectTuble attribute

E.g.: You want to interact between 2 entities with differend component sets
The first entitie group has MyComponent1 and MyComponent2 assigned to it.
The second group has MyComponent1 and MyComponent3 assigned to it.

Code (CSharp):

1.  
2. [InjectTuble("Group1")]
3. ComponentArray<MyComponent1> myComponents1;
4.  
5. [InjectTuble("Group1")]
6. ComponentArray<MyComponent2> myComponents2;
7.  
8.  
9. [InjectTuble("Group2")]
10. ComponentArray<MyComponent1> myComponents1;
11.  
12. [InjectTuble("Group2")]
13. ComponentArray<MyComponent3> myComponents3;
14.  

Remember groups have different array sizes!

I also changed the template naming for ComponentWrapper a little!
If you'll create a new component from the context menu. The Component it self will be named #ComponentName#Component
And the wrapper will be named #ComponentName#
You don't have to change the filename for ComponentWrappers anymore

I also renamed the ECSTransform and ECSAnimator component to
TransformComponent and AnimatorComponent

Kind regards,
Spy-Shifty

 

Hello - Thanks for this nice ECS framework !
I was using Entitas previously, but like the structure and speed of using structs. Nice work.

I may have missed this, but is there a way to subscribe to changes at a group-level, or even component-level? I saw there is an option to subscribe to changes for a specific Entity. Entitas had a concept for this for groups that made processing all the collected changes rather easy. I could subscribe to individual entities, I just wasn't sure if I missed something.

thanks again

 

Hi and thank you!

you can subscribe to an event of the group that fires if an entity was added or removed from the group.

Code (CSharp):

1. myGroup.SubscripeOnEntityAdded(new EntityAddedEventListener((entity)=>{...}));
2.  
3. myGroup.SubscripeOnEntityRemoved(new EntityRemovedEventListener((entity)=>{...}));
4.  
5.  

in systems you can use

Code (CSharp):

1. class MySystem : ComponentSystem {
2.  
3.     public override void OnEntityAdded(Entity entity) {
4.  
5.     }
6.     public override void OnEntityRemoved(Entity entity) {
7.  
8.     }
9. }

There are also events for a specific ComponentArray

Code (CSharp):

1.  
2. myComponentArray.SubscripOnEntityAdded(new ComponentAddedToEntityEventListener((entity, component)=>{...}))
3.  
4.  
5. myComponentArray.SubscripOnEntityRemoved(new ComponentRemovedToEntityEventListener((entity, component)=>{...}))
6.  
7.  
8. //additional for structured componets only:
9. myComponentArray.SubscripOnComponentChanged(new ComponentChangedToEntityEventListener((entity, component)=>{...}))
10.  

for more information take a look at:

BaseEventListener.cs
ComponentSystem.cs
GroupComponent.cs
ComponentArray.cs

 

In my current version of the ecs I've solved the instantiation of prefabs slightly different.
I dont need the Initialization method anymore.
So that Monocomponents are direct referenced to the ecs component now.
This is more intuitive to the Unity instantiation way.

the change on the component layer looks like this:

Code (CSharp):

1. //OLD
2. [Serializable]
3.     public class TransformComponent : IComponent, ICloneable {
4.         public Transform transform;
5.  
6.         public object Clone() {
7.             return MemberwiseClone();
8.         }
9.     }
10.  
11.     // this wrapps the component tfor Scene & Prefab workflow
12.     [HideInInspector]
13.     public class ECSTransform : ComponentWrapper<TransformComponent> {
14.         public override void Initialize() {
15.             TypedComponent.transform = gameObject.transform;
16.         }
17.     }
18.  
19.  
20.  
21. //New
22. [Serializable]
23.     public class TransformComponent : IComponent, ICloneable {
24.         public Transform transform;
25.  
26.         public object Clone() {
27.             return MemberwiseClone();
28.         }
29.     }
30.  
31.     // this wrapps the component tfor Scene & Prefab workflow
32.     [HideInInspector]
33.     public class ECSTransform : ComponentWrapper<TransformComponent> {
34.         private void Awake() {
35.             TypedComponent.transform = gameObject.transform;
36.         }
37.     }

you can even assign monobehaviours directly to the component through inspector:

Code (CSharp):

1.  
2. [Serializable]
3.     public class AudioSourceComponent : IComponent, ICloneable {
4.         public AudioSource audioSource;
5.  
6.         public object Clone() {
7.             return MemberwiseClone();
8.         }
9.     }
10.  
11.     // this wrapps the component tfor Scene & Prefab workflow
12.     public class ECSAudioSource: ComponentWrapper<AudioSourceComponent> {
13.     }

I'll upload this soon. I have to do some test first

 

Hi y'all

I've update for you! Check out the git repos

Changes:
- ComponentWrapper won't longer need Instantiate() method. References will now copied natively!
- Enabling, Disabling of ComponentWrapper adds or removes the ecs component from the entity
- HasComponent method was added to the EntityManager to check if an entity has an component
- Update of the examples
- Update of ecs template files -> please run the setup (Unity->BrokenBricks->ECS->Setup) again and restart Unity
- Some minior fixes

 

Hi @Spy-Shifty

I was looking at ECS and came across your thread and Entitas. I am new to ECS and was wondering where to start. Could you let me know the key differences between your system and Entitas and which system in your view is better suited for whom?

Thx!

 

Hi,

I have some aspects for you, but without completion:

Entitas uses fixed indices for components. Components are stored in an array and each has a fixed postion there.
E.g. you have a position component. Each entitiy in the same context has an array for all components. The position component has the index e.g. 5 on that array. That makes accessing really fast O(1) because I know exactly there the component is stored in the array (5 in this example).

The downside of this system is:

• each entity generates n*12 Byte where n is the number of components in that context
• they provide a code generator so that the component index, accessor and getter and other helper will be generated for you. Because of the code generator: each time you change something on the components you have to execute the code generator. And if you arn't carefully, this will break the hole project and you have to investigate some time to clean it up...

On the other side:

• you have a really fast access to components
• you have component filter for systems (e.g. give me only that health components that has a health under 25%,...)
• you have on demand systems (reactive systems that only handles components that have changed)

My Unity-ECS interpretation:

• No code generator at all. I use filterd groups of components. So each entity in each componentarray in that group has the same index.
• Performance boost through Dataprefatching (Short: Take advantage of loading the next bytes of RAM into the CPU cache)
• We have a better Unity integration. Supports Unity Prefabs, Monobehaviour integration and so on. It feels more a part of unity as entitas.
• But it's still not perfect and in an early stage of development!

Both approaches can be used without unity. (server client integration)
The code Generator thing was the main reason to me to not use Entitas anymore.

So my conclusion:
If you want to use a more Unity like framework, use mine. If you don't care about some changes in the future...
If you want a more stable and approved system, take entitas. But notice the code generator and the not so good integration of Unity.

It's up to you what you prefer.

 

Cool - thx - appreciate you taking the time to write this up. I think I will play around a bit with your version first. By the way, I noted that the latest version (downloaded yesterday) has an issue on the GameObject Example. The Cube prefab has 2 missing script errors (RotationSpeedComponent & ECS Transform) --- or something went wrong when I imported this.

 

Do you have some compiler errors? Just put both the ECS and the example folder into your assets folder.

this should work.
If not tell me pls.

 

I fixed it, but I had 2 script references missing on the "Cube" prefab. Initially I removed them, but that created a runtime null ref exception. I figured the missing scripts were RotationSpeedComponent and ECS Transform. Once I updated the prefab with those, all was working fine.