Official API usability

I'm reading through this: https://docs.unity3d.com/Packages/com.unity.entities@0.5/manual/entity_command_buffer.html

and this line stands out:

EndSimulationEntityCommandBufferSystem m_EndSimulationEcbSystem;

it's a bit of a mouthful. Would it be terrible to change

EndSimulationEntityCommandBufferSystem

to

EndSimECBSys

or something similarly brief? It's just a quality of life thing when it comes to reading through the code.

 

From the C# Naming convention (https://docs.microsoft.com/en-us/dotnet/standard/design-guidelines/general-naming-conventions):

Using Abbreviations and Acronyms
❌ DO NOT use abbreviations or contractions as part of identifier names.

For example, use GetWindow rather than GetWin.

 

Oh... I guess it would be terrible.

 

Did i misunderstand you? My english is not the best.

 

Abbreviations are quite inconsistent.

If you know what you are looking for you, may end up typing more than what was written.
If you don't know what you are looking for, you may not understand what the abbreviation means.

If you really want to abbreviate it, you can with

using EndSymECBSys = Unity.Entities.EndSimulationEntityCommandBufferSystem;

, but I do not really suggest if you are not working alone.

 

Also https://docs.microsoft.com/en-us/do...lines/names-of-classes-structs-and-interfaces:

I kind of don't like it as it makes similar types misaligned and tougher to find. Imo doing opposite, using namespaces or nested types is better (Exceptions.ArgumentOutOfRange). For ex. https://docs.unity3d.com/Packages/com.unity.entities@0.6/manual/system_update_order.html

Compared to:

Obviously in that case "Systems" could be removed completely in that post.

 

In some cases are better the base class in the name increases readability. As example Exceptions.ArgumentOutOfRange, you can have an Logger for ArgumentOutOfRange with the same name, and is more logical throw an out of range exception.
Sometime you have component like "Rotate" and an "RotateSystem", if you strip the System, the system class has the same name as the component struct.

 

EntityCommandBuffer API suggestion:
1. Can It has a lambda overload with EntityManager as input? So you can use the command buffer more customized?
2. Or maybe a NativeContainer version of SetComponent, like SetBuffer, so these data can be supply by a later job. (Just overload for single entity version)

The main reason is that at point a job being scheduled or running in a worker thread. The data you might want to set with SetComponent might not be ready yet. But they are certainly ready when Command buffer runs (with correct JobHandle of cause). Getting data at command execution time is very helpful.
So a pre-allocated NativeContainer that hold these data you collected in one command buffer and used in a later one, will do the job.
Lambda command with EntityManager will be even more convenient.

 

As I write my own IJobChunk. I found this problem.
https://forum.unity.com/threads/ijobchunk-and-componentdatafromentity-conflict.835810/

ArchetypeChunkComponentType<T> and ComponentDataFromEntity<T> are very likely to be used together.
And according to "alias" exception, they are more or less the same thing (NativeArray).
But each server it's own use case better.
ArchetypeChunkComponentType<T> for data in chunk
ComponentDataFromEntity<T> for all entity.
The alias conflict forces me to choose one over the other. I believe it can be adjusted to fit in the need of both cases.
Read from ComponentData outside selected chunk by entity id, as well as read/write to the data inside the chunk by gathering NativeArray.
Maybe just add a ComponentDataFromEntity<T> overload on chunk's GetNativeArray function.
[NativeDisableParallelForRestriction] is still needed as users should be aware of job
Schedule.

 

I found another problem with EntityCommandBuffer.
The Entity return by CreateEntity can only be used by EntityCommandBuffer itself. as it uses a temporary negative index to keep track of Entity that is not created yet. This is miss leading. I tried using these entities in other ComponentData and end up with exceptions. And it take me some time to figure out why.
EntityCommandBuffer.CreateEntity return some alternative struct or even just an int index. to notify users that it is not an Entity at all.
Or maybe a NativeContainer that holds a reference to an entity that will be filled by EntityCommandSystem,as I posted previously. Lambda overload is also an option.

 

For this problem, I made a System called DeferedEntitySystem.
It holds a properly indexed array. That will be filled with the correct Entity after EntityCommandBuffer created those Entities. These data exist for only one frame.

Code (CSharp):

1.  
2. public DeferedEntityCreater GetSimBeginCreater()
3. public DeferedEntityCreater GetSimEndCreater()
4. public DeferedEntityAccessor GetAccessor()
5. public DeferedEntityAccessor.Parallel GetParallelAccessor()
6.  

These are the main interface mathod of

DeferedEntitySystem

.
Creaters: Main API function

public DeferedEntity CreateDefereddEntity(in EntityArchetype archetype = default)

• returns a DeferedEntity that contains
• An Entity return by internal EntityCommandBuffer that should be used with Creater's EntityCommandBuffer only.
• A DeferedEntityID that holds an integer index that can be used with Accessor to get created Entity IN THE NEXT FRAME, when EntityCommandBuffer finishes its job.

Accessor: Main API function

public Entity GetDeferedEntity(in DeferedEntityID idx)

• Returns Entity linked with the creator given DeferedEntityID

Accessing Example

Code (CSharp):

1.     public class MyAccessSystem : JobComponentSystem
2.     {
3.         public struct MyDeferedTarget : IComponentData
4.         {
5.             public DeferedEntityID did;
6.         }
7.  
8.         public struct MyTarget : IComponentData
9.         {
10.             public Entity eid;
11.         }
12.         internal DeferedEntitySystem deferedEntities;
13.         internal GetDeferedEntityJob mJob;
14.         internal EntityQuery myQuary;
15.         protected override void OnCreate()
16.         {
17.             deferedEntities = World.GetOrCreateSystem<DeferedEntitySystem>();
18.             myQuary = GetEntityQuery(
19.                 ComponentType.ReadOnly<MyDeferedTarget>(),
20.                 ComponentType.ReadWrite<MyTarget>());
21.             mJob = new GetDeferedEntityJob();
22.             //...
23.         }
24.         protected override JobHandle OnUpdate(JobHandle inputDeps)
25.         {
26.             var accessor = deferedEntities.GetAccessor();
27.             //Main Thread usecase
28.             DeferedEntityID deID = default;
29.             //get your id somehow...
30.             Entity eid = accessor.GetDeferedEntity(deID);
31.  
32.             //Parallel Job usecase
33.             mJob.accessor = accessor.ToParallel();
34.             mJob.dpTp = GetArchetypeChunkComponentType<MyDeferedTarget>();
35.             mJob.pTp = GetArchetypeChunkComponentType<MyTarget>();
36.             inputDeps = mJob.Schedule(myQuary, inputDeps);
37.  
38.             //remove defered component as defered id is only valid for one frame!
39.             var esecb = World.GetOrCreateSystem<EndSimulationEntityCommandBufferSystem>();
40.             esecb.AddJobHandleForProducer(inputDeps);
41.             esecb.CreateCommandBuffer().RemoveComponent<MyDeferedTarget>(myQuary);
42.  
43.             return inputDeps;
44.         }
45.  
46.         public struct GetDeferedEntityJob : IJobChunk
47.         {
48.             [ReadOnly] internal ArchetypeChunkComponentType<MyDeferedTarget> dpTp;
49.             internal ArchetypeChunkComponentType<MyTarget> pTp;
50.             internal DeferedEntityAccessor.Parallel accessor;
51.             public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
52.             {
53.                 var chunkDp = chunk.GetNativeArray(dpTp);
54.                 var chunkP = chunk.GetNativeArray(pTp);
55.                 for (int i = 0, len = chunk.Count; i < len; i++)
56.                 {
57.                     var mComp = chunkDp[i];
58.                     //read entity id
59.                     chunkP[i] = new MyTarget() { eid = accessor.GetDeferedEntity(mComp.did) };
60.                 }
61.             }
62.         }
63.     }
64.  

Creater Example:

Code (CSharp):

1.     public class MyCreateSystem : JobComponentSystem
2.     {
3.         internal DeferedEntitySystem deferedEntities;
4.         internal MyCreateJob mJob;
5.         internal EntityQuery mQuary;
6.         public struct ShouldTargetDeferedEntityTag : IComponentData { }
7.         public struct TargetData : IComponentData { int targetHP; }
8.  
9.         protected override void OnCreate()
10.         {
11.             deferedEntities = World.GetOrCreateSystem<DeferedEntitySystem>();
12.             mJob = new MyCreateJob();
13.             mQuary = GetEntityQuery(ComponentType.ReadOnly<ShouldTargetDeferedEntityTag>());
14.             //...
15.         }
16.         protected override JobHandle OnUpdate(JobHandle inputDeps)
17.         {
18.             var creater = deferedEntities.GetSimBeginCreater();
19.             mJob.creater = creater.ToParallel();
20.             inputDeps = mJob.Schedule(mQuary, inputDeps);
21.  
22.             //Schedule target ECBS to run after mJob
23.             creater.entityCommandBufferSystem.AddJobHandleForProducer(inputDeps);
24.             creater.entityCommandBuffer.RemoveComponent<ShouldTargetDeferedEntityTag>(mQuary);
25.             return inputDeps;
26.         }
27.         public struct MyCreateJob : IJobChunk
28.         {
29.             [ReadOnly] internal ArchetypeChunkEntityType eTp;
30.             internal DeferedEntityBuffer.Parallel creater;
31.             public void Execute(ArchetypeChunk chunk, int chunkIndex, int firstEntityIndex)
32.             {
33.                 var chunkEntities = chunk.GetNativeArray(eTp);
34.                 for (int i = 0, len = chunk.Count; i < len; i++)
35.                 {
36.                     var eid = chunkEntities[i];
37.                     var de = creater.CreateDeferedEntity(chunkIndex);
38.                     var dp = new MyAccessSystem.MyDeferedTarget() { did = de.DeferedID };
39.                     var p = new MyAccessSystem.MyTarget() { eid = Entity.Null };
40.                     creater.entityCommandBufferConcurrent.AddComponent(chunkIndex, eid, dp);
41.                     creater.entityCommandBufferConcurrent.AddComponent(chunkIndex, eid, p);
42.                 }
43.             }
44.         }
45.     }
46.  

 

ECB automatically remaps invalid entities to valid entities within the command buffer.

Code (CSharp):

1. // This totally works
2. var newEntity = this.CommandBuffer.CreateEntity(entityInQueryIndex);
3. this.CommandBuffer.SetComponent(entityInQueryIndex, entity, new Component {Entity = newEntity});
4.  
5. // Also for buffers
6. var buffer = this.CommandBuffer.AddBuffer<ComponentBuffer>(entityInQueryIndex, entity);
7.  
8. for(var i = 0; i < 10; i++) {
9.     var newEntity = this.CommandBuffer.CreateEntity(entityInQueryIndex);
10.     buffer.Add(new ComponentBuffer {Entity = newEntity});
11. }

 

Sure but what if you want to pass Entity created by EntityCommandBuffer to other ComponentData?
for example, you need to create Parent and Child at the same fame via EntityCommandBuffer, and you want them to reference each other by Entity. That will end up in exception.

 

Code (CSharp):

1. // This totally works
2. var newEntity = this.CommandBuffer.CreateEntity(entityInQueryIndex);
3. this.CommandBuffer.SetComponent(entityInQueryIndex, entity, new Component {Entity = newEntity});

This is not working In my case. No no. what version are you using? I'm on 0.5.1-preview.11
Then

new Component {Entity = newEntity}

will stay nagtive as long as you dont touch it.
And how could this be fixed? the Component is passed by value... does DOTS check for any CompoentData pass to "SetComponent" for fields of Entity type? by how? reflection? that will break DOTS. Even if it does check Entity typed fields. changing data is not always what the user wanted. I don't think this pattern works.
CommandBuffer.CreateEntity needs to pass an AgentType instead of an Entity.

 

Sorry, after a quick test. It works.
My case is just a bit complicated. I have a NativeQueue of events. And It's part of A ComponentSystem. And the Entity is tracked in this NativeQueue. That's why Entities created by EntityCommandBuffer keep their negative index.
So I should use DynamicBuffer instead.

 

Thank for the reply, It would be perfrct if it comes two days earlier.... so I would not waste my time on this
DeferedEntity thing...

 

I've seen the term "GameObject scene" used in a few places in the docs. For example, "Find the scene given a guid. For GameObject scenes, this will only return the first matching scene."

What exactly is a "GameObject scene"? Is it the opposite of a subscene?

 

I am bored so here I go. I think DOTS is terrible. It is ugly and overly complicated. Hell, I think if you were to allow us to drop to c++, make implementation of entities there, aligned arenas for them, ways to perform vector intrinsic operations on arrays of entity's variables, multithreaded job system with layers and dependencies and make it all be basic and simple, everything would get way easier and way more nice to read and understand and lots of performance would somewhat remain.

Or create and edit all jobs and DOTS and operations on entities' variables in visual graph editor that would allow everything stated above, which would take away all this boiler plate, burst optimizations would stay, and make DOTS pleasant to work with and fast.

 

Well, it's not done yet. They *are* adding a visual graph editor, and they *are* working on removing the boilerplate. This is, unfortunately, part of the problem with preview releases. They have stated many times over, it's not finished, it will take time, and these improvements will come, you just have to be patient. If you don't like it, you don't have to use it yet, you can wait until it's completed.

 

DOTS is can be considered C++ somehow. If you work with unsafe containers, you will find yourself writing "almost" C++ codes. It's not like the old UnityEngine where all critical codes are hidden in C++ layer. Now you have all the source code, and free to hack around.

And the key to getting boosted by DOTS is not whether to code with C# or C++ or even Graphic editor. But the way to design and think in the ECS way. Here's a pretty blog about ECS https://gametorrahod.com/tag/unity-ecs/ check it out.

And by trying to stay sync with DOTS over versions will help you understand it much better. I believe it is perfectly Okay to work with DOTS for now even for productional projects.
There are problem somethings but, as we have all source access now, we don't have to wait. We can always find a solution by going deep into source code.

So, Good luck with DOTS!

 

And I am blind for missing that. That looks like what I want. Thx.

I know, but that is my point. It is like c++, but with more ugly syntax. It should be the other way around.

Thx, but nah, I am fine . Gonna wait a year or two for that visual scripting.

 

Don't feed trolls plz.

 

Remember the 80/20 rule:

• 80% of your code will be fast enough so you won't need DOTS.
• 20% of your code will be too slow so you will need DOTS (or at least multithreading + Burst).

Only profiling your code will show you where you need to Optimise your game.

 

Netcode is dots only :/

 

Havok is also DOTS only. It's very likely that the number of exclusive features will only increase as time goes on.

 

Good point isn't Unity going to ensure it's DOTS systems integrate well with the engine/IDE and include Monobehaviour style interfaces?

 

No idea but netcode shouldn't rely on dots. For ex. from what I've seen SpatialOS works with both dots and mono https://documentation.improbable.io/gdk-for-unity/docs/monobehaviour-introduction

 

Why does it matter you could be using a Physics or Network solution that relies on a third party DLL and it could be written in a ECS/DOTS style or a more Object and Multithreaded style.

As long as it does it's job and performs well it should not matter on it's Architecture, Design or Code Standards.

All you need is a good API interface and Engine integration to make it easy to use.

 

It's early days but has Unity considered UPMEMs PIM (Process In Memory) DRAM, as PC gaming hardware could just add PIM RAM and potentially get 20x performance boost on DOTS style processing in RAM as opposed to on the CPU.

PIM RAM is just DRAM with processors for each block of memory that can be triggered to run operations on blocks of data. It's the DOTS architecture style but applied to the hardware of RAM.

UPMEMs SDK is C based on LLVM8 so that might be compatible with the Burst IL2CPP pipeline.

I am not aware of any gaming hardware that has adopted PIM DRAM at the moment and it may be that it will be more applicable to servers and workstations first. But even here a Unity game server could take advantage of a DOTS Burst and PIM with massive improvements to performance and possibly more importantly to power usage.

 

This is hardware. Making your players buy additional hardware just to run your game is one of the most reliable ways to achieve abysmal sales.

 

I do recall wanting to play a game that required hardware transform and lighting so bad, when it was introduced, that I went out and bought a new card asap. I miss the days when games were exciting enough to make you do that.

 

Tell that to Sony, Nintendo, Microsoft and Apple.

Also check your current PC, does it have a Floating Point unit built in or a GPU or Sound Chips?

They all used to be extra hardware accelerators above and beyond the basics.

Personally I can remember buying a 3Dfx GPU accelerator because it made supporting games look and play amazingly fast. At the moment you might have or plan to own an Nvidia RTX card for the same reason.

The thing is PIM DRAM could accelerate any bandwidth intense task from video, compression, sorting, searching as well as games, apps and OS. So they could boost more than just your games with compatible software.

And Unity's adoption of DOTS or ECS makes it an ideal accelerator for future Unity games.

In theory all you would need to do is update your RAM to PIM DRAM and ensure your software is compatible.

But it's not widely available yet...

 

You can use transport package without dots, of course you will have to use part of it, like native containers, however it's quite easy to implement it together with monobehaviour.

 

Sure, there are games that make you upgrade your computer but those upgrades tend to be generally useful for other games that come after yours, so that's a sensible thing to ask for. Having to buy some kind of special device for just that one game is a much higher price to ask.

Sony et al. are companies that own hardware platforms. If you release your own console then by all means start selling accessories, but if you're just a software developer then if you plan to make people buy extra hardware just for your game you better make sure that you have the next Guitar Hero on your hands although I have a feeling you need to have something way better in order to justify a $400 price tag for UPMEM.

AGEIA/Nvidia tried to do pretty much this with the old PhysX PPU cards. They couldn't pull it off despite it being useful for a number of games, and Nvidia is a large and powerful company. PhysX has been since completely written off by Nvidia and made open source (NV doesn't like open-sourcing their stuff, see GameWorks that still has some value to them).

 

Often one of the cheapest performance upgrades you can make to a system is boosting memory performance/size then for gaming getting a better GPU. If you need to upgrade your CPU then that often requires a Motherboard/CPU/RAM and for best performance GPU upgrade so nearly a full system overhaul.

In the case of UPMEM a RAM upgrade can not only provide you with more memory performance but also a 20x boost in performance from memory intensive applications*.

Even with a CPU and GPU upgrade you would probably only see a 2-3x performance boost.

So a RAM upgrade to PIM RAM could have a massive boost to your systems performance and save you money on overhauling your entire system to gain less in performance overall.

*for applications that are compiled to take advantage of UPMEM PIM RAM.

Note: Mind you I think the real potential of PIM RAM would be in both CPU RAM and GPU RAM so all aspects of game programming can benefit from bypassing the RAM > CPU/GPU latency bottleneck.**

** Still hoping the Unity DOTS system will become compilable to GPU so developers can take full advantage of all available processing power on a platform as well as PIM RAM.

 

I definitely agree with what you are saying. I should have noted in my comment that my comment really had nothing to do with the discussion. I was not endorsing these actions, I was more just remembering fondly, lol.

 

Q: Why don't systems show up in the Unity Editor Scene?

I thought the whole point of the Editor in Unity was a WYSIWYG system that makes nearly everything visible and obvious to the developer. Or am I missing something e.g. A view mode or some gizmo settings?

Note if there are any keynote videos or blog posts I've missed please let me know.

 

Did you mean "entities" instead of "systems"?

 

No Systems: Where are the Systems in a scene, or Worlds with Systems in them?

 

Window > Analysis > Entity Debugger.

 

My point is why is DOTS not becoming integrated with the Editor in the Inspector and Hierarchy. Then you can see what systems you have in your scene/world setup and potentially enable/disable them during play to test them e.g. find a tricky inter-system bug or just to check that your systems are doing what you expect them to.

Or make DOTS systems a first class element of the Unity Editor WYSIWYG experience and not something you have to read documentation on or ask about on the forum to find out e.g. Unity DOTS had a built in translation and rotation system that will activate with components with the right data...

Or think of it as Transparency if you can see your DOTS system list in the inspector you know it's active and running and you can even toggle debug mode to see it's state e.g. memory usage, CPU usage, elements processed per ms, element pool size.

 

Entity Debugger currently does this. It even shows the system hierarchically and you can filter to active running systems or show full player loop.

Entity Debugger also shows if the system is running, which entities matched the query, and main thread CPU usage just to name a few.

Also, when you select an entity, all of its data shows up in the inspector. So the only things that aren't there yet are the hierarchy view of entities and the ability to modify runtime entity data from the inspector for fast iteration and debugging purposes. DOTS is still in preview, so not all features are there yet. But I encourage you to try out the tools that are there because as this post has hopefully shown, they are a lot more powerful than you might realize.

 

@Arowx, new thread just popped up yesterday that might answer some of your questions.

tl;dr: it's actively being worked on.

 

It's being worked on = it's not ready by a long shot. This is why I made the unfortunate decision to stick with MonoBehaviours.

 

Here's an idea why not do a game jam where developers have a limited time to make a Mono game then the same time to convert the game to DOTS. In theory you should just have to cut and past OOP functions to systems and create Entites from their calling parameters...

>Rant that triggered this idea...

Is Unity in developing DOTS forgetting the core programming/API heristic K.I.S.S. (Keep It Simple Silly).

I'm seeing huge amounts of complexity being added to do things that were built into the Editor with Monobehaviours:

• The use of authoring components just to work on your game in the editor.
• Convert to Entity Code seems really hacky and wasteful. If Entities/Systems were made first class Editor components there would be no need for this conversion code base system.
• Simple aspects of the OOP system made way more complex with Entities e.g. Translate replacing Transform.
• Ditto where API knowledge/styles from OOP could have been moved over to Systems/Entities e.g. Awake() on Systems.

It's great to see such a huge amount of work being done but I have to question some of these decisions. From the perspective of a Monobehaviour developer who keeps trying to pick up and work with ECS/DOTS and just keeps hitting a lot of knowledge barriers that could be made a lot easier to work through if Unity just thought about how it could make working with DOTS easier and quicker.​

 

Unity being easier to use was the main reason why I picked it over Unreal few years ago. I didn't care about graphics or performance.

Now Unity makes almost a new engine - dots, urp/hdrp/hybrid, netcode, uielements (or ui toolkit) etc. Next 2-4 years will be really interesting. Could be the greatest victory of data oriented design or the biggest fail of something that sounds good but isn't needed in majority of projects.

 

I think it's just a case of the Unity head shed of programmers are working on DOTS and really they should be bringing in the UI/UX developers to make it easy to use. DOTS done well and visually could make programming easier as you just write simple functions in systems and set up data in Entities then combine them to make a game. You are breaking down a game into simple problems.

Instead they are managing to make it way more complex which is great for smart programmers who can show off how they understand this complex system and make it do their bidding and work around problems it generates.

Only... are most of those problems being caused by the growth in complexity and lack of good visibiltiy?

Imagine being able to create systems that appear in your editor scene (with DOTS gizmos turned on) and being able to add and edit entities in the editor. Systems that generate and consume entities could show visual input output lines to entity pools.

 

Don't forget that DOTS is not ready by a long shot and they're still changing the API as they figure basic stuff out. It's good work and I'm excited to see where it ends up but as of right now it's just a fun toy to play with. If you actually want to ship something its use is extremely limited.

 

Not sure if this is the right thread, but I just converted my ComponentSystems & JobSystems to SystemBase and I love it. Great job Unity, it's so much more elegant and so much easier to work with!

SystemBase providing Get/SetComponent is also extremely useful for readability.
Would it be possible to add GetBuffer<T> as well?

 

May I ask for the documentation link to the Get/SetComponents, I dont see this anywhere