Official API usability

You can't (in safe way), SCD is not about data storing in general idea.

Don't forget - it's in preview, and player loop not in final state

 

Hmm that's a shame. I figured you could because of things like shared render components, but I suppose those are not accessed after assign to GPU or something.

Yeah, just figured it's therefore important to provide the feedback early. Also the more the play loop changes, the more important it is to be able to easily reorder your own systems.

 

Not sure if this is right thread. It would be cool to have sticky thread like "General ECS feadback from users". But anyway.

I have proposal to add little function to EntityQuery API that calculates chunk count, like it has already for entity count - EntityQuery.CalculateLength().
Usecase for example, when you need to allocate some temporary native container with capacity of chunk count, for this I need first to allocate nativearray for chunks, than take length, and than dispose(), and sometimes this chunk array is not used anywhere except only for getting length from it.

Instead of this:

Code (CSharp):

1. var query = EntityManager.CreateEntityQuery(queryDesc);
2. var chunks = query.CreateArchetypeChunkArray(Allocator.TempJob);
3. var array = new NativeArray<int>(chunks.Length, Allocator.TempJob);
4. chunks.Dispose();

Will be cool to have:

Code (CSharp):

1. var query = EntityManager.CreateEntityQuery(queryDesc);
2. var array = new NativeArray<int>(query.CalculateChunkCount(), Allocator.TempJob);

 

I've been trying to implement a ecs framework with cpp recently.
And I found that inline job via lambda is so well suited to express the work flow.
Some concept code.

Code (CSharp):

1. void ScheduleFrame(ecs::Flow& flow)
2. {
3.     static ecs::Entity e[1];
4.     static ecs::Entity es[10'000'000];
5.  
6.     flow.Sync([](ecs::Context & context)
7.         {
8.             context.CreateEntities<Test, TestInternal>(e, TestMeta{ 1,2 });
9.             context.WriteComponent<Test>(*e) = { 1 };
10.             context.WriteComponent<TestInternal>(*e) = { 13,2 };
11.             context.Instantiate(*e, es);
12.         });
13.  
14.     flow.ForEntitiesPar(/*filter,*/[](Test& test)
15.         {
16.             test.a += 1;
17.         });
18.  
19.     flow.Task([](ecs::Accessor<const TestInternal> test)
20.         {
21.             /*async*/
22.         });
23.  
24.     flow.Sync([](ecs::Context & context)
25.         {
26.             context.DestroyEntities(e);
27.             context.DestroyEntities(es);
28.             context.RemoveComponent<TestInternal>(e);
29.         });
30. }

 

What about discoverability, e.g. when working with the API sometimes you will literally be searching for the feature you want with the tooltips provided.

Just recently I spent a good half an hour trying to find a way to Scale an entity, as the default .Scale(float s) function was all I could find until I dug into the documentation and found .NonUniformScale(float3 s)

You could make the API more 'discoverable' by naming the functions like this by what they do first then how they do it e.g. I would have found ScaleNonUniform(float 3) at the same time I found Scale().

 

@Arowx most of the major naming conventions suggest to use more "english" sounding names, more readable names. And Unity ECS is made following NET standards as far as I know.

The current API should be pretty discoverable with IDE like VSCode at least.



In any case things should be okay when the stable api and Documentation are in place. That transform documentation looks really crazy though. Could use more clearly highlighted definitions and key points and maybe fewer examples.

 

GetSingletonBuffer<T>()

would be nice to have.

 

Having a more flexible singleton api would be great. The team from Blizzard that built Overwatch said they use singleton components as a means to abstract systems away from one another. One of the key things being that only the system that owns the singleton is allowed to write to it. In my view exposing the physics world for Unity Physics would be a great example of this as it's a data structure that needs to be queried in some way by other systems.

 

I have proposal and question about NativeQueue.

Proposal. It would be nice to have method

NativeQueue.ToNativeArray(Allocator)

Question. Is concurrent version of this queue realy deterministic? Will order of elements be the same on different processors, or at least on x86?

 

Rename ISharedComponentData to IChunkComponentData?

 

It'll confusing, cos we have chunk components

 

I think shared component makes more sense.

 

Now it's technically not API, but It'd be nice if when you're hovering your mouse over a variable that is a struct that derives from an ECS interface like IComponentData or IBufferElementData, instead of displaying the type of the variable it would display the interface the struct derived from. Because we all know an IComponentData is a struct, but we don't know if a struct is an IComponentData!

Of course, I'm not actually sure if it's even possible to do this because it might be code-editor based (and thus you have no control over it), plus it might not be a good idea from a consistency standpoint... But it's something i noticed and it bugged me.

 

This isn't direct feedback on Unity's ECS, but I was inspired to try making my own ECS library in C++ to see what it would turn out like. It's not as fast as Unity's ECS for various reasons (lacking many batch optimizations, and this is basically my first dive into multi-threading). I am, however, quite proud of its API and user experience, so I thought I'd post it here for the Unity DOTS team to look at:

https://github.com/RyeToastyO/Ecs

 

I think the "data" is more misleading than "shared", it make you think you could store and use that data at will. It need to feel more auxillary-ish.

Maybe something like IHashedComponent / ISharedHashComponent / IHashedTagComponent ? When removing "data" it is more explicit that your data were transformed and is not technically there with the entity anymore.

Then the old GetSharedComponentData that could get the real thing with SCD index/with associated entity could be instead named GetDataOfHashedTag(...). By reserving the "data" wording to use here it may be more explicit that you are converting the index to real data. (and that you couldn't do this in the job)

 

In my opinion adding Hash word to the Component name, will get confused with other "hashes".
But I know where you come form and why this proposition. However, I am not convinced, "Hash" is more descriptive in this case.

Yet data is data, weather is shared or not. In fact all is data, as DOD indicates

 

To me it is an ArchetypeSplitterData (without restriction what data, ie can be managed, etc)

And it’s split by an index to the actual data / object (whereas only the index is accessible in a job)

 

technically, it's not strictly a component (it's associated with a chunk, not on an entity). it is data though.
what about

IChunkPartitionData

/

IArchetypePartitionData

?

 

Isn't "partition" associated with "division" into parts?
Which may lead to potential confusion. Maybe?

 

indeed it divides/partitions the entities into different chunks

 

I like ISharedGroupingData myself. Says exactly what it does on the box without getting too technical.

 

Would converting a game from No API to ForEach to Fluent and then to Chunks be a good way to test the API?

The aim would be to see how much the code changes between versions and to see how much boilerplate is requried for each version of the API.

Also it could highlight elements that do not translate well between APIs.

 

So I went back through the Unity API to compare big differences:

1. Singleton Components
   • The Unity implementation is basically just a helper function to assert there's one of a component and to get it. This places the burden of guarantees onto the user.
   • My implementation requires a singleton component to inherit a separate interface. It guarantees that GetSingleton and SetSingleton always succeeds, and that exactly one exists per World. It also allows jobs to access them in basically the same way that normal components are accessed, affecting the read/write dependencies, but not the entity query being iterated.
   • I also believe singleton components are significantly more useful if they can contain non-POD data. Storing things like a native multi map built in one system for several others to consume is a relatively common occurrence in my experience. The current unity method of getting the other system and manually applying input deps again places the burden of guarantees on the user. This is easier to allow if they are treated as actually separate concepts instead of just shortcut accessed normal components.
2. Job Data Access
   • Component access in unity is very inconsistently accessed based on the type of access, type of job, etc. I solved this by basically making the entity query and the data access the same thing, managed by the job itself. Shared, Singleton, require only, and exclude components are accessed using the same, or extremely similar APIs.
3. Job Ordering
   • Tagging for update before or after should pass the deps handle returned by the depended upon job.

 

Watching this talk right now. I could imagine complex objects having many different components, yet SetComponentData only accepts one at a time. Would it make sense to have an EntityManager.SetComponentData overload that accepts a `params` as componentdata? That way we could write:

Code (CSharp):

1. manager.SetComponentData(Entity entity,
2.     new Translation { Value = transform.position },
3.     new Rotation { Value = Quaternion.Euler(rotation) }
4. );

 

You can do that as an extension method pretty easily. Note that params allocates an array, so it's probably not ideal.

 

In the meantime, Eric Lippert is not convinced that generic templates are useful enough to be implemented.

 

I'm erring on the side of templates not being as useful as well, because with a little boilerplate management, the program is the same work. The programs are absolutely tiny so I would like an actual use case to see the benefit of this argument.

Viewing latest Unite (videos), I saw very some great signs which indicate I would probably be best off writing code to change data as opposed to writing code to change code.

 

The only viable alternative of generic templates is code generation.
I hate code generation.

 

I only want to write code that transforms data. Anything else should be code gen or hidden from me. Why would anyone ever care about the things that do not affect their program execution?

So count me in for as much code gen as possible so my hours are spent transforming data not wasting it preparing to transform data.

So it's this thinking that allows me to accept codegen

 

@hippocoder just to add, in the end we don't much care, how on atomic level machine code writes to memory (sure we can look if is optimized). Is already code gen-ed for us, if form of compiler At least for most of us?

Times when I loved write assembly code by hand are long gone ..

 

Codegen is magic. It is not configurable, something gets broken in the generated code and you have to wait for people who developed it to fix the generator. I want to be in control of the code I write.

On a side note, I am not against compile-time code generation, that is fine. What I am against is the use of development-time code generation.

 

But this "magic" is not optional. If it breaks here, then it would've broken if you did it. The magic here is mandatory magic. I don't see why we should concern ourselves with mandatory and repetitive code. It's the same as accessing an API and find it's wrapped stuff that has to be wrapped no matter what.

Now if we talk about the bastard that UNET became then I agree, that's bad codegen. That's codegen you should have control over.

But this is not the same thing & I'm sure Unity's learned their lessons where to draw the line on it (essentially if its mandatory then it makes sense).

 

I was skeptical about codegen until we started using Entitas ECS framework. It uses attribute-driven code generation that supercharged our workflow, and over the past year we haven't hit a single problem with that generated code. It exists in the background, it doesn't interfere with anything we do and it just gives us a rich, useful surface to write systems on.

What's more, it's important to remember that most of Unity is not a black binary box. Almost everything, especially all these new packages, is either out in the open C# or in IL that can be shown as C#. Whenever I'm unsure about how something is implemented, I hit F12 in Rider and get a clear view of a part that interests me, even if it's inside some library. Furthermore, knowing where certain functionality is and how it looks, I can easily patch it without source access using tools like Harmony. So I don't foresee being in a situation where I'll be a hostage of some bug in Unity code, with nothing to do but wait for a fix - there are plenty of nice workarounds that can keep a Unity project chugging until patches are out.

 

Codegen is not magic. Especially if you have clear rules what you're generating and you have tests to ensure hat the end product code is what you expect.

 

Let's hope you guys are right.

 

What do

ComponentType.ReadOnly<T>()

/

ComponentType.ReadWrite<T>()

in a GetEntityQuery actually do? I tried to verify my assumptions in the docs and it's not clear or at least easy to find.

 

Meaning that a component in a query is only for the reading or for the reading and writing? It is self descriptive..

 

How to use it is self-evident, what it actually does is not at all obvious. What additional features / performance / saftey etc do you get from using ReadOnly instead of just ReadWrite everywhere?

 

It's for jobs so the scheduler knows which jobs can be executed in parallel for better performance.

And for what jobs need to be completed before a system is run.

-edit- an example

SystemA : JobComponentSystem, with a Job
SystemB : ComponentSystem, with a query
C : IComponentData
SystemA executes before SystemB

Case 1
If SystemA has a Job with a ReadOnly<C>
If SystemB has a query with ReadWrite<C>
Main thread needs to be stalled and Job completed before SystemB can run to ensure the data in job is valid.

Case 2
If SystemA has a job with a ReadWrite<C>
If SystemB has a query with ReadOnly<C>
Again main thread needs to be stalled and Job completed before SystemB can run but this time to ensure data in SystemB is up to date.

Case 3
If SystemA has a job with a ReadOnly<C>
If SystemB has a query with ReadOnly<C>
This time the Job does not need to be completed and therefore main thread does not need to be stalled as SystemB can run in parallel to the job in SystemA without concern.

 

I have some small feedback about create menu.
When we are using Unity physics then things related to it are in:

create>DOTS>Physics>[...]

And ECS are in the separate menu down below. I think it would be more consistent if all DOTS related things were it DOTS menu where physics is.

 

I hate the naming conventions used for the math library. Not only does it make it confusing, but it also makes converting existing code a laborious process. I understand the naming conventions used for the data types (eg float3), and actually quite like them, but boy does it hurt typing out "math.clamp()" right next to other functions which are formatted differently.

 

It is tailored toward shader convention.

 

Right, I understand that, but when it's placed next to normal C# code it looks wrong. It makes the readability of normal code confusing.

 

Classic OOP naming conventions, if that what you refers, will be used less frequent, once DOTS expand their API, with user friendly functionality. So it will be eventually less important. Not to say abandoned.

Current Unity C# don't even follow C# naming convention in many cases, according to standards documentation. So as long some convention in DOTS will be maintained, I will be happy for.

 

Suggestion:

GameConversionSystem.CreateEntity(): Create an empty Entity in the ConversionWorld. If conversion is happening inside a SubScene, create all the required SubScene components. The only way to create entities in a GCS is by replicating some prefab, which will certainly trivver some converters, and often we need plain empty entities, without Transform.

 

I mean classic C# naming conventions. We're not going (at least we shouldn't) abandon C# standards for DOTS. I see no reason as to why we should, or would.

 

There's such thing as classic C# naming conventions? Every place that I or my friends have worked at do things completely differently. In fact the only difference between Unity's C# style and my C++ style is that they are inconsistent with capitalization with member variables whereas I keep all variables consistently lowercase. I use m_ for private (so does Unity) and I have a syntax color for member variables. Also I use the same C++ style for DOTS C#. It transfers perfectly.

More importantly, in my C++ style, there are sometimes low level free function libraries wrapped in a namespace, and those libraries use camel_case to represent that they are low level essential functions, as if they were part of the language. It is a nearly identical paradigm to Unity making the mathematics library using all camelCase to reflect HLSL. While I don't remember where I got my C++ style from, I can tell you I didn't invent it.

Sorry, I get annoyed when people complain about a convention not following a standard when they don't understand why the individual pieces of the convention are the way they are to begin with. I'll go to bed now.

Edit: Fixed English

 

Code (CSharp):

1. I mean classic C# naming conventions. We're not going (at least we shouldn't) abandon C# standards for DOTS

Unity anyway never (strictly) followed the .NET Guidelines. Which should not be surprising considerung UnityScript and Boo were once in place.

Whatever they pick for everything in the future I only hope they will be consistent.

 

I am wondering, why in Entities.ForEach there are F_BDDD & F_DBBB but no F_BBDD (two components & two buffers)? Which is useful in some cases.