Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

API usability

Discussion in 'Data Oriented Technology Stack' started by Adam-Mechtley, Sep 4, 2018.

  1. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,267
    You can't (in safe way), SCD is not about data storing in general idea.

    Don't forget - it's in preview, and player loop not in final state
     
    FROS7 likes this.
  2. Shinyclef

    Shinyclef

    Joined:
    Nov 20, 2013
    Posts:
    338
    Hmm that's a shame. I figured you could because of things like shared render components, but I suppose those are not accessed after assign to GPU or something.

    Yeah, just figured it's therefore important to provide the feedback early. Also the more the play loop changes, the more important it is to be able to easily reorder your own systems.
     
    GliderGuy likes this.
  3. vitautart

    vitautart

    Joined:
    May 3, 2018
    Posts:
    29
    Not sure if this is right thread. It would be cool to have sticky thread like "General ECS feadback from users". But anyway.

    I have proposal to add little function to EntityQuery API that calculates chunk count, like it has already for entity count - EntityQuery.CalculateLength().
    Usecase for example, when you need to allocate some temporary native container with capacity of chunk count, for this I need first to allocate nativearray for chunks, than take length, and than dispose(), and sometimes this chunk array is not used anywhere except only for getting length from it.

    Instead of this:
    Code (CSharp):
    1. var query = EntityManager.CreateEntityQuery(queryDesc);
    2. var chunks = query.CreateArchetypeChunkArray(Allocator.TempJob);
    3. var array = new NativeArray<int>(chunks.Length, Allocator.TempJob);
    4. chunks.Dispose();
    Will be cool to have:
    Code (CSharp):
    1. var query = EntityManager.CreateEntityQuery(queryDesc);
    2. var array = new NativeArray<int>(query.CalculateChunkCount(), Allocator.TempJob);
     
  4. BenzzzX

    BenzzzX

    Joined:
    Jun 4, 2018
    Posts:
    6
    I've been trying to implement a ecs framework with cpp recently.
    And I found that inline job via lambda is so well suited to express the work flow.
    Some concept code.
    Code (CSharp):
    1. void ScheduleFrame(ecs::Flow& flow)
    2. {
    3.     static ecs::Entity e[1];
    4.     static ecs::Entity es[10'000'000];
    5.  
    6.     flow.Sync([](ecs::Context & context)
    7.         {
    8.             context.CreateEntities<Test, TestInternal>(e, TestMeta{ 1,2 });
    9.             context.WriteComponent<Test>(*e) = { 1 };
    10.             context.WriteComponent<TestInternal>(*e) = { 13,2 };
    11.             context.Instantiate(*e, es);
    12.         });
    13.  
    14.     flow.ForEntitiesPar(/*filter,*/[](Test& test)
    15.         {
    16.             test.a += 1;
    17.         });
    18.  
    19.     flow.Task([](ecs::Accessor<const TestInternal> test)
    20.         {
    21.             /*async*/
    22.         });
    23.  
    24.     flow.Sync([](ecs::Context & context)
    25.         {
    26.             context.DestroyEntities(e);
    27.             context.DestroyEntities(es);
    28.             context.RemoveComponent<TestInternal>(e);
    29.         });
    30. }
     
    Jes28 and illinar like this.
  5. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    6,924
    What about discoverability, e.g. when working with the API sometimes you will literally be searching for the feature you want with the tooltips provided.

    Just recently I spent a good half an hour trying to find a way to Scale an entity, as the default .Scale(float s) function was all I could find until I dug into the documentation and found .NonUniformScale(float3 s)

    You could make the API more 'discoverable' by naming the functions like this by what they do first then how they do it e.g. I would have found ScaleNonUniform(float 3) at the same time I found Scale().
     
    Shinyclef likes this.
  6. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    547
    @Arowx most of the major naming conventions suggest to use more "english" sounding names, more readable names. And Unity ECS is made following NET standards as far as I know.

    The current API should be pretty discoverable with IDE like VSCode at least.

    upload_2019-5-6_20-23-52.png

    In any case things should be okay when the stable api and Documentation are in place. That transform documentation looks really crazy though. Could use more clearly highlighted definitions and key points and maybe fewer examples.
     
    Last edited: May 6, 2019
    AndrewKaninchen and TeagansDad like this.
  7. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    547
    GetSingletonBuffer<T>()
    would be nice to have.
     
    siggigg likes this.
  8. LazyGameDevZA

    LazyGameDevZA

    Joined:
    Nov 10, 2016
    Posts:
    74
    Having a more flexible singleton api would be great. The team from Blizzard that built Overwatch said they use singleton components as a means to abstract systems away from one another. One of the key things being that only the system that owns the singleton is allowed to write to it. In my view exposing the physics world for Unity Physics would be a great example of this as it's a data structure that needs to be queried in some way by other systems.
     
    illinar likes this.
  9. vitautart

    vitautart

    Joined:
    May 3, 2018
    Posts:
    29
    I have proposal and question about NativeQueue.

    Proposal. It would be nice to have method
    NativeQueue.ToNativeArray(Allocator)


    Question. Is concurrent version of this queue realy deterministic? Will order of elements be the same on different processors, or at least on x86?
     
  10. illinar

    illinar

    Joined:
    Apr 6, 2011
    Posts:
    547
    Rename ISharedComponentData to IChunkComponentData?
     
  11. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,267
    It'll confusing, cos we have chunk components
     
  12. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    4,572
    I think shared component makes more sense.
     
    Shinyclef likes this.
  13. GliderGuy

    GliderGuy

    Joined:
    Dec 14, 2018
    Posts:
    26
    Now it's technically not API, but It'd be nice if when you're hovering your mouse over a variable that is a struct that derives from an ECS interface like IComponentData or IBufferElementData, instead of displaying the type of the variable it would display the interface the struct derived from. Because we all know an IComponentData is a struct, but we don't know if a struct is an IComponentData!

    Of course, I'm not actually sure if it's even possible to do this because it might be code-editor based (and thus you have no control over it), plus it might not be a good idea from a consistency standpoint... But it's something i noticed and it bugged me. :oops:
     
    Last edited: May 17, 2019
  14. Ryetoast

    Ryetoast

    Joined:
    Mar 8, 2015
    Posts:
    34
    This isn't direct feedback on Unity's ECS, but I was inspired to try making my own ECS library in C++ to see what it would turn out like. It's not as fast as Unity's ECS for various reasons (lacking many batch optimizations, and this is basically my first dive into multi-threading). I am, however, quite proud of its API and user experience, so I thought I'd post it here for the Unity DOTS team to look at:

    https://github.com/RyeToastyO/Ecs
     
    GliderGuy likes this.
  15. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,182
    I think the "data" is more misleading than "shared", it make you think you could store and use that data at will. It need to feel more auxillary-ish.

    Maybe something like IHashedComponent / ISharedHashComponent / IHashedTagComponent ? When removing "data" it is more explicit that your data were transformed and is not technically there with the entity anymore.

    Then the old GetSharedComponentData that could get the real thing with SCD index/with associated entity could be instead named GetDataOfHashedTag(...). By reserving the "data" wording to use here it may be more explicit that you are converting the index to real data. (and that you couldn't do this in the job)
     
    illinar likes this.
  16. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    4,572
    In my opinion adding Hash word to the Component name, will get confused with other "hashes".
    But I know where you come form and why this proposition. However, I am not convinced, "Hash" is more descriptive in this case.

    Yet data is data, weather is shared or not. In fact all is data, as DOD indicates :)
     
  17. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    860
    To me it is an ArchetypeSplitterData (without restriction what data, ie can be managed, etc)

    And it’s split by an index to the actual data / object (whereas only the index is accessible in a job)
     
    illinar likes this.
  18. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    423
    technically, it's not strictly a component (it's associated with a chunk, not on an entity). it is data though.
    what about
    IChunkPartitionData
    /
    IArchetypePartitionData
    ?
     
    illinar likes this.
  19. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    4,572
    Isn't "partition" associated with "division" into parts?
    Which may lead to potential confusion. Maybe?
     
  20. M_R

    M_R

    Joined:
    Apr 15, 2015
    Posts:
    423
    indeed it divides/partitions the entities into different chunks
     
  21. recursive

    recursive

    Joined:
    Jul 12, 2012
    Posts:
    546
    I like ISharedGroupingData myself. Says exactly what it does on the box without getting too technical.
     
    jdtec, Sarkahn and Antypodish like this.
  22. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    6,924
    Would converting a game from No API to ForEach to Fluent and then to Chunks be a good way to test the API?

    The aim would be to see how much the code changes between versions and to see how much boilerplate is requried for each version of the API.

    Also it could highlight elements that do not translate well between APIs.
     
    GliderGuy likes this.
  23. Ryetoast

    Ryetoast

    Joined:
    Mar 8, 2015
    Posts:
    34
    So I went back through the Unity API to compare big differences:
    1. Singleton Components
      • The Unity implementation is basically just a helper function to assert there's one of a component and to get it. This places the burden of guarantees onto the user.
      • My implementation requires a singleton component to inherit a separate interface. It guarantees that GetSingleton and SetSingleton always succeeds, and that exactly one exists per World. It also allows jobs to access them in basically the same way that normal components are accessed, affecting the read/write dependencies, but not the entity query being iterated.
      • I also believe singleton components are significantly more useful if they can contain non-POD data. Storing things like a native multi map built in one system for several others to consume is a relatively common occurrence in my experience. The current unity method of getting the other system and manually applying input deps again places the burden of guarantees on the user. This is easier to allow if they are treated as actually separate concepts instead of just shortcut accessed normal components.
    2. Job Data Access
      • Component access in unity is very inconsistently accessed based on the type of access, type of job, etc. I solved this by basically making the entity query and the data access the same thing, managed by the job itself. Shared, Singleton, require only, and exclude components are accessed using the same, or extremely similar APIs.
    3. Job Ordering
      • Tagging for update before or after should pass the deps handle returned by the depended upon job.
     
    illinar likes this.