Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

I get the impression that ECS means I have to write way more code to do the same thing?

Discussion in 'Data Oriented Technology Stack' started by Arowx, Jun 7, 2018.

  1. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,574
    Do you have a specific example of that boiler plate code that obfuscates readability? Are you using IJobForEach or Entities.ForEach API's to write the majority of your game code?
     
  2. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    204
    I'd be willing to bet that you could forget about concurrency by only using ComponentSystems and scheduling and immediately completing jobs and still get better performance than GameObject architecture. Most ECS architectures I've worked with did this as they didn't have a true job system. Each "System" either "ran single" or "ran wide". And if that is simple enough that it saves you considerable development time to ship your game, go for it.

    A lot of people are getting hung up on the few use cases where a NativeContainer is required for several jobs across multiple systems. While these should never be the first goto solution, there are lots of real-world problems that require their use, and right now there is not a clean way to manage dependencies when these get involved. I use generic ICDs as trackers as a hackish workaround, but really some hook point to add rules to the dependency management would be a welcome addition from me personally.
     
    wobes likes this.
  3. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,574
    Generally i would advise not to use shared containers from different systems. By default i'd always try to represent things as IComponentData or IBufferElement.

    But of course there are sometimes reasons to do it for much more complex systems. Unity.Physics for example shares some containers between different systems.

    The recommended pattern is to store a Dependency JobHandle in the same place as the container.
    The callsite is responsible for chaining the job correctly. I am not sure there is anything we can do beyond just teaching that pattern. The container has nothing to do with any components in this case, so associating it with some specific components doesn't seem like a good idea.

    Code (CSharp):
    1.  
    2. class LookupTableSystem : JobComponentSystem
    3. {
    4.     public JobHandle Dependency;
    5.     public NativeHashMap<int, int> LookupTable = new NativeHashMap<int, int>(0, Allocator.Persistent);
    6.  
    7.     // Simple & not perf by default main thread API
    8.     // Automatically wait for any jobs that have accessed LookupTable
    9.     public int Lookup(int value)
    10.     {
    11.         Dependency.Complete();
    12.         return LookupTable[value];
    13.     }
    14. }
    15.  
    16. class ConsumeLookupTable : JobComponentSystem
    17. {
    18.     LookupTableSystem lookupTableSystem;
    19.  
    20.     struct Job : IJob
    21.     {
    22.         public NativeHashMap<int, int> lookup;
    23.  
    24.         public void Execute() {  ... }
    25.     }
    26.  
    27.     override void OnCreate()
    28.     {
    29.         lookupTableSystem = World.GetExistingSystem<LookupTableSystem>();
    30.     }
    31.  
    32.     JobHandle OnUpdate(JobHandle handle)
    33.     {
    34.         var job = new Job
    35.         {
    36.             lookup = lookupTableSystem.lookup
    37.         };
    38.         // When scheduling the job, ensure the lookupTableSystem dependency is included in the job dependency
    39.         var jobHandle = job.Schedule(JobHandle.Combine(handle, lookupTableSystem.Dependency));
    40.         // after scheduling the job assign it back to the dependency,
    41.         // this creates a dependency chain for any access of the lookup table
    42.         lookupTableSystem.Dependency = jobHandle;
    43.  
    44.         return jobHandle;
    45.     }
    46. }
     
    Sarkahn, eizenhorn and optimise like this.
  4. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    411
    Why not further relax IComponentData restriction to make it able to support NativeContainer?
     
  5. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    641
    In case of NativeQueue, disabling safe checks would be enough? Since the NativeQueue uses job thread indexes to write.
     
  6. nsxdavid

    nsxdavid

    Joined:
    Apr 6, 2009
    Posts:
    266
    You somewhat misunderstand me. The problem is not that the syntax of the boilerplate is not clear, it's that the amount of it needed to do ECS makes it hard to read code. Most of what you read is setup to do a thing, and winding down doing a thing, not doing a thing.

    Here is (admitidly cherrry picked and imperfect) example of what I mean. This coded map is from BiodSystem.cs. I highlighted the part of the code that actually expresses the behavior of the fish.



    Admittedly this is not the greatest example because, among other ECS-type and Job-type things, there is also some spatial data structure work here that would also be part of a non-ECS implementation. But even then, most of what's going on is copying memory from one format to another format in order to get it such that you can squeeze maximum performance and parallelism of the inner loop.

    To see how the fish actually avoid each other, the logic of that behavior, only the highlighted part is relevant.

    Understand, I don't think there is anything wrong with this in so far is it relates to making maximally performant code! What I am arguing is that this is never going to be the general case for expressing behavior in Unity. Or at least I hope not, least you just lose the vast bulk of your users.

    I also don't think you can abstract this away or hide it some 'viscual scripting' metaphore because, above all things, visual scripting is about expressing behavior not about the under-the-hood (thus, in theory, making it more accessible to more people). But DOTS is ALL about under-the-hood. It's about understanding how CPUs work at a detailed level and leveraging that. Sure, Jobs and Burst compiling does some heavy lifting... but made possible only because you put some serious limitations about what you can do in such code.

    And those limitations have a huge cost in expressiveness. They make it so you spend most of your time trying to figure out how you can even express a behavior in a way that'll work in DOTs rather than what that behavior actually is. And experimentation is equally hampered.

    Again this is an argument about DOTs being a generally useful things vs. a special-case useful thing.

    I get conflicting messages about how much DOTs is intended to be a supplemental thing vs. "The Future of Unity" scripting. And even today, I can't figure out how to make Hybrid DOTs work because the mechanisms don't seem to support anything I ever try to do. Again: the Unity Physics example I gave. So just expressing my concerns here.
     
    dadude123 and pachash like this.
  7. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,574
    I think you are comparing apples and oranges. If you want to write non-optimal order boid simulation code in DOTS you can do that. It will run significantly better than similar MonoBehaviour style code, you can use burst & Jobs to make 10-50x faster. The boid simulation code goes beyond that by using hashtables and sorting to create highly optimized boid simulation code. The boid sample code is quite advanced.

    If you want to write simple straightforward code similar to what you would write with MonoBehaviour for normal game code where you don't want to optimize a lot you can do that. Looks like this:

    https://github.com/Unity-Technologi...S/HelloCube_01_ForEach/RotationSpeedSystem.cs

    or this for if you want the same code jobified:
    https://github.com/Unity-Technologi...lloCube_02_IJobForEach/RotationSpeedSystem.cs

    You don't have to know about memory layout, because the entity component system lays data out for you. ForEach iterates over the data in the layout of the memory etc.

    If you want to you can dig in and get access to the lowest level API's but by default you don't have to.
     
    Zoey_O, jdtec and optimise like this.
  8. nsxdavid

    nsxdavid

    Joined:
    Apr 6, 2009
    Posts:
    266
    @Joachim_Ante

    Even in that case, many of the same issues apply. One is restricted to blitable types, and an inability to reference other things (except maybe in shared component data). It's like taking everything that you use to express behavior of a complex system and ripping out all the affordances and tools you come to rely on. It asks you to wear a blindfold and tie one hand behind your back. The moment you use Jobs and ECS, that's where you are at.

    How can that ever be made more than a niche way to express a game?

    Here's a concrete example:

    I have a character that I want to use in a world that uses Unity Physics. The character has some IK needs, so I'm using your new Rigging system as well. The former is DOTs the latter is GameObject and Monobehaviors. How do I use them together?

    (Unity Physics is needed because I need stateless for efficient rewinding)
     
  9. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    641
    @Joachim_Ante any chances to get my answer for the NativeQueue question?
     
  10. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,574
    I am not sure where this misconception comes from but you can reference other entities and components just fine, and you can read & write to their data from a job safely. Simply put a Entity on a IComponentData struct and use ComponentDataFromEntity<MyComponent> to safely access the referenced entity from a job.

    We are working on DOTS.Animation which is written completely in dots which will result in that you can have both physics & animation completely in dots and you can interact with it from jobs. We are also going to have the runtime IK rigging features in DOTS.animation and are planning to integrate Unity.Physics so we can have physics based IK rigs. Having full control over the physics system helps us do this properly.

    We have not released DOTS animation yet.

    Until then you have to provide that bridge between the two things yourself. Reading data or raycasts and feeding them into the GameObject based IK rigging system from code.
     
  11. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,574
    It's not how that works. If you want two different jobs to access the same container, they need to have a dependency between each other.
     
    GliderGuy likes this.
  12. wobes

    wobes

    Joined:
    Mar 9, 2013
    Posts:
    641
  13. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,251
    wobes likes this.
  14. sngdan

    sngdan

    Joined:
    Feb 7, 2014
    Posts:
    860
    I am also interested in the answer (I played with this a couple of ECS versions back). My conclusion was the same as @tertle's but I would have liked it to work as @5argon explained in his post.

    I thought the thread index is not guaranteed to be unique between different jobs (in different systems) - I had to share a container between different systems
     
  15. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,251
    Yep that is my understanding, thread index isn't unique between jobs and is what concurrent containers use to ensure thread safety.
     
  16. pachash

    pachash

    Joined:
    Apr 2, 2014
    Posts:
    55
    @nsxdavid I do completely I agree with you. Looking at the ECS examples I get the impression these are highly optimised technical demos which are, yes, blazing fast but...overcomplicated for a gameplay logic? I understand where pure ECS would shine but I'm not sure whether if it's suitable for gameplay scripting.

    I really want to be wrong but I see no sensible evidence. For instance look at the excellent FPS sample. It's an absolutely great example of code in many areas: networking, rendering, etc. But looking at its ECS usage for implementation of gameplay logic makes me scratch my head: it requires 5-10x more code for no real benefit. Doom and Quake source code is way more readable in this regard.
     
  17. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    4,577
    I wold say, if you want simplicity, use monobehaviour. But if you aim for high performing optimized code, look into DOTS/ECS. You can write simpler as well, but you may loose potential, to gain most of DOTS.
     
  18. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    232
    Everything has its trade-offs. Code monkey on youtube has good examples with simple ECS he uses only fluent API.
    Overwatch is written using ECS and developers said it made it easier to develop a game because of it. You need to take time to learn ECS.
     
    pachash likes this.
  19. pachash

    pachash

    Joined:
    Apr 2, 2014
    Posts:
    55
    For simplicity in model layer I'd rather stick with super simple OOP without any excessive abstractions and use monobehaviors in presentation layer only. Anyway as I mentioned earlier I hope to be wrong and hope to embrace the potential of ECS someday.
     
    Antypodish likes this.
  20. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    783
    I pretty much agree with @nsxdavid so maybe I can express it in my own words to hopefully make it more clear.

    As it is currently, the syntax, the amount of code required, and thus the amount of mental overhead required to read/write code like that is not really small. There are many things that force tons of tiny "syntax inefficiencies".

    Yes, I know why things are currently designed that way, and I can see where each of those design decision came from, but together they're death by a thousand cuts.

    Let me give a few concrete examples of problems:

    Example 1:

    Entities are just the sum of their components, but since each component is a struct we often have to "dig" into the component to get its value. Which is especially bad when the component is just a name for a single value.
    Like 'Rotation' requires 'rotation.Value' which seems like a tiny thing, but contributes a considerable amount of visual pollution.

    Example 2:

    Its not just the huge amount of boilerplate code.
    But there's also the way the core functionality itself has to be written.

    From the sample: here
    Code (csharp):
    1. rotation.Value = math.mul(math.normalize(rotation.Value), quaternion.AxisAngle(math.up(), rotSpeedIJobForEach.RadiansPerSecond * DeltaTime));
    Even the single line that represents the actual intent is a lot longer now:

    Code (csharp):
    1. transform.Rotate(Vector3.up, degPerSecond * deltaTime);

    However I believe it is possible to improve all of this significantly.

    APIs that use context more is one idea. Like Entities.ForEach, or fluent query (Entities.WithAllReadOnly...).
    Another idea would be to generate helper functions and properties, for example generating helper structs for known archetypes, so users can write things like "entity.rotation = ...".
     
  21. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    204
    Implicit casts for IComponentData solve this. Before the Transforms rewrite, the transform components actually had these implicit casts. They were nice. Unfortunately I don't know of a way to autogenerate the cast methods from an interface or attribute in C#. This is one of those things that C++ macros are good at.

    The mathematics library could use some maturing. quaternion.Rotate would be a nice addition. I suggest posting examples like this in the mathematics thread.