Search Unity

My uneducated opinion on this

Discussion in 'Data Oriented Technology Stack' started by DerDicke, Sep 12, 2019.

  1. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    Ok, disclaimer: I'm not really into this, just watched some videos, maybe I've some wrong assumptions.

    Have a look at this video:

    It shows a C++ beginner example from the early 90s that was made to get an idea how C++ works (as opposed to C). Then it shows how you can mess up your code by using shown design pattern in a hypothetical real world example, followed by performance considerations and finally ECS to the rescue.
    Some thoughts on the video:

    1.) It's already in!
    Unity already has a powerful Entity-Component-System in it. The Entity is the GameObject, it's a container holding *Components* (hence the name), which give functionality to your objects. So you would *never* write an 'Animal' class and inherit different animals from that. This is early 90s C++ sample code. Not Unity. In Unity you would describe what an animal can do by adding a script for every specific functionality your specific animal (e.g. a bird) can do and save the GameObj as a prefab. So we want to have a bird in our game.
    We need a script describing how it flies. If we want the bird able to take damage and die, we add another script handling this. If we want it to make some crow noises, we add another script handling that. We would never assemble all of that in a single 'Bird : Animal'-class.

    2.) No clue about real world game development.
    An actual bird in a actual game that needs hitpoints would most likely also have:
    - a behaviour state machine, maybe a behaviout tree.
    - a hitpoint/stats system, perhaps skill and/or weapon based.
    - an Animator
    - collision

    3.) Dots is limited.
    Tell me if I'm wrong but what can it actually do? Calculate positions and rotations multithreaded? But that's all, right? So it's a niche solution for a niche problem.

    Wouldn't it be much more useful to have a special kind of restricted GameObject used for exactly this purpose? Say you want many birds flying around with no effect on gameplay. Or particles. At least as a first step into your brave new ECS world?
    Creation and update of this ECSGameObj would be different, Monobehaviours wouldn't work, and any code on it must be compatible to your ECSSystem. So people can use multithreading on systems when they actually need it, the moment they hit that performance border.
     
    Last edited: Sep 12, 2019
  2. ndesy

    ndesy

    Joined:
    Jul 22, 2019
    Posts:
    9
    I'm not sure what you are expecting from this thread, but...

    DOTS is all about data layout and execution flow. If you put your mind to it, you can do pretty much anything!
     
  3. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    Yeah yeah, I do understand this. But this is only interesting as an engine programmer. It doesn't matter at all in 99% of Unity-applications. If your engine's data-layout is bad, improve it. But ok, if you give me the opportunity to have a hand in there (in case I'm on one of the 1%) that's nice.

    But tell me. What else can i put in my IJobProcessComponentData<>'s Execute() function?

    I'm expecting some reason to use this besides of fanciness.
     
  4. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    106
    Maybe become more educated. Please come back when you have at least tried to answer some of your own questions.

    Also, look up Blizzard’s GDC talk on using ESC in Overwatch.
     
  5. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    1,667
    Everything?
     
  6. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    815
    Pretty much anything that needs to do some sort of "action", be it calculation, movement, logic processing, you name it. Any of those IJobWhatever, pretty much allows you to take most things you would have done in a normal Update(), but do it multithreaded and with Burst compilation.

    Just as an example. To learn how to use it all I have been working on a waypoint based pathfinding system. Using DOTS / Jobs, etc, I am able to spawn 50,000 different vehicles, all following their own routes (even though it's hard to tell because there are so many), each one independently doing their path finding, etc, and it's running like a beast.

    https://i.imgur.com/a91Ky6O.gifv

    Another example was I took that, added in the new Unity physics using Burst and everything. I originally made it all using gameobjects and such a year or two ago. I was able to have probably 40 cars with physics, a handful of people walking around and performance was beginning to take a hit. Now, with this in my build I was getting over 700 FPS even with a bunch of collisions going on, the pathfinding, the movement, interacting with the mouse, etc.

    What it comes down to is pretty much everything and anything you can do with gameobjects currently, you can do with the new systems, but do it 10's to 100's of times for effeciently, which allows you to either A. take what you already have and make it run way better than it was before, or B. it allows you to add way more depth and complexity to your game and systems while still running way better than it was before. (Generally speaking, as some stuff isn't completed yet, or is still going through iterations, etc, but there is plenty enough available currently to make heavy use of it, or even a full game)
     
    Last edited: Sep 12, 2019
    wobes and PublicEnumE like this.
  7. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    106
    Wow, you’re getting 700 FPS with all of this? That’s mad! What kind of machine are you working on?
     
  8. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    815
    Just to clarify, I wasn't getting 700FPS when there was 50,000 cars, as it was something like 75 million polygons on screen or somewhere there about. 700FPS was with about 40 cars each with physics running around in that same play space above running into each other.
     
    PublicEnumE likes this.
  9. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    Ok, did have a look on the video. My opinion:
    All the problems with increased complexity on Actor based systems can be easily avoided in vanilla Unity:
    As for Object destruction, Destroy() happens at the end of the frame for exactly that reason. No need to do anything. As for effect creation causing z-fighting, this is a special case. It can be handled by some singleton EffectCreationHandler which takes requests and synchronizes effect creation at the end of the frame. Exactly what their system does. But the big advantage in vanilla Unity is that you don't have to build that functionality into your basic ECS system. You just add it in case of z-fighting, making development much more incrementally.

    Also let me mention that many (if not all) of these problems arise in C++ only games with bad engine design. Reason is that any newly created actor also changes state of a lot off other sub systems like renderer, resource loading, physics and so on. So if your engine does not delay object creation to the end of the frame, you get exactly that: lots of dependencies that you have to handle somehow. AFAIK Unity handles this gracefully, probably by delaying until the end of the frame, but I'm not sure about this. But it works out of the box.
    My impression here is that they simply didn't understand that Object creation at the end of the frame would have made their life a lot easier. I'm a former C++ engine developer and know how hard it is to get out of your C++ box and figure that out.

    Did not see the whole second part of the video. It is about the NetCode and seemed unrelated to our topic here. Correct me if I'm wrong.
     
  10. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    Ok, performance. A good reason. But not if your game is fast enough. Wouldn't it be nice to have a system that automatically arranges it this way when just giving hints? Or a system parallel to vanilla Unity that you can use for exactly these cases? Instead of relying on this totally new stuff. Always remember that new Unity features need years to mature to a point to be actually usable.
     
  11. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    106
    If you’re an experienced C++ engine developer, then you should know better than to poop on this tech initiative. The benefits to perf, design, and organization are well documented, with test cases. Lift a finger like the rest of us here. :p For all of your experience, you just sound silly.

    This is early tech that is in active development. Unity has made it public to get user data and feedback. Instead of talking about how terrible it is, please apply your experience to suggest specific improvements (to DOTS).

    Most of your comments read as if you have only done a cursory amount of research to understand the goals and current state of DOTS. You comment on missing features. DOTS is in active development, and while you can use it for some things in your project today, no one should expect to be publishing pure DOTS projects for at least a few years.

    Good day, silly guy.
     
  12. Griz

    Griz

    Joined:
    Aug 27, 2013
    Posts:
    108
    You can think about DOTS as a tool for solving performance problems and for products that wouldn't be possible without it. Of course you might never use it for games that already run "fast enough", but the truth is that Unity as an engine missed such a toolbox for a long time.

    As you said MonoBehaviour workflow is great for quick iterations and for just getting more results in less time, but when some solutions get figured out and finished, experienced developers can just move them into the DOTS world to make place for more cool features in your game, instead of fighting to keep your game run bare minimum 60, or worse 30 FPS.

    Notice how nowadays lots of software is just "fast enough". When first SSD drives got out to public, software loaded blazingly fast. Now, since this drive speed is expected, developers just turn a blind eye for increased drive access needs, when ealier they needed to be heavily optimised. With games it's similar. It's nice to have a game that doesn't eat your phone battery in 1 hour, when it doesn't need to, and which works in 60 fps instead of 30 fps, or even more for those gamers that enjoy high refresh-rate experience or VR on lower end hardware.
     
    PublicEnumE likes this.
  13. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    No hate please. I just don't like the (currently not deserved) hype.
    Already had a few constructive 2 cents in my first post:
     
  14. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    381
    I think its all simpler.

    DOTS is all about "performance by default"
    As author mentioned, its all the same concept as currest GameObjects and Components.

    The only difference is the named Entities and Components now.

    Yes current api somewhat different but in future DOTS will be mature enough to have all systems and just seamlessly replace current OOP design but with performance by default.

    For casual games - just forget about DOTS for at last 3 years and then just seamlessly jump into it.

    For any heavy games jump into DOST now and open access to yourself for that performance by default thing :)
     
  15. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    381
    For Unity it is the way to be performant Game Engine, chance to do everything from ground up and right and not to support 2 thing inside engine in future, but spend time to improve one good solution.
     
  16. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    381
    Personally I have very Appreciate new Unity way of growth
     
    wobes likes this.
  17. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    That's what I'm thinking about it. But does it currently go together with vanilla Unity? Can I have both in my project? AFAIK this is not the case, but like I said, I'm uneducated here.
    Of course I would love to have the opportunity to add effective multithreaded ECS when needed... but aren't there more important changes needed in Unity? Just saying... No offense btw.
     
  18. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    106
    DerDike, what Is DOTS? What are its features?
     
  19. Griz

    Griz

    Joined:
    Aug 27, 2013
    Posts:
    108
    Yes, you can! And that's the way I'm using the DOTS at the moment, as i like what I have available in the MonoBehaviour world.

    About importance, I think that each person has his own favorite features he looks forward to. I myself am happy that DOTS is here and that it is mature enough to handle production stuff.
     
    PublicEnumE likes this.
  20. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    Basically separate data and code. Have an array of data and manipulate it with your systems. Each system performs an specific calculation on some specific data. Objects are combinations of that data. Can be easily multithreaded.
    Have a look here: https://unity.com/dots

    But I'm sure you know something about it... Say, does Unity pay people like you to shy away critisism?
     
  21. DerDicke

    DerDicke

    Joined:
    Jun 30, 2015
    Posts:
    176
    Mature for production? Okay... didn't expect this. Cool.
     
  22. Griz

    Griz

    Joined:
    Aug 27, 2013
    Posts:
    108
    Feel free to check Eizenhorn's sticky post, where he explains how he got his game project rolling with DOTS, and is a good proof that it's viable for production: https://forum.unity.com/threads/unity-ecs-and-job-system-in-production.561229/ . It helped me to wrap my head around DOTS and I can't recommend it enough.
     
  23. PublicEnumE

    PublicEnumE

    Joined:
    Feb 3, 2019
    Posts:
    106
    Thanks, but I still don't get it. Why do you want these things? Or automatic cache optimization?
     
    Last edited: Sep 12, 2019
  24. Razmot

    Razmot

    Joined:
    Apr 27, 2013
    Posts:
    262
    my 2 cents :

    Because of Dots, I can do one thing that I wasn't able to do before: reliable multi threaded code. Because it's designed for it, with the jobs, struct based components, dependencies, and last but not least : the safety system that helps me a lot to not shoot myself in the foot with race conditions.
     
    Jes28 and Antypodish like this.
  25. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    I would really suggest to look into megacity demo. Or at least talk. Then we have FPS sample, highly related to DOTS. Then we got samples on github, which allow us to learn by example, and create what we desire. See for example boids fish school. Also we got massive battle of minions. Worth to watch demo and technical talk. Plus other ECS/DOTS related talks.

    Then we have tons of examples, what people achieved here with Jobs, ECS / DOTS And that's with relatively short period of time, since DOTS is still in preview. Officially is not for production. But multiple project proved production usability already.

    Anything other than that, is left to own imagination, what want to do with available tool.
     
  26. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,417
    For best performance and fast access. Cache misses is one of many challenges, which DOTS helps on that matter greatly.

    Things only can get obvious, when start working with large volume of data.

    Please don't forget also, we have Burst, which has critical impact on DOTS performance.
     
  27. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    436
    Wow. I see a lot of confusion here. Let me try to clarify a few things. First off, DOTS != ECS. ECS is just a small piece of the puzzle. The other pieces are Burst, Easy, multithreading, new physics engine, new networking stack, partial scene streaming, new audio engine, and a whole bunch more down the pipeline. The officially released pieces, Burst and C# Job system, specifically address performance issues with C# and performance in general. If none of these things matter to you, fine. Stick with GameObjects. They aren't going anywhere.

    Now about ECS. First off, GameObjects are not an ECS, they are an Entity-Component architecture. This architecture modularizes the data and logic and helps promote single-responsibility principles. ECS is an Entity-Component-System architecture. Logic (systems) is separated from the data (components) to create a higher level a decoupling and simultaneously allow for batch reasoning. The extra decoupling makes code extremely reusable, far greater than anything I have ever had with GameObjects. And even better, I don't have to re-optimize the code for each project because the code is already able to make use of all the batch processing optimizations.

    Is all this stuff production-ready? No. I would best describe it as "production-usable". It's missing a lot of "nice-to-haves", but people have been re-implementing Unity features with custom ones for performance or design reasons for a long time now, and ECS only makes that easier. What is present currently is holding up pretty well for the early adopters, and it is only going to get better.
    How do you want to combine them? If you just want to swtich back and forth between them on a per scene basis, that's the easiest to get working. If you want to do all of your logic in systems, Entities.Foreach work on MonoBehaviours. If you want to do a true mix, that can be more involved but is still possible.

    All of this tech is meant to make Unity much more AAA-viable, and at the same time, giving powerful AAA tools to indies to go nuts with.
     
  28. Tudor-Nita

    Tudor-Nita

    Joined:
    Dec 10, 2009
    Posts:
    336
    I will avoid most of the things already mentioned here but will restate that DOTS is an umbrela term for Burst, Jobs, ECS and their dependencies. We'll also agree to ignore that ECS solutions have been built - commercially - ( in c/c++ ) for well over a decade now, with fairly ok results ( a lot of DS games made heavy use of this to get something resembling a AAA game on the platform ). We have at least 1 programmer on our team that wrote one, commercially, more than 10 years ago.

    My two cents follow.
    Forget ECS for now unless:
    a) you have something that would specifically take advantage of this and are willing to go through fire for it.
    b) you need a really quick number cruncher ( but be very weary of your bandwidth in this case )
    c) you need a "giblet generator" ( or any other mostly self-contained small but heavy system ).
    d) you have an easy enough case for it ( think uber casual games for mobiles ) and you want to really optimize for battery-life, better effects than the competition, etc.
    e) you are planning to have a lot of custom bits anyway ( might as well have them be ECS )
    f) you are a huge company, making a lot of re-skins or "clones" ( ties in with d ) where you only build a game once ( and sell it 30 times )

    When I say forget it I mean learn it, keep up with its development, practice it and ... forget it.

    The Burst stuff is A-mazing. ( not comparing to any c++ engine here, just to Unity without it )
    Jobs are stable and really easy to work with.
    Native collections are, overall, well-done.

    Physics stuff looks promising but is more likely a reaction to PhysX development slowing down ( or wrapping-up, depending on your view of things ). Not a replacement yet and not really part of DOTS ( it's a byproduct or consumer of said stack ).

    Unity is also making use of this below the hood in various other places, and performance is clearly improving overall.
     
    Last edited: Sep 12, 2019
  29. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    1,432
    MB land is COP/COA (component oriented programming\approach), COP != ECS, it's like compare MVC with ECS. At first look they seems same, but completely different.

    Only while you thinking in OOP way, after switching your mind to DOD you'll break this wall

    Our game is extremely utilize DOTS everywhere, you can do everything. In our case we it's citizens logic, our custom navigation, our custom physics, world generation, our custom rendering, our custom animation, war logic, building system, disease system, indicators, even fog of war calculation etc. etc. And we use it in production from the very beginning to the present. (you can see it from my signature or pinned thread on this forum or from link from guy above.)
    (playlist https://www.youtube.com/playlist?list=PLLjxduedQBqGKN4zZ_ATt1Yoatr1pF9dP)


    DOTS is NOT only about performance, and, techniсally, performance is "side effect" of perfect memory layout and data structure which results in such a performance. In first place DOTS, if be more specific ECS, wrapped around DOD - paradigm on which it is builded it's effective data usage, it's great scalability in comparsion with OOP\COP, it's much better code reusability and modularity, as bonus result for mobile - is low battery consumption, because your code using multithreading, finishing work early, and sleeping more time.
     
    Last edited: Sep 12, 2019
    brunocoimbra, MostHated, Griz and 2 others like this.
  30. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    815
    Anytime anyone says something like "ECS / DOTS / DOD has no use outside of XYZ" or any sort of limiting factor making it unfit for creating an entire game, your game is my go-to link to share, lol.
     
    Last edited: Sep 12, 2019
  31. thelebaron

    thelebaron

    Joined:
    Jun 2, 2013
    Posts:
    277
    calculating position and rotation along with some meshes :)
    only gameobjects in this scene are the lights and camera.
     
    Griz, eizenhorn, wobes and 2 others like this.