Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

[Sources Available] Physics in Pure ECS

Discussion in 'Data Oriented Technology Stack' started by PhilSA, May 16, 2018.

  1. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,080
    did a little update containing a bunch of cleanup, 2018.3b1 upgrade, and some exploration with new broadphase algorithms (but so far none of them seem to be an improvement)

    Also did a public trello to keep track of what has to be done: https://trello.com/b/XcYxQFrh/ecsphysics
     
    deus0, Razmot, dzamani and 7 others like this.
  2. mmankt

    mmankt

    Joined:
    Apr 29, 2015
    Posts:
    35
    btw heres my first try at fullt jobified 2d physics system:
     
  3. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,638
    Nicely done.
    Oh reminds me an interactive LionsHead Studio logo in Black & White 2 Game


    If that can be improved performance wise, that will be cool result.
     
  4. MostHated

    MostHated

    Joined:
    Nov 29, 2015
    Posts:
    849
    That was, I believe, first time I remember experiencing interactive physics in games. It was one of the coolest things I had ever seen.
     
    JustTheCoolDude and Antypodish like this.
  5. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,638
    Yep I loved it too.
    And the fact, discovering by accident, that you can interact with logo, moving it about ;)
    Such unique and fresh idea. I think even in this days.
     
  6. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    456
    Wow. Do u plan to open source it? Btw what's the tool u are using at top left?
     
  7. Micz84

    Micz84

    Joined:
    Jul 21, 2012
    Posts:
    236
    optimise likes this.
  8. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,080
    Seb-1814, Vacummus, Roni92pl and 8 others like this.
  9. mmankt

    mmankt

    Joined:
    Apr 29, 2015
    Posts:
    35
    I coded this in around 2 hours so I think I'll share this soon-ish cuz its really simple for now. It's not a full physics engine - just a proof of concept. I will be developing this further though.
     
    Last edited: Sep 16, 2018
    TZ-, benzsuankularb and optimise like this.
  10. Roni92pl

    Roni92pl

    Joined:
    Jun 2, 2015
    Posts:
    268
    Yo, is it possible to make it work completely async(without sync point on main thread), and just read results from unity thread when needed?
     
  11. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    This not how Unity job system works, you have to use it from the main thread. I wish it did allow it tho (it's one of the biggest limitations for me).
     
  12. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,080
    It's not possible. The physics engine needs a fixed timestep twork properly, hence why the sync points
     
  13. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    Job systems limitations aside, you can run the physics engine from separate thread too (not talking existing physx integration on Unity now). Some engines do this and I have proof of concept even with Bullet and Unity with this. Doing fixed timestep loop in main thread is just the traditional way how people do this.
     
    Last edited: Oct 8, 2018
    Roni92pl likes this.
  14. benzsuankularb

    benzsuankularb

    Joined:
    Apr 10, 2013
    Posts:
    129
    @PhilSA I take a look at your source. I'm really new for ECS now.
    There are pretty much states in the systems and some systems know another one.
    I'm not sure if it a good practice?
     
  15. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,080
    I am guessing you are talking about stuff like the "BVHArray" and "SphereSphereCollisionPairsQueue" in BroadphaseSystem. If you want good performance in situations where the order of elements in an array matters a lot, or when many entities would need to be created, I find it better to avoid using Entities and instead manage everything manually in containers.. If I had all my bvh nodes represented as Entities, I would have to sort them in the right order every frame because I don't have control in the ordering of componentDataArrays. And if I had my collision pairs represented as entities, I would need to create 1000's of entities every frame. It's much faster to enqueue simple structs instead

    This is required in many different situations. Some systems have to call updates to other systems in their own update (like PhysicsSystem for example). Also, some systems need access to global information stored in other systems. In many cases, these couldn't be replaced by static variables in the case where the same kind of system is in several different ECS worlds
     
    Last edited: Oct 10, 2018
  16. benzsuankularb

    benzsuankularb

    Joined:
    Apr 10, 2013
    Posts:
    129
    @PhilSA I think we can also put the "BVHArray" and "SphereSphereCollisionPairsQueue" sorted within a single entity.
    Now systems are stateless we don't need to access across systems anymore?

    Btw thanks a lot for beautiful source codes. gain a lot from that. :)
     
  17. Roni92pl

    Roni92pl

    Joined:
    Jun 2, 2015
    Posts:
    268
    Let me follow this subjecy though. As if you think fixed timestep is the only problem with this - it should be easy fix, - just run separate thread with fixed update calling sync point on another- physics thread? That works?
     
  18. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    @Roni92pl
    Ok, I'll reword this differently: You can't queue or complete Unity job system jobs from anywhere else than from main thread, it's a built in limitation of the system and therefore you can't trigger the physics jobs from custom thread.

    If you don't want to take my word on it, read following page from official Unity repo wiki, it tells the reasons why they put such limitation (check topics "Jobs can only be completed on the main thread - but why?" and "Jobs can only be scheduled on the main thread - but why?":
    Scheduling a job from a job - why not?

    I really wish it would be possible to schedule from custom threads as I dislike the currently popular main-thread gameloop using purely framerate dependent design for syncing it's major components: it compromises a lot of things in my used game genre but is obviously is easier to manage. I'd rather approach it from completely different angle (and is what I'm currently doing but without job system and burst).
     
    Last edited: Oct 11, 2018
    Roni92pl likes this.
  19. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,080
    I don't think this would be possible since they are arrays, and components cannot hold arrays. We could use Dynamic Buffers, but I'm guessing they are much slower than NativeArrays/Queues since they allocate on the heap after a certain point. Also, they would necessitate an extra step of array-copying into the DynamicBuffer since they probably can't be written-to in parallel jobs

    All in all I doubt there's anything wrong with storing state in systems, but I'm curious to hear about why it should be avoided. We could create an instance of a "PhysicsState" class to hold all that data, but that would be more or less the same thing

    I misunderstood your question originally and my answer didn't make sense, but refer to rizu's explanation above
     
    Last edited: Oct 11, 2018
    rigidbuddy likes this.
  20. Roni92pl

    Roni92pl

    Joined:
    Jun 2, 2015
    Posts:
    268
    Oh, I forgot about this ECS limitation. Thanks for explanation. BTW congratulations on this work so far, really impressed and I hope you keep working on it and eventually commercialize it.
     
  21. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,711
    Making cached structures like BVH be stored on entities is just crazy talk... That makes things unnecessarily complicated.

    Keeping data on systems is fine. The important part is for them to only be caches.
     
    The5, Rewaken, Seb-1814 and 6 others like this.
  22. IsDon

    IsDon

    Joined:
    Feb 27, 2015
    Posts:
    24
    Do you need to recalculate every Morton code every frame, given its an approximation? Just thinking about potential for optimisation, although I believe you were pretty happy that the Morton calcs weren't a huge time-sync.
    The AABB for pairs would be the next bit but I'm not across the code or algorithm so I'm just spit-balling. Is there potential for skipping some bits of the broadphase on some iterations, or is it going to drop accuracy / introduce misses?
     
  23. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    @PhilSA I updated this for 2018.3.0b7 and latest Burst, ECS and Mathematics packages and moved ECSPhysics itself into package manager package here:
    https://github.com/0lento/ECSPhysics/tree/package

    I didn't PR this yet as I'm still wondering what would be the best folder structure for this. What I put there definitely works if you just clone it for the whole project but it doesn't let you use the new 2018.3.0b7 package manager functionality to just add the git link into your manifest for ECSPhysics as that only works if the package is at the github branch root. See more info on the git linking here: https://forum.unity.com/threads/git-support-on-package-manager.573673/#post-3819487

    In theory, for packages it would be the best if the ECS sample and actual physics implementation were on separate git repos because then the sample project could just ref to the ECSPhysics on it's packages/manifest.json like:
    but this would then fail for people who don't have git setup for their computers (as git functionality on package manager depends on it).

    You'll probably think next that why can't the example itself be on subfolder under the ECSPhysics package.. There's one issue with this, you can't open scene files that are inside packages, so this would prevent people from running the example itself. One option would be to embed the samples into old .unitypackage and let people expand it to their projects if they need it but that pretty hacky and not that nice to maintain.

    If nobody cares about being able to fetch the ECS Physics simply by putting that one line into package.json, the structure I've now put on my custom package-branch does work and it does keep the library separate from the example. On my branch now, actual physics lib is now located at Packages/com.philsa.ecs-physics. Any thoughts on this?
     
    Last edited: Oct 30, 2018
  24. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
  25. sterlingcrispin

    sterlingcrispin

    Joined:
    Oct 23, 2013
    Posts:
    12
    Hey this is awesome thanks for posting this. I've got a C++ project I want to port to ECS but I had bullet physics for that and was curious if there were any options out there, Im glad to have found this! https://twitter.com/i/moments/782734407996104704 What do you think the ETA is on joints? I want to try to connect a bunch of rigid bodies together with soft constraints so they act like a cloth to produce leaves for a plant simulator https://twitter.com/sterlingcrispin/status/772530367857176576
     
  26. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    @sterlingcrispin I don't think PhilSA's had much time to work on this lately, he hasn't even replied to my PR from 2 weeks ago which just implies he's busy with other things (also pointed out in this message: https://forum.unity.com/threads/sources-available-physics-in-pure-ecs.531716/page-3#post-3644221)

    But if you only look at his roadmap: https://trello.com/b/XcYxQFrh/ecsphysics, joints are not in immediate future of this project (unless someone else picks it up and keeps developing it forward).
     
  27. PhilSA

    PhilSA

    Joined:
    Jul 11, 2013
    Posts:
    1,080
    yeah sorry guys, I've been extremely busy for a while and will be completely absent from the internet for at least the next 3 weeks. I'll see about picking the project back up after that
     
    TZ- and slimshader like this.
  28. nensanders

    nensanders

    Joined:
    Aug 4, 2017
    Posts:
    2
    Hi PhilSA and ECS community,
    thanks for sharing this nice library.
    I played a little bit with the GravitySystem.
     
  29. avvie

    avvie

    Joined:
    Jan 26, 2014
    Posts:
    72
    @nensanders could you share your project? i want to see if orbits would work
     
  30. nensanders

    nensanders

    Joined:
    Aug 4, 2017
    Posts:
    2
    Sure @avvie
    https://github.com/nensanders/ECSPhysics/tree/planet-gravity
    Orbits shouldn't be a problem

    Code (CSharp):
    1.  
    2.                 // UniformGravitySystem.cs
    3.                 double3 aPos = new double3(0f, 0f, 0f);  // Planet
    4.                 double3 bPos = p.Value;                  // Vessel
    5.  
    6.                 double3 fromAToBVector = aPos - bPos;
    7.  
    8.                 double vesselInvMass = r.InverseMass;
    9.                 double distance = math.lengthsq(fromAToBVector);
    10.  
    11.                 double comp1 = 0.006674 / distance;  // G
    12.                 double comp2 = vesselInvMass/75000000.0; // Planet Mass
    13.  
    14.                 double resultingForce = comp1 / comp2;
    15.                 fromAToBVector = math.normalize(fromAToBVector) * resultingForce;
    16.                 v.Value += fromAToBVector * DeltaTime;
    17.  
    18.                 Velocity[index] = v;
    19.  
     
    starikcetin, avvie and Flurgle like this.
  31. JoNax97

    JoNax97

    Joined:
    Feb 4, 2016
    Posts:
    19
    Hi @PhilSA
    First of all, congratulations on your wonderful work!
    I'm currently looking to implement some simple fluid simulation for my game, but so far available solutions are either not performant (eg. Obi fluid) or not interactive (eg. Nvidia flex). I'm wondering a few things:
    - How difficult would it be to use this as a base for an approximate liquid sim algorithm?
    - Can elements from your system be raycasted and interacted with?
    - Is this completely decoupled from rendering, so I could use a metaball/raymarching shader to render particles as a cohesive fluid?

    PS: Sorry I'm asking about things that I could try personally, but I've never used ECS and I can't afford to learn it right now as I'm still investigating different pathways.
     
    Last edited: Dec 26, 2018
    Flurgle likes this.
  32. melian

    melian

    Joined:
    Dec 29, 2012
    Posts:
    5
    Hi @PhilSA and watchers of this thread,

    Thank you for your informative and inspiring contributions to the mergence of ECS with physics.

    I have run into a problem though. When I checked the original Github project shared, ECS scene was so slow as can be seen in the screenshot below. Process took around 50ms as opposed to a 7ms mentioned a few posts earlier by @PhilSA .

    upload_2019-1-4_15-11-36.png

    And PhysX scene took around 20ms.

    upload_2019-1-4_15-13-55.png

    My hardware specs are Intel i7-7700hq and GeForce GTX1060.

    I wondered whether there are others that get slow performance as mine. In this case, ECS does not seem to be giving much of a performance boost wrt PhysX.

    By the way, I just started to search ways to improve the performance of my academic project which requires interaction of thousands of rigidbodies, so will often be checking this thread for updates.
    Thanks again for your hard work and generosity :)
     
    deus0 likes this.
  33. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    @melian you actually have Burst enabled? Also the point here isn't to compete with PhysX on brute force speed but to have fully ECS driven physics so you don't have to use hybrid approach for physics.

    edit: if you benchmark in editor (you really should test in builds rather) make sure you've disabled native containers leak detection and burst safety checks as they can make things slow down a lot.
     
    Last edited: Jan 4, 2019
  34. melian

    melian

    Joined:
    Dec 29, 2012
    Posts:
    5
    Hi rizu, yeap Burst is enabled.
    Sorry for my hasty approach. I was not commenting on the aim of this successful work but just for the results I got from the project. Performance is generally the most sought after attribute however building something from scratch for a norm like ECS is by far the most exciting part of course ;)

    Edit: I will look into your suggestions, thanks.
     
    Last edited: Jan 4, 2019
  35. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    @melian, I see you posted about the same time as I appended the reply, see my edit :)
     
  36. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    4,102
    Can you profile a build, rather than the editor?
     
  37. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    4,102
    Rewaken, RaL, S_Darkwell and 3 others like this.
  38. melian

    melian

    Joined:
    Dec 29, 2012
    Posts:
    5
    Thanks @rizu and @Peter77 for the suggestions.
    I tried various combinations to observe performance changes; build, development build with autoconnect profiler and on editor all with disabled leak detection and burst safety checks.
    Editor gave the most performant results for ECS scene surprisingly, process time decreased from 50ms to 27ms, fps increased from 3.0 to 10.0.
    However, ECS scene build was so much slower than editor :confused: fps dropped from 3.0 to 0.5.
    Disabling Leak detection and Burst safety checks provided the best performance gain.

    Edit: I guess it is related to an issue with Burst compiler not working for the builds. I will have a look at it.
     
    Last edited: Jan 6, 2019
  39. starikcetin

    starikcetin

    Joined:
    Dec 7, 2017
    Posts:
    245
    That sounds wrong. The build should be more performant than the editor.
     
    hippocoder and Antypodish like this.
  40. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,562
    Sounds like hidden error spam (tm) or some other bad issue. For a release build, it should never be slower.
     
  41. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,638
    Testing on different hardware, would help diagnostic issue. If possible of course.
     
  42. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,174
    I'd check the logs too for errors or warnings. Also build target has to have Burst support for it to work of course (I don't remember which all platforms are supported atm).

    @melian Can you try my https://github.com/0lento/ECSPhysics/tree/master branch for this? It's using newer packages and is updated to 2018.3 (2018.3.0f2 should work fine with this). Besides minor API changes, newer Entities packages don't need separate incremental compiler anymore (that incremental compiler package was deprecated as 2018.3 comes with better compiler out of the box now).
     
    Arkade likes this.
  43. melian

    melian

    Joined:
    Dec 29, 2012
    Posts:
    5
    Just installed missing c++ tools for visual studio. @rizu , Burst worked in desktop build for your version. Thanks.
    Now the desktop build worked faster (fps ~30) than editor mode (fps ~10).
     
  44. Arkade

    Arkade

    Joined:
    Oct 11, 2012
    Posts:
    577
    Hey, I've not used Burst (or this package) yet but how did you know you need a an external dependency? (I haven't seen anything about this prerequisite)
     
  45. melian

    melian

    Joined:
    Dec 29, 2012
    Posts:
    5
    I searched forum for the reason Burst not working with the build (Jobs did not show Burst in the autoconnect profiler). Some others solved this problem by updating their visual studio according to the errors shown during build generation. I am not entirely sure why this happened, I had downloaded VS from Unity Hub.
    Anyone with an explanatory answer is most welcome.
     
  46. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,079
    Philippe, With this project, you got the attention of the Unity team. Your primary code will impact strongly in the world community as well as J. Ante ECS will do. I wish to improve it but looks like you are not accepting pull requests.

    I like this simple code optimisation made up in an interview and I wish to share it with you.
    Maybe this video is too simple for you but I hope will inspire you to keep going.

     
    Last edited: Feb 2, 2019
    Flurgle, PBaille and deus0 like this.
  47. Reborn1214

    Reborn1214

    Joined:
    Jan 19, 2019
    Posts:
    15
    If I want to delete an already occurring collision, what should I do? Get the CollisionManifoldsArray and add deletion tags? At what time should I delete it?
     
  48. deus0

    deus0

    Joined:
    May 12, 2015
    Posts:
    17
    Has anyone used Unity's new ECS physics package yet?
     
  49. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,638
    deus0 likes this.