Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

Experimental build with the PhysX 3.4.1 upgrade

Discussion in 'Physics Previews' started by yant, May 7, 2018.

  1. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    Hi,

    We're happy to open access to the experimental preview build of Unity 2018.2 with the PhysX 3.4.1 upgrade. It's the latest PhysX release to date.

    ------
    EDIT: Here is a download link to Mac & Windows editors https://beta.unity3d.com/download/9193bd6b07b9/public_download.html. Updated to Unity 2018.2beta6, and has these new features:

    - creation order independent simulation (‘enhanced determinism’ flag)
    - negative mesh scaling no longer requires baking (!!)
    - more friction types (one directional, two direction, patch)
    - terrain collisions are processed the same way as mesh collisions (‘unified heightmaps’)
    ------

    PhysX 3.4 comes to replace PhysX 3.3 that is currently available in the mainline build. It's not a major upgrade like 2.8 to 3.3 was, but is still a result of a few years of evolution in areas like performance and accuracy. That said, most of the changes are more or less internal, and not that many new features were added (and even less are going to be visible through the Unity integration initially), but the good news is that not that much of the old stuff was removed either. We learned from the past that the latter caused major pain and we're trying to address that by rolling out an upgrade that has as few regressions as possible, compared to a release that is full of all sorts of new bells and whistles exposed at the cost of less attention to the currently used feature set. We're also going to test it for longer than before. Assuming the feedback is good overall, we're releasing the upgrade with 2018.3 at earliest. Note that the final 2018.2 will still be using PhysX 3.3 as our PhysX 3.4 is not tested enough yet.

    There will be a full upgrade guide available just a bit later (based on your feedback), but here is a quick outline of the changes.

    - there is a massive boost in all the physics queries. Raycasting, shape sweeping should be up to 2x faster than before. The accuracy of tests against thin, long, and otherwise tricky objects for physics are greatly improved.
    - mesh cooking is up to 10x faster, including a better convex hull computation algorithm (no mesh inflation needed, we deprecated that flag)
    - broadphase & solver are now much better split in jobs, and should saturate the cores noticeably better as a result
    - there is a new speculative continuous collision detection mode available (rotational CCD anyone?)

    Can't wait to see what you think.

    Anthony
     
    Last edited: May 31, 2018
  2. strich

    strich

    Joined:
    Aug 14, 2012
    Posts:
    375
    This is awesome! FYI for others, here are the official v3.4 PhysX release notes.

    @yant my primary concern is whether you have integrated all the Vehicle/Wheel changes listed in there? The current implementation of wheel physics are very approximated and inaccurate for low-speed sim games. Some of the features in v3.4 such as PxVehicleSuspensionSweeps seem to greatly improve collision response on more complex surfaces. Is this something we can expect to see in this release?
     
    ZiadJ and Deeeds like this.
  3. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    Regarding the release notes, the ones you linked seem to list January'17 updates. It's surely been a ton of changes since then. I couldn't however find the release notes anywhere public to link to. This build has PhysX as of late April'18.

    Vehicles-wise, I'm still considering options. As mentioned above, we want to ship a release that's probably thin of new features but the old functionality works reasonably well. I'm looking for bad regressions in this thread mainly.

    However, there is no clear cut regarding what ends up shipped in 2018.3 at the moment. Initially, I was planning to include nothing new at all but then realised we needed to include the new CCD modes. Then, we're looking into exposing more friction modes right now. We're going to rewrite our transform readback sequence too, for faster updates.
     
    pixelshader1 and cxode like this.
  4. JakubSmaga

    JakubSmaga

    Joined:
    Aug 5, 2015
    Posts:
    417
    +1 For the Vehicle/Wheel changes.
     
  5. cxode

    cxode

    Joined:
    Jun 7, 2017
    Posts:
    268
    Holy crap, thank you, thank you, thank you so much!!!!! I was resigned to waiting for "2018.3 or later", which is what was previously stated for the timeline of PhysX 3.4.

    My game needs 3.4. It's a game where you can build things, and currently there's a hard limit on the complexity of what you can build due to the hard PhysX limit of 262143 active colliders. Combining colliders is not an option for me as each one needs to be individually identified via raycast. I am ecstatic right now that I'll be able to remove the limit in my game's next release! Will you continue to provide early builds with PhysX 3.4 for the latest Unity version until it's fully integrated?

    Also, this is just my opinion, but I think you shouldn't be shy of introducing new features and deprecating old ones. If people want support for old features, well, that's what the LTS stream is for.
     
    FROS7 likes this.
  6. strich

    strich

    Joined:
    Aug 14, 2012
    Posts:
    375
    Understood. Obviously take whatever integration course fits best for the release schedule.

    If you have the spare cycles it might be nice to see a different experimental beta with those features added that we can play with, with the view that it'll be a 2019.x thing.
     
  7. RisingMos

    RisingMos

    Joined:
    Aug 28, 2015
    Posts:
    17
    Awesome! I was waiting for this since the preview of physx 3.4 dropped. I will start testing with massive amounts of RBs and report results.

    It would be great if we can get GPU Physx as an option, even though it is platform specific.
     
  8. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    We will post a new build here with the bug fixes and changes, though I'm not sure when/if we will be shipping players as well. Note that currently it's just about Editor binaries. I imagine creating commercial games with this would be a stretch really.
     
  9. cxode

    cxode

    Joined:
    Jun 7, 2017
    Posts:
    268
    @yant so 3.4 works in the editor, but you cannot create builds with it?
     
  10. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    That's correct. The intention is testing and not derailing the users from the mainline build.
     
    cxode and FROS7 like this.
  11. cxode

    cxode

    Joined:
    Jun 7, 2017
    Posts:
    268
    Aw man. Thanks for the information.
     
  12. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,365
    Recent sdk, no breaking change, loving this.
    Are there accelerated world queries like "closest object" in 3.4?
     
    TooManySugar likes this.
  13. AdamMGardner

    AdamMGardner

    Joined:
    Sep 26, 2017
    Posts:
    2
    Yeah, the PhysX-GRB would be spectacularly useful even though it is specific to boxes.
     
  14. LordTocs

    LordTocs

    Joined:
    Apr 29, 2018
    Posts:
    1
    Cool stuff. Any chance we could see PxArticulation exposed in the future?
     
    tjmaul likes this.
  15. Zergling103

    Zergling103

    Joined:
    Aug 16, 2011
    Posts:
    392
  16. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    We'll be adding articulations just a bit later, and the currently used solver is still the good old iterational one. There's been lots of changes/fixes here and there (read the PhysX' RNs for more details) so I encourage you to download the beta build and try if there's been anything new regarding your particular use cases. Thanks.
     
    Cynicat, cxode and hippocoder like this.
  17. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    VERY happy with this news!
    Very nice is that with V-HACD, or just single objects for now?
     
  18. eobet

    eobet

    Joined:
    May 2, 2014
    Posts:
    176
    Is this going to be a package one can install and test at any point?
     
  19. cxode

    cxode

    Joined:
    Jun 7, 2017
    Posts:
    268
    Unlikely, @eobet. It's quite a fundamental part of the engine, I highly doubt it could be made hot-swappable in the way packages are.
     
    hippocoder likes this.
  20. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    We won't be able to make it into a package at this point in time. However, if/once we have it completely re-written in C# with little to no native code - there would be no more obstacles I guess.
     
    tapawafo and cxode like this.
  21. pavelkouril

    pavelkouril

    Joined:
    Jul 22, 2016
    Posts:
    129
    Here's hoping it means exposing the cooking functions so we can cook the mesh colliders ourselves outside of a main thread asynchronously to prevent framedrops in the game.

    Sure, the speedup (up to 10x may be nice; will do some perf benchmarks* in the upcoming weeks I guess), but being able to offload it to another thread completly would be superb. :)

    *) Shame this experimental build cannot create standalone players tho, I'd prefer profiling the game outside of Editor. :)
     
    cxode likes this.
  22. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    I hear your threading ideas. We have something in the backlog that will help. Unfortunately, despite the fact that PhysX is multi-threaded itself, the overall design of our current integration is mostly single-threaded when it comes to C# API simply because the scripts were running on the main thread until recently. For that particular reason, all the job-based stuff feels fairly alien currently, not a first class citizen definitely (see RaycastCommand for instance).

    The long-term vision is, of course, to create a new generation physics API that is built around the concept of jobs and ECS but that won't happen this year I'm afraid.
     
    TooManySugar and cxode like this.
  23. AHGS_Jens

    AHGS_Jens

    Joined:
    Nov 16, 2012
    Posts:
    1
    I was hoping this would fix the suspension issue on the Wheel Collider, but apparently it is still there. The issue is that if the suspension ever get compressed to the very bottom it bounces back up in an unnatural fashion instead of just remaining fully compressed. Is this something that will be fixed or can it be mitigated somehow? (besides avoid applying too much force on the suspension).
     
  24. pavelkouril

    pavelkouril

    Joined:
    Jul 22, 2016
    Posts:
    129
    What about a checkbox on a Mesh Collider, that would turn the asynchronous cooking on/off (on Unity's side of the engine), instead through a C# API? I reckon this could bring some nice performance improvements as well and be really really easy to use for the end user? Or does this go directly against the future plans you already have?
     
    cxode likes this.
  25. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    There are plenty of approaches like that, but the core problem is that there is no clear (or performant) way of waiting on the cooking to be finished. To make it worse, some applications generate MeshColliders on the fly, insert into the scene and raycast against them all in the same frame. This checkbox approach wouldn't work for them I reckon.

    Actually, what I had in mind is there is a really nice, jobified API for cooking meshes that is used on the AR targets like Tango. It cooks meshes every frame as they get generated from the sensors. It's currently only available in the native code, but I see no obstacles in making that available in C# eventually. That would enable you guys creating a chain of jobs that represent the whole pipeline of mesh processing from generation to queries and collision response. Would also work with the colliders persisted in the scene itself during the scene load.
     
    cxode and hippocoder like this.
  26. hippocoder

    hippocoder

    Digital Ape

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    That's really powerful but in the near term being able to turn mesh colliders into several convex mesh colliders would be a positive improvement across our entire project. Is that coming too or did plans change and we should figure out a 3rd party option? (thanks for any reply).
     
  27. pavelkouril

    pavelkouril

    Joined:
    Jul 22, 2016
    Posts:
    129
    Personally, I wouldn't expect the checkbox to be checked in scenarios where you need to immediately work with the changed collider. However, it would be IMHO suitable for usecases like loading new parts of a terrain in distance (where you don't need to do any raycasts/collisions yet) and not wanting to use main thread time to cook colliders before you start to spawn objects in the game world.

    As for checking (at in userland C# code), I guess a simple boolean flag on the component (IsCooking) would be enough? Then the user can in his update loop check if the component he's waiting for to cook + integrate is still cooking or not.

    Also, being able to switch the asynchronous cooking on/off during runtime would be really important, so colliders around the player are always recomputed synchronously and just the unimportant stuff that can wait can be done asynchronously. :)

    However, if your goal is to move as many people to use the job system, then this would probably be counterproductive, I guess.

    Well, this certainly sounds nice, but ... what is eventually? ;) Any performance improvements can't come soon enough! :)
     
  28. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    VHACD-style stuff is still expected to be integrated at some point in the future, but I'm afraid I'm already derailing the upgrade testing thread so much by answering all sorts of future features requests questions here. ;-)
     
    cxode, hippocoder and pavelkouril like this.
  29. RisingMos

    RisingMos

    Joined:
    Aug 28, 2015
    Posts:
    17
    So far, I am liking this build. It fixed collision detection issues (objects don't collide!) that crept in 2017.2 onward that prevented . Overall, I feel objects are more stable and there is a noticeable performance improvements.
     
    Last edited: May 22, 2018
    cxode likes this.
  30. pavelkouril

    pavelkouril

    Joined:
    Jul 22, 2016
    Posts:
    129
    Tested 2018.2.0b4 and the PhysX experimental build (2018.2.0b3); while baking seemed to be faster, the sweeping OverlapBoxNonAlloc seems to take more time than before (especially during the startup of the game, when spawning a lot of stuff, when doing tens of thousands of these calls).

    Is anyone else experiencing worse performance with the sweeps?

    (However, these are just initial findings; I'll try to do proper testing later, both at the start of the game and after a while of playing, ideally with more samples.)
     
  31. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
  32. kstorey

    kstorey

    Joined:
    May 21, 2018
    Posts:
    6
    PhysX-GRB is not restricted to just boxes. The demo at GDC 2016 used a lot of boxes because it was all programmer art. PhysX-GRB supports every feature of the PhysX SDK besides articulations. You do need to cook convex and mesh assets with additional data to accelerate those on GPU, otherwise contact gen for those shapes will be processed on the CPU.
     
    yant likes this.
  33. funkyCoty

    funkyCoty

    Joined:
    May 22, 2018
    Posts:
    727
    Thanks for this beta build! About to try it out.

    Jobs api sounds great. However, I would love the option to simply bake the mesh collider manually off thread somehow. If this is easy to implement on your end, this would make our lives a thousand time easier.
     
  34. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    @yant Can you tell if there's any plans for exposing Physx Contact Modification callback with PhysX 3.4.1? I've asked this before but haven't really seen any response. I know the physx api is kinda messy on this part, so I could suggest some structures that could streamline it for unity integration purposes if it helps anything.
     
  35. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    As I mentioned in the first post, the intention is to break as little as possible (fewer regressions than before). We won't include too many of the new features for that reason too. However, I'm interested to see your contact modification ideas, could you elaborate please?
     
  36. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    Additionally, I'd like to see "Enhanced Determinism" scene setting exposed, it's just optional flag you set and afaik then physx will internally preserve the order or rigidbodies for internal solver even if you remove or add rigidbodies to the same scene. This feature is super trivial to implement, it really just requires you to set the flag on scene creation and that's it. You'd also definitely want to keep it optional as it involves some CPU overhead so not everyone would want to enable it.

    I understand that the initial release will be more close the old setup to keep things simpler and not having to chase bugs created by the new systems. But I'd love to see more physx feats exposed later on if possible. Contact Modifications are something I've implemented on UE4 myself and I think it's pretty fundamental thing to have access to for any serious physics driven game. Like the Enhanced Determinism, it's not something that requires you to change existing features at all, it's addition to them but in this case, the implementation will require a lot more work than just the scene flag.

    Default PhysX API for Contact Modifications is kinda messy as they have separate functions for each operation and collision pair. This could be simplified to make it more user friendly and to act more like current collision events do so it wouldn't feel so alien to users. I'll draft something up and send a message.
     
  37. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    Look, I think I can try to expose the enhanced determinism in a build I was about to share later this week. As for the contact modifications, I think there are two major problems -- 1) those can potentially be processed off the main thread and it doesn't go friends with our scripting 2) the whole idea of calling script per contact pair is unscalable I think. For reading contacts, I think we'll be returning an iteratable array of contacts directly, with little to no processing on top of what PhysX offers. This will be a little more verbose than via OnContactXXX as right now, but hey - you get zero GC allocations that way... Thinking that pays the price, would you agree?
     
  38. rz_0lento

    rz_0lento

    Joined:
    Oct 8, 2013
    Posts:
    2,361
    Thank you :)
    Usually when people do these, they only need it for few objects so the cost isn't that huge. Think of a racing game where you only need to make sure the cars themselves handle collisions gracefully, this usually means 1-30 colliders that would be routed to get the events. It doesn't scale nicely for larger approaches but I don't think the contact modification itself does either, it's supposed to be use used to handle edge cases that you couldn't handle otherwise.

    All that being said, just exposing just the array of contacts to modify in single global event + related functions is probably easiest and it still gets the job done. Many would be really happy if Unity can make this happen, the feature request for it is 126 votes atm which is pretty good, considering how little physics feats get voted in general. Also, people who really need this can handle it regardless how it's exposed as long it's there in some form (it's not exactly a feat beginners would use).
     
    goncalo-vasconcelos and Invertex like this.
  39. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    Thanks for your input.
     
    cxode likes this.
  40. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    I've just updated the download link with a Unity 2018.2beta 6 based build. Here are a few more notes to what's already outlined above.

    Exposed creation order independent simulation (‘enhanced determinism’ flag)

    Asked by many people, and by @rizu in this thread in particular. Try it out, would love to hear your feedback.

    Negative mesh scaling no longer requires baking (!!)

    In Unity 4, every scale change to a MeshCollider caused producing a new copy of the original mesh where the scaling was baked into the vertices themselves. In Unity 5, we reduced that to only negative and non-uniform scale to cause that. With PhysX 3.4, we can allow both negative and positive scale not to cause baking. However, non-uniform scale still has to be baked in for now.

    Exposed more friction types (one directional, two directional, patch)

    Again, asked by many people over the last couple of years. Exposed all the options PhysX has to offer currently. The default is still patch friction, as before.

    Terrain collisions are processed the same way as mesh collisions (‘unified heightmaps’)

    This affects when something overlaps with terrain and bounces back. Priorly, the geometric tests involved a special query to fetch triangles from nearby heightmap cells and then a custom step to compute the feedback. This had a number of downsides such as inaccurate response, wrong collision normals and poor performance among others. A new approach, called 'unified heightmaps' in PhysX-speak approaches this differently. The idea is just to use our usual mesh collision algorithm. In that case, a terrain collision response is no different than a mesh collision response. In addition, that gives us great robustness and accuracy. However, a downside is, as with normal meshes, the tunneling effect may appear. It's recommended to enable CCD on fast moving objects that hit terrain for this reason. The old collision method used something called thickness https://docs.unity3d.com/ScriptReference/TerrainData-thickness.html, where an object below the heightmap would still be treated as overlapping the terrain until it's deeper than terrain thickness. I'm interested to learn if we could deprecate thickness in 2018.3.

    Thanks for trying the build out.
     
  41. strich

    strich

    Joined:
    Aug 14, 2012
    Posts:
    375
    Do you know if this'll affect the performance/response of runtime terrain heightfield modifications?
     
  42. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    What about caves/basements/underground bunkers or holes in the terrain, surely a thickness setting would prevent underground structures physics being impacted by terrain physics?
     
  43. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    It's a pretty good question I would trust you to discover an answer to ;-) I don't assume that changing the heightmap would magically become faster or slower because it's just the collision response algorithm that's changed.
     
  44. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    The unified heightmaps (in the build I shared) give you a thin surface. Only the objects sunken a small portion would pop up. In contrast, the old heightmaps made it so that anything that's deeper than 'thickness' meters below the surface never pop us. Everything else pops up. The default thinkness was 1 meter and I'm not sure many devs used that at all. Correct me if I'm wrong please.
     
    cxode likes this.
  45. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    Just to make it clear - thickness is not supported with this new mesh-like collision response. Anyone willing to use thickness would have to opt for the old terrain code.
     
    cxode likes this.
  46. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,620
    When duplicating GameObject's in the Hierarchy, the editor sometimes messes with the Scale value. For example, duplicating a GameObject or just changing its parent might change the Scale value from 1 to 1.000001 or 0.999999.

    If the Scale is x=1, y=1.0000001, z=0.99999999 due to this Unity (editor) bug, is the Collider considered non-uniform and therefore has a negative impact on performance?
     
    AlanMattano likes this.
  47. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    no. I think by non-uniform I somehow meant stuff like skew transform. Having different x,y,z on its own should be fine
     
    cxode likes this.
  48. nikescar

    nikescar

    Joined:
    Nov 16, 2011
    Posts:
    165
    It's always exciting to see new physics stuff coming especially with those kinds of performance improvements!

    I was wondering if 3.4 and Unity's integration will fix the Linear Limit and Angular Limit bugs on the ConfiguableJoints I've mentioned to you in the past?

    Thank you!
     
  49. Cynicat

    Cynicat

    Joined:
    Jun 12, 2013
    Posts:
    290
    It would make my life way easier if we got articulations implemented. Their basically required for VR physics to not be hell to work with. basically anything with multiple joints isn't going to be stable in VR. =<. its super cool to see the new version though!
     
    tapawafo likes this.
  50. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    596
    Will make articulations availble in 19.1 or right afer that, stay tuned, that's coming.
     
    web76, tjmaul, SuppleTeets and 4 others like this.