Search Unity

  1. How has 2019.2 and the beta been for you so far? Give us feedback in this thread.
    Dismiss Notice

Garbage Collector API

Discussion in '2018.3 Beta' started by hippocoder, Sep 27, 2018.

  1. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    • Scripting: Added UnityEngine.Scripting.GarbageCollector API for enabling and disabling the garbage collector on Mono and IL2CPP scripting backends.
    Just wondering the use cases and rationale for it, and if it's leading somewhere...
     
    Razmot likes this.
  2. LennartJohansen

    LennartJohansen

    Joined:
    Dec 1, 2014
    Posts:
    2,237
    I think it could be usefull if you can disable the cleanup and force it when it is good for you. In many cases you allocate memory on the heap that you do not plan to free up, but the cleanup still triggers, it does not know that it still is used.
     
    hippocoder likes this.
  3. lukaszunity

    lukaszunity

    Unity Technologies

    Joined:
    Jun 11, 2014
    Posts:
    411
    Being able to control the garbage collector (GC) in some way has been requested by many customers. Until now there has been no way to avoid CPU spikes when the GC decided to run. This API allows you disable the GC and avoid the CPU spikes, but at the cost of managing memory more carefully yourself. I've copy-pasted the API docs and manual entry for this API at the bottom to provide more details.

    I'm not quite sure what you mean with it leading anywhere, but this isn't the beginning of any work to remove the GC or otherwise make fundamental changes to how you make games in Unity with MonoBehaviours. This is an on/off switch for the GC to avoid CPU spikes.

    If you are interested in how to make high performance games, then the new Entity Component System betas provide a new alternative way of making games in Unity.

    You could argue that the current GC behavior does the cleanup when it thinks it is good for you :) With this new API you can disable the GC and when YOU think it is a good idea to free up memory in your game, then you can enable the GC, call GC.Collect() and then disable it again.

    Current API docs:

    Current Manual docs
    Will be added to https://docs.unity3d.com/Manual/UnderstandingAutomaticMemoryManagement.html

     
  4. mh114

    mh114

    Joined:
    Nov 17, 2013
    Posts:
    276
    Any chance of this getting backported into 2017.4 LTS?
     
  5. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,231
    "Not supported on WebGL platform and .NET scripting backend." What is ".NET scripting backend" ? I thought there's just 2 backends that is mono and IL2CPP :
    • Scripting: Added UnityEngine.Scripting.GarbageCollector API for enabling and disabling the garbage collector on Mono and IL2CPP scripting backends.
     
  6. lukaszunity

    lukaszunity

    Unity Technologies

    Joined:
    Jun 11, 2014
    Posts:
    411
    Since this is a new feature, there will be no backport to 2017.4 LTS.

    https://unity3d.com/unity/qa/lts-releases
     
  7. lukaszunity

    lukaszunity

    Unity Technologies

    Joined:
    Jun 11, 2014
    Posts:
    411
    When you target Universal Windows Platform (UWP), you can use the .NET Scripting Backend:

    https://docs.unity3d.com/Manual/windowsstore-dotnet.html
     
    5argon likes this.
  8. mh114

    mh114

    Joined:
    Nov 17, 2013
    Posts:
    276
  9. LennartJohansen

    LennartJohansen

    Joined:
    Dec 1, 2014
    Posts:
    2,237
    This is great.

    Lennart
     
  10. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    Seems to me providing Unity can clean up it's own allocations such as when you use OnCollisionStay contacts, I could freely use this new feature without ever needing a new garbage collector.
     
    Ryiah likes this.
  11. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    79
    Lets break normal expected behavior. Instead of having a GC optimized for many short lived objects lets have a no GC at all. Memory is cheap let it leak. After all everyone is already using object pooling why not script pooling? But lets not use c++ instead lets create a strange beast I will call it unity scripto pattern.
     
    Last edited: Sep 28, 2018
    Sebsc likes this.
  12. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    3,935
    This is an thing you can do, not something that's on automatically. It also in no way implies that work has stopped on a generational GC.

    So what on earth are you angry about? They made a feature that's not useful for you?
     
  13. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    79
    As if Unity will give you so much control when this is called. Lets have one big leaky spike rather then many small ones. Lets just manage memory ourselves but without any of the normal tools. Really this makes no sense unless you feel like making hacky C# scripto.

    The question you need to ask yourself why all this trouble instead of using c++;

    The use case here is not setting when to call the GC. No it is lets just turn the GC off and leak, great idea bro.
     
    Last edited: Sep 28, 2018
  14. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    Use C++ then? just link to it in Unity... bro.
     
  15. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,217
    Be very very careful with this:) I've been playing around with custom collectors in .net core (http://tooslowexception.com/zero-garbage-collector-for-net-core-2-1-and-asp-net-core-2-1/), and it's really easy to crash your machine if you don't have a very good handle on everything that is allocating memory. Whatever you think your object allocation is, it's more. Plus I hit numerous times where I had an unexpected exception or something cause an alternative code path I hadn't thought about, resulting in a pattern that took just a couple of minutes to kill my computer.
     
    phobos2077 likes this.
  16. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    Yep, I'm not planning on touching this for the live game. I will probably use it for editor tools though.
     
  17. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,217
    In practice there really wouldn't be much difference between the two with Boehm. It's not like a generational collector where you can fine tune the behavior (never mind that .net runtimes don't expose the knobs to do that, but they are there).
     
  18. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    5,671
    It does make sense. We use C#. It's garbage collected, that's not going to change. But being able to manually call when it collects is much better than letting it do it whenever it wants. If we're going to drop frames because of the GC, being able to control when it happens can result in a much better experience.

    I believe the INSIDE had manual GC collection as well. I believe they only called it when you died. They were able to do it this way because they had source access, but now we can do it too.
     
  19. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    There are very few built in Unity features that allocate garbage that remain. I can entirely avoid GC regardless if only things like Physics didn't ever allocate on my side...
     
  20. Deeeds

    Deeeds

    Joined:
    Mar 15, 2018
    Posts:
    739
    Only if you know how. And getting to know how to avoid garbage creation isn't an easy path, nor short.
     
    hippocoder and phobos2077 like this.
  21. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    Yep but if Unity eliminate it from their oldest and most-used API calls then it might be relief much sooner :(
     
  22. xVergilx

    xVergilx

    Joined:
    Dec 22, 2014
    Posts:
    1,735
    I'd imagine that would upset a lot of people that cannot live without OnCollisionX events, or Physics.RaycastAll calls etc.

    Hope to see someday that change though.
     
  23. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    Well I just want similar API to the 2D collision system or just Stay without allocs, where I can query current contacts if I wish, it means I can run my own OnCollision without garbage, and just use the Enter callback to wake up.

    It's sliding slightly off topic now but I'll ping @yant just in case he has had a good day and wants to reply regarding allocations.

    I imagine that turning off the GC and having OnCollisionStay running will lead to a lot of crashing on different devices.
     
  24. Deeeds

    Deeeds

    Joined:
    Mar 15, 2018
    Posts:
    739
    There's the perfect example. I had no idea that OnCollisionStay makes a tonne of garbage.
     
  25. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    If you return without accessing the input parameter, you can avoid the allocations, so you only get a performance penalty equivalent to SendMessage. If you access the Collision parameter that's optionally passed in - usually to gain access to contacts so you know where something hit, or the normal of the hit, you will feed the GC monster.

    So in most cases you will feed the GC monster. I can do something about this by tracking it manually on Enter and using ComputePenetration, ClosestPoint to compensate but that is something a) most people can't work out well and b) doesn't really help perf much despite avoiding GC. It's just swapping one problem for another due to a design that's over a decade old.
     
  26. Deeeds

    Deeeds

    Joined:
    Mar 15, 2018
    Posts:
    739
    There's two features of my game design that absolutely rely on that data being passed in, for little chunks of time... and I'd wrongly presumed these were garbage-less like other things from particles and physics2D, and based on other things I'd read about accessing contacts being garbage free.

    I do this:

    other.contacts[0]. access stuff here....

    because I know it's the first contact, and because I thought this way was garbage free...
     
  27. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    If I am wrong it's probably a change unity did between my last project and this one. But profiler will let you know.
     
  28. Prodigga

    Prodigga

    Joined:
    Apr 13, 2011
    Posts:
    722
    Excellent! We are currently using Jackson Dunstan's method to disable and enable the garbage collector.

    Very happy to hear that we don't have to do that any more!

    We keep allocations as close to 0 as possible, but they do happen, and the spikes are nasty. So we disable the GC, collecting manually in loading screens, and only re-enable the GC if the memory usage starts to climb past some threshold to ensure we don't eat up all the available memory on the device. No GC collections during gameplay, no unexpected stutters.
     
    Last edited: Sep 30, 2018
    Deeeds and Peter77 like this.
  29. Deeeds

    Deeeds

    Joined:
    Mar 15, 2018
    Posts:
    739
  30. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    473
    @lukaszunity Any news about migration to SGen? I know that the native side of Unity requires access to the managed memory for some stuff and that's why a moving GC is still not here. But some Unity guys (@xoofx) knows a thing or two about Immix. Isn't it a better alternative to Boehm? It solves many problems of conservative GC in theory, but who knows how it works in practice. I would like to give it a try if any mature implementation exists.
     
    Last edited: Oct 1, 2018
  31. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,563
    We did a lot of work recently on making sure that access to managed memory always goes through APIs notifying the GC of the changes, which now puts us into a position where updating to a more modern GC becomes feasible. Now, we have experimented with Sgen, but did not find it to generally perform better then Boehm for the cases we care about. Instead, our current plan is to run Boehm GC in incremental mode (which also requires said changes to perform well), which allows Boehm to time-slice the marking process across multiple frames, thus significantly reducing the garbage collection spikes. We have seen very good results on testing this against content which was struggling with GC pauses, and hope to ship this functionality as an experimental feature on some of our platforms (basically Desktop, iOS, Android) in 19.1 (fingers crossed!).
     
  32. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    509
    Physics.RaycastNonAlloc has been there for a while.
     
  33. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    785
    Sounds awesome!

    Just out of curiosity, do you have any idea why sgen did give the expected improvements?
    At least in theory it sounds like a generational approach should help a lot.

    Anyway, time-slicing will definitely get the job done since all we care about is eliminating stuttering. Great work! Looking forward to it!
     
  34. xVergilx

    xVergilx

    Joined:
    Dec 22, 2014
    Posts:
    1,735
    People still tend to use alloc version. Enforcing non alloc versions should be a better solution.
    Or slowly transitioning them via [Obsolete]
     
    AlkisFortuneFish and dadude123 like this.
  35. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,563
    I honestly don't know, and I was also surprised to see it not faring better. I would agree that a generational approach should theoretically improve GC spikes at least for certain allocation patterns common in Unity content. It is entirely possible that I have configured sgen with less-then-optimal setting when I did the testing.

    We decided to go with incremental Boehm GC for now, because it gives quite decent results for use cases we care about, and because it is the smallest change we can do which improves the situation (as our current builds of both mono and il2cpp use Boehm). Which makes it easier to focus on getting the native code changes needed for any more modern GC (including incremental Boehm) to work correctly. This way, if there are issues, it is easier to pinpoint them. Once all of this is in place and shipping, we will re-evalute the situation, and asses if Incremental Boehm is something we consider to be "good enough" for our user's need, or if we want to invest further resources into replacing the GC with a completely different one (which should be easier at that point, as we'd have confidence in the required native code changes by then).
     
  36. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    79
    Generational GC's help the most when you have many long lived objects which is counter intuitive since you expect that most garbage is short lived. My guess is that most Unity code is not optimized at all for this.

    SGen also has a concurrent collector. I am very dubious about benchmarking different garbage collectors since real world architecture will give different results. SGen is also still in active dev.
     
  37. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    It sounds like it's good enough for my needs IF you can help relieve pressure on things out of my control such as Unity physx callbacks. These are honestly the only remaining bottlenecks this side.
     
  38. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    262
    Don't mean to derail this conversation, just jumping in since someone said 'physics' three times here...

    1) RaycastAll, SphereCastAll etc (all unbound shape sweeps) are likely to get deprecated during 19.x in order to encourage the use of non-allocating API instead of using the wrong one and learning about the non-allocating one the hard way. It will no longer be postfixed with 'NonAlloc', it'd just be Raycast, SphereCast, etc. (A discussion topic still).

    2) There is a lively internal discussion about making it so that we share the GC objects required for the Collision class instance. There are several forms of that, all of which would be completely optional, but that means you won't be able to store a ref to that instance for long, and instead would have to copy the data you really need inside a callback. Great perf wins here potentially, without too much of breakage.

    3) We're working on a different approach to contacts that you'd be able to fetch right after the simulation in a buffer. I'll be starting a separate thread a bit later this year right about that. Think of being able to iterate a NativeArray of contact pair descriptor structs with no allocations at all. Only computed for the colliders that got manually marked as needing that info.

    4) There's likely to appear a small API (based on Immediate Mode) to compute contacts up on request. Pass two shapes, their poses and get contacts written to another NativeArray with no allocations made.

    anthony.
     
    Seb-1814, Ryiah, LeonhardP and 7 others like this.
  39. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    7,009
    Yay!

    What about thread specific GC with thread juggling* couldn't all the threads in Unity be cleaned of garbage with minimal impact on performance on multi-threaded CPUs.

    *The main thread would need to be switched over to allow it to be cleaned.

    Or does the world have to stop for GC, could a GC process run on the vsync or CPU gap between frames, slowly cleaning up as opposed to halting everything?
     
  40. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    Perfect, make strong performant changes! <3 I can take any form of API abuse so long as I get performance so I can actually use Unity's features instead of reinventing the wheel so much. Thank you. Cannot come soon enough.
     
    Deeeds likes this.
  41. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    509
    Threads do not run in isolated memory spaces, one thread can hold references to data of another thread.
     
  42. Deeeds

    Deeeds

    Joined:
    Mar 15, 2018
    Posts:
    739
    Is this similar to how contacts[0]. works in Physics2D?
     
  43. Deeeds

    Deeeds

    Joined:
    Mar 15, 2018
    Posts:
    739
    hippocoder likes this.
  44. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    I don't think I can risk turning off the GC for a production game so I won't do it. Way too wild west for commercial. To do something like this I would require staff or a much simpler game so that testing over an extended period of time throughout the project is something I could afford to do.

    In short I don't see turning the GC off as a step forward for my situation. Also, it would be a bad thing to use for physics because the memory will be too large, and will also need additional code to manage and detect how much memory is in use.

    Also, Unity hasn't sorted out it's own allocations which simply must come first. I do not incur the wrath of the GC in my own code at all.

    Example would be HDRP which constantly feeds the GC (it is a WIP so I can accept it for now), so it would simply just crash my game using this feature - in the editor and builds.

    And so on (this feature of turning it off is VERY situational)...
     
    dadude123 likes this.
  45. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,217
    I think what would work better is knobs to control when Boehm runs based on thresholds. Not that you can't do that already somewhat, but I'd rather see a safe way to control the GC as the official suggested approach. Also because it might encourage exposing more knobs.

    Effective GC tuning is about tuning behavior specifically for your application. At least in generational collectors. I know Boehm only in theory really, but I'm assuming it's a very similar situation just simpler.

    For instance with java I can take an app that with the default settings and decrease time spent in the GC by several hundred percent on your average game server, while at the same time making it's behavior fit the needs of the app better. That's partly a result of how inefficient default GC algorithms are, coupled with the fact that collections can have significant and varying amounts of housekeeping that are often application specific.
     
    nxrighthere likes this.
  46. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    I'm happy with a GC that simply has a clear cap on the amount of time it consumes during it's work. So I can set the limit to a conservative 0.5ms and it'll happily do that work every frame until it doesn't have to.

    It would also have a fail-safe threshold when it will do a full trigger, for example we might not want it to get higher than 2mb, in which case it'll run the GC fully when the threshold is met.

    I honestly see no drawbacks to this approach. How could there be? it would be extremely empowering for a lot of people to spend a fixed amount on the GC and help a heck of a lot of projects that don't have the resources to tackle eliminating GC, and it does take a fair bit of resources to code around it.

    The worst offender is of course Unity's own code. I cannot work around those.
     
    nxrighthere and AcidArrow like this.
  47. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,563
    This is basically what the new incremental GC will do. It is not a trivial change - the current GC cannot just spend 0.5ms collecting "a little", as it needs to scan all memory to understand which objects reference which other objects to be sure that an object is not used - and this scanning of memory is the part which takes time. To be able to time slice this, we need Unity to tell the GC whenever an object reference has changed, so that the GC knows it will need to scan this object again. This required a lot of code changes to Unity and il2cpp to work.
     
    dadude123, phobos2077, Ryiah and 4 others like this.
  48. Peter77

    Peter77

    Joined:
    Jun 12, 2013
    Posts:
    3,711
    Do you refer to updating the reference count (link)?
    Is this code a person would write or compiler generated?
     
  49. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,276
    And that is why I continue to be impressed with your work, young lad.
     
  50. Claytonious

    Claytonious

    Joined:
    Feb 16, 2009
    Posts:
    554
    What is the roadmap and status of this?