Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Garbage Collector API

Discussion in '2018.3 Beta' started by hippocoder, Sep 27, 2018.

  1. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,326
    This sounds interesting. Any chance you'll open a thread on that too?

    I don't think I've ever kept a reference to a Collision, so that speaks in favour of this. On the other hand, having an object that just randomly stops existing isn't exactly standard C#, and there's a lot of good ideas Unity has had over the years where not doing the standard C# thing causes very real problems later on (if(this == null) comes to mind!).

    The thing is, if you do your suggestion 3, you can just remove the contacts-array from the Collision type. Then the collision will just contain two Vector3's (impulse, relativeVelocity), and two references (rigidbody, collider). In that case, you can just turn Collision into a struct, and save everyone a lot of headache. No GC, no breaking with C# standards.

    Or, I guess the C# standards say that you shouldn't have references in structs, but that's just straight up idiotic, so don't mind that. Anyway, a struct-based solution seems infinitely preferable to having an object that becomes invalid after a few frames.

    For OnCollisionEnter/Stay, you just make two different overloads:

    Code (csharp):
    1. void OnCollisionEnter(Collision c) { // I don't care about the contacts!
    2.     ...
    3. }
    4.  
    5. void OnCollisionEnter(Collision c, ContactPoint[] contacts) { // I do care about the contacts!
    6.  
    7. }
    Or you could go with the "ask for a NativeArray that matches this collision" approach.
     
  2. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    319
    That's correct. The problem is we can't switch from class to struct without breaking too many projects, especially the ones where pre-compiled libs are used. It's been discussed here on forums a few years ago and the solution has to revolve around introducing a new API for retrieving contacts: either a new set of callbacks exactly like the current ones but accepting a new struct Collision, or a completely different approach like I outlined above with no callbacks used at all. Now, if we were to introduce a new API the question still holds what happens to the old one, could we still improve that for the upgrading users. The experiments we're running are to help with exactly that. Have a switch that enables some extra perf wins for the old path if possible. As I said, it's just experimenting at this moment in time. Maybe we won't ship anything like that.
     
  3. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    542
    @jonas-echterhoff Thank you, this is good news. The last couple of months I'm working on a modern networking system for Unity. It's heavily inspired by LMAX Disruptor and some other inter-thread messaging and concurrency technologies. To be honest, I don't suffer from GC at the client-side, it's not hard for me to control memory-related stuff there for the player. The hard part is a fully concurrent server right inside Unity. Yes, we have the .NET Core with great Server GC for this purposes, but we don't have it in the Unity. So, I want to show you something interesting.

    (Dozens of Unity processes with network entities and physics replication is too much for my CPU, but yea...)


    I successfully achieved almost GC-free runtime even during heavy-loads with a crazy amount of buffers, classes, events, and other data-related stuff. All these things are under the control of self-stabilizing concurrent pools that's warmed-up based on predictions like how many clients will be connected to the server and so on. Everything is computed in parallel and dispatched to the main thread using non-blocking circular buffers/queues. Memory allocation happens only when resources not enough in the pools due to a very dynamically growing number of clients, and accordingly, the data for processing. But usually, such allocations are very small, around a dozen bytes, and the pools are adapting themselves very fast.

    The goal of this project is high-performance networking backend that works right inside Unity. And this goal was finally reached a few days ago. The resources usage and latency of the parallel processing is very low even with hundreds of clients (20 state updates per second for each entity). The latency of parallel computations is around ~0.25 ms per 100 clients with 64 updates per second on 4-core AMD FX-4300 (the latency on the video is a bit higher because of those Unity processes for clients on the background and video recording). Even if latency starts growing due to high-loads the system is never interrupting a game loop within hours of profiling:

    ScionNet 1.0.4.png
     
  4. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,797
    For what it's worth I don't really care about C# correct standards. That doesn't make a great deal of sense and never has in Unity context - consider ECS for example.

    Regarding if you will break projects or not break projects and so on, I think anything is better than letting the problem fester like it has for over a decade.

    There's this mortal dread in Unity of changing a few lines in a major version release that requires some code. So what, really? Sometimes I get a little tired of that. Other engines just breaks things on major versions. People code the fix and everyone is better off.

    I figure Unity does this with semantic versioning on packages, why not elsewhere?
    We have to make an omelette :)
     
  5. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    2,425
    I think they also have to worry about indirect breakage just from inadvertently changing behavior. In my game my single largest consumer of cpu is an api they are afraid to touch, one they know how to fix but breaking old stuff is at least partially holding them back.

    I get both sides of that, it sucks being on the losing end though :)
     
    phobos2077 likes this.
  6. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    No, we don't do ref counting. This is about informing the GC when an object references another object, and those references are changed, so the GC will know that it needs to rescan that object, when it is in the middle of an incremental collection.
     
  7. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    Currently planned to land in 19.1 as an experimental feature on a subset of platforms.
     
    rizu, Claytonious and optimise like this.
  8. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,326
    This has come up quite a few places, where there are improvements that can't be made due to "breaking projects".

    Here's the thing, your updates always break projects. Unity has never been particularly backwards compatible. With the tech and LTR streams, you're even more capable than ever of changing something like this. Especially since the ways you're proposing to change contacts would already be pretty breaking.

    Am I missing something? Is there something special about a class->struct change for precompiled libs compared to say an API change?
     
    GameDevCouple_I and hippocoder like this.
  9. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    Having used few other engines, I'd strongly disagree here. Sure, Unity deprecates some old systems eventually but I actually tried what would happen if I opened my old Unity 4.3 project in 2018.2. In two hours, I had fixed most of the issues and had the game playable. I estimate it would have taken me from few days to a week to make it play exactly as on 4.3. With other engines(tm), this amount of work and more can happen when updating your project from single engine update to another. We are now talking about updates we get like 3-5 times a year for those engines vs Unity updates that have happened in total of past 5 years now.

    In my experience Unity is really good at keeping the API backwards compatible which is reason why I can even make old Unity 3.x projects run with small effort. I do get that sometimes this approach gets in the way but in general, it's what makes Unity also easier to use in longer term. I've noticed that people making new projects in general don't care about this that much but it is also so easy to forget that there are thousands of assets that rely on the existing API. Changing it in ways that break the old approaches will always break hundreds or even thousands of third party assets and there might not be people maintaining them anymore. I'm sure things like these are on the table when Unity has to redo some important API part.
     
  10. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,797
    Yes but with package manager we can make both camps happy. I'm firmly in the forward looking camp. I don't care nor do I want back compat. Why would I?
    • package manager lets me go back in time
    • LTS lets me have stability with older versions
    It's solved. There's no need to prevent breaking changes moving forward. It's the whole rationale about Semantic Versioning. You can please everyone because everyone makes informed decisions. And breaking changes have to happen or it'll be another 10 years of dodgy physics performance and Unity allocations.
     
  11. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    Don't get me wrong, I do think both going forward and keeping API compatible are equally important and there are ways to have them both (which is what Unity has been doing already). You can always bring new API functions along the old ones and slowly deprecate the old approach to drive people use the newer thing. Sometimes it does take more time to do things like these carefully as even trivial looking changes can have big consequences.
     
  12. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,797
    I'm just seriously happy these things are Unity's focus. Unity is growing up in a great way despite the relatively closed source nature.
     
  13. yant

    yant

    Unity Technologies

    Joined:
    Jul 24, 2013
    Posts:
    319
    I shouldn't probably post here anymore, but there's this sense of transparency, so there you go.

    We have to agree some users care about compatibility to a certain degree while the others don't. It's certainly nice not having to redo your whole project from scratch every time an update comes online.

    That said, I don't necessarily get the argument. There are things planned to please both the ones willing to go the new route and the ones not quite willing to invest into figuring out how to upgrade. All that hopefully scheduled to be available early to mid next year. It's not about exactly 10 years really.
     
    hippocoder and optimise like this.
  14. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,797
    Sorry I meant no offence - just a figure of speech talking about how historically, it's been a long time coming for changes that enable us to get around on collision stay (sometimes you need updated contacts each frame). Did not mean to offend.
     
    GameDevCouple_I and yant like this.
  15. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    They also marked GC update "on track" on the roadmap now (still under development section instead of 2019.1 but these feats usually tend to get moved to final slot only when it's fairly certain the feat can make into that release)
     
  16. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    790
    So you'd somehow inject code into every reference-assignment statement?
    That sounds pretty scary, no? Would that be just setting some flag in the object header directly?

    Just trying to understand the performance implications here.
    Isn't that like at least a global 10%+ reduction in performance?
     
  17. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,797
    I don't understand the 10% figure. But typically, how often do we assign references? When I do so it likely is when I am working with pooled assets, AI, general player logic. So it comes down to how often you'll be doing these actions. For me it's never often out of habit.

    Interested in any response about the cost of marking though for purely educational purposes...
     
  18. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    Yes, it is setting a flag (but on a table of GC memory pages, not on the object). Yes, this adds overhead for changing references. For most content, changing reference does not have a significant share in spent CPU times, so the overhead should be neglectable. It has been for content we have been testing on. But it is certainly possible to construct artificial test cases where there would be a noticeable performance impact. We hope that this won't be the case for a lot of real life content, but sometimes, people do unexpected things in their code, so I would not go on a limb to say that this won't affect anyone negatively. The cost should be much lower if incremental GC is not enabled, though. It will initially be a default-off, experimental feature.
     
    dadude123, hippocoder and pvloon like this.
  19. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    790
    Thanks for the insight!
    Will code generation cause any trouble with this?
    In our game we're heavily using Expressions for pretty much everything. From in-game scripting, over networking, to serialization in general.
    I'm pretty sure this will be a change to the JIT so things like this won't be a problem. But I'd still like to ask just to be sure: code generation won't be a problem, right?
     
  20. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    code generation won't be a problem.
     
    dadude123 likes this.
  21. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,797
    I might be able to do this now, thanks Unity :)

    (although I still want to see what the new GC is about)
     
  22. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    this popped up in 2019.1.0a8 changelog as a new feature (available now):

     
    Peter77 and hippocoder like this.
  23. dadude123

    dadude123

    Joined:
    Feb 26, 2014
    Posts:
    790
    @jonas-echterhoff
    The changelog says "incremental". Is it actually an incremental GC now and not the reference tracking stuff with time-slicing?
     
  24. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    That's the same thing essentially, they need to track references to make the time-slicing so they can collect garbage incrementally.

    Here's a screenshot from editor:
     
  25. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    542
    Can we control the time-slice intervals? I see mono_gc_set_max_time_slice_ns(int64_t maxTimeSlice) in the internal C API, but don't know if this stuff is self-calculated/hard-coded or available at high-level for tuning.
     
    Last edited: Nov 7, 2018
    Prodigga, Kolyasisan and Peter77 like this.
  26. Player7

    Player7

    Joined:
    Oct 21, 2015
    Posts:
    1,432
    Is this something you are releasing on github? I'm not sure if its the same project I've heard bits about in the other benchmark thread..is it?
     
  27. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    542
    Yep, it is. The latest info is here, latest demonstration video, and another one.
     
    Last edited: Dec 2, 2018
  28. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    @jonas-echterhoff
    Official roadmap just got modified to include "Garbage Collector Upgrade" for 2019.1. It's description puzzles me though:
    "Upgrade to a modern, generational, low latency Garbage Collector (GC). This will reduce stutters and time spikes spent in GC during collection.".

    From what I've read on these forum posts, it makes me think the description is outdated and should be changed to reflect current plan for the GC in 2019.1 (that is, unless you still want to upgrade the whole GC for it, besides making it work incrementally).
     
    Last edited: Nov 18, 2018
  29. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    Yes, we will expose and API to do this. But as of 19.1a10, we will also automatically give any available time when waiting for vsync or Application.targetFrameRate to the incremental GC dynamically.
     
  30. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,633
    You are right, this is not accurate. I will ask the people responsible for that page to update the text, thanks!
     
  31. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    4,326
    Well hello there. From what's new in Unity 2018.3.0 Beta 11:

     
    xVergilx likes this.
  32. xVergilx

    xVergilx

    Joined:
    Dec 22, 2014
    Posts:
    2,012
    Yay, backported goodness!
     
  33. nxrighthere

    nxrighthere

    Joined:
    Mar 2, 2014
    Posts:
    542
    That's great, thanks! I'm okay with trading memory in favor of more constant frame-rate, it's really nice to have such control over GC.

    By the way, I've recently brought smmalloc to the .NET (inspired by jemalloc.NET, but it's faster, more portable, and lightweight). I'm experimenting with it using Span<T> for managing data in native memory blocks. The memory allocator is highly scalable in a multi-threaded environment, TLS greatly boost the performance for allocations within a thread. It's a very useful thing in right hands.
     
    wobes likes this.
  34. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,186
    Last edited: Nov 26, 2018