Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

"Fatal error in gc: Too many heap sections"

Discussion in '2018.3 Beta' started by LazloBonin, Nov 26, 2018.

  1. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    813
    Since 2018.3, we've been getting this error frequently while doing an allocation-intensive operation:



    Admittedly, we allocate a lot of memory during that operation (~40MiB), but it was never an issue in previous versions of Unity.

    Could something have changed under the hood in how GC is managed? Maybe in preparation for incremental GC? Is there any way to avoid this error?
     
  2. dbarnhart-angle

    dbarnhart-angle

    Joined:
    Mar 15, 2018
    Posts:
    6
    Make that two people. It seems to occur whenever the application exceeds 3.5GB of CPU memory allocation. It's like Unity is running in 32-bit mode somehow. This is running in editor mode.
     
  3. Tristan-CodeHatch

    Tristan-CodeHatch

    Joined:
    Nov 18, 2013
    Posts:
    3
    We also have this happening when we recompile or try to make a build. It always happens around compilation but the crash stack trace is always different. We can repro it with only script and assembly(.dll) assets in the project. This was after we upgraded our project to 2018.3. We assumed it was something in our source that the compiler was having troubles compiling. We've had bugs like that before unfortunately. It takes a while to figure out what peice of code it was so we are just avoiding the upgrade for now. Though, I am not sure it is the same issue the OP is having.
     
    Last edited: Dec 4, 2018
  4. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,619
    Could you please submit a bug-report, as described in this document:
    https://unity3d.com/unity/beta/guide-to-beta-testing#tab-3

    After you submitted the report, you receive a confirmation email with a Case number. Please post this bug-report Case number here (in this thread) for Unity staff to pick up.
     
    LeonhardP likes this.
  5. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,136
    Yes, bug reports would be much appreciated.
     
  6. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    813
    I can submit a bug report but I can't guarantee a 100% repro. It seems to depend on memory layout & usage prior to the intensive operation. Would it help more if I waited to get an actual crash from it so that the dump is included?
     
  7. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,619
    It probably does.
     
  8. dbarnhart-angle

    dbarnhart-angle

    Joined:
    Mar 15, 2018
    Posts:
    6
    I just submitted a crash dump. Case 1104564.
     
    Peter77 likes this.
  9. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    Thanks. It would help a lot if you could attach a repro project to the bug report, is that possible?
     
  10. Tristan-CodeHatch

    Tristan-CodeHatch

    Joined:
    Nov 18, 2013
    Posts:
    3
    I submitted a bug report on November 15th. The case number is 1101240. I attached several logs with crash stack traces in them. I cannot provide a repro project as it would currently require us to send our entire code base.
     
    Last edited: Nov 29, 2018
  11. Carpe-Denius

    Carpe-Denius

    Joined:
    May 17, 2013
    Posts:
    842
    Do you use Vegetation Studio? I think someone with Vegetation Studio Pro has already submitted something. If not, I would try to get a test project. In my project it is reproducible in 5 or 6 clicks.
     
  12. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,136
    Thanks for the hint! We have this bug logged for it: https://issuetracker.unity3d.com/product/unity/issues/guid/1086743/

    This might be related to your cases, but it's hard to tell for sure without looking at your projects. More reproduction projects would be very welcome.
     
  13. dbarnhart-angle

    dbarnhart-angle

    Joined:
    Mar 15, 2018
    Posts:
    6
    I just attached a very simple repro project to my fogbugz case. It just spams allocations in an update loop until it crashes. The issue isn't that it crashes, so much as _when_ it does. Which is just after about 4GB of allocations occurs. Then unrelated variables reset to zero, and other weird stuff happens. I used 2018.3.0b12 for this.
     
    LeonhardP and Peter77 like this.
  14. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    Hey Darryl,

    I wrote this reply on the bug itself, but since there is this forum discussion about the topic, I thought I'd share it here for others interested:

    I looked at your repro project. I don't consider this a bug, as it is a script which just allocates memory until it runs out, which it is bound to do eventually.

    Now, in regards to your assumptions on when it should run out vs when it actually does: You say that you have 32 GB of RAM on your machine. But that hardly makes any difference when it comes to how much memory you can allocate.
    1. In modern operating systems with virtual memory, allocations don't need to be tied to physical memory but can be swapped to disk. So it is actually possible to allocate more memory then you have, it will become very slow.
    2. However, when it comes to managed memory in Unity, the garbage collector keeps data structures of the memory it manages, as it needs to keep track of it. The size of these data structures limits how much memory you can allocate, and may be much less than the amount of raw memory you could allocate. Now, it would certainly be possible to change these values in the GC to allow more managed memory - but there is a tradeoff between the size of these values and performance. Bigger values would mean bigger indices into bigger data structures, making GC operations more expensive to process. The current values have been chosen to be what can be considered a reasonable tradeoff by monos developers. Now, I have not done any testing in practice as to what the performance impact of changing this would be, and there may well be valid reasons to increase them (possibly just for the editor), if we find that developers are unable to write code performing reasonable tasks within these boundaries. But a script allocating memory until it crashes does not seem like a reasonable use case for this.
    3. As for your question why your memory counter wraps around at 4GB: That's because the counter is a uint, which is an unsigned 32-bit integer, which can only store values up to 4294967295.

    jonas
     
  15. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    I'll reply to myself here - it seems I may have been a bit too fast to dismiss this. Sorry for that!

    While it is correct that there are hard-coded limits in the GC, when testing this project in 2018.2 it does indeed get much further then in 18.3 - which seems to indicate that we have changed these limits (probably by accident when we merged some GC code between mono and il2cpp). This means that code could previously assume that it could use more memory, and that reducing this amount makes this a regression. I will look into what might have caused this change!
     
    Cromfeli, ProtonOne, olavrv and 2 others like this.
  16. LazloBonin

    LazloBonin

    Joined:
    Mar 6, 2015
    Posts:
    813
    Thanks for following up. In our case, we are far from reaching even the 1GB threshold, we're still in the MBs, so I doubt the heap is actually running out of space.
     
  17. dbarnhart-angle

    dbarnhart-angle

    Joined:
    Mar 15, 2018
    Posts:
    6
    For the benefit of others who are hitting an absolute allocation limit: I was able to work around this issue by using NativeArrays. Of course, you have to manually dispose the memory, but that's often acceptable.
     
  18. jonas-echterhoff

    jonas-echterhoff

    Unity Technologies

    Joined:
    Aug 18, 2005
    Posts:
    1,666
    FYI, this regression has been fixed and we switched the limits back to what it was in 18.2. The fix should either be in the 2018.3.0 first release or in the first patch release, depending on timing.
     
  19. aurelien-morel-ubiant

    aurelien-morel-ubiant

    Joined:
    Sep 27, 2017
    Posts:
    275
    that's a good news for the fix !
    So I can deduce that you will release rc or maybe final soon ?
    Even with all the warning in the poll feedback :/
     
  20. GorkaChampion

    GorkaChampion

    Joined:
    Feb 6, 2018
    Posts:
    103
    Good news, let's hope is coming in 2018.3.0
     
  21. GorkaChampion

    GorkaChampion

    Joined:
    Feb 6, 2018
    Posts:
    103
    Is it fixed ( regression has been fixed and we switched the limits back to what it was in 18.2.) in the 2018.3.0f1?

    Thanks,
     
  22. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,136
    It should be. The fix is in that version.
     
  23. Chaz32621

    Chaz32621

    Joined:
    Apr 17, 2015
    Posts:
    199
    I got this today, on the latest build 2018.3.0f1. It looks like it might have something to do with dragging and dropping a script into a button event. But I am not 100 percent.
     
  24. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    3,136
    If you can reproduce it, a bug report + reproduction project would be great.
     
  25. Phantomb

    Phantomb

    Joined:
    Jul 15, 2015
    Posts:
    1
    I had this issue on machines running 2018.3.0b2, and it seems fixed fixed on 2018.3.0f1.
     
    Cromfeli and LeonhardP like this.