Search Unity

Bug Reproducible memory leak in Graphics and Graphic Device pool when instantiating an entity

Discussion in 'Graphics for ECS' started by jrf407, Mar 20, 2023.

  1. jrf407

    jrf407

    Joined:
    Feb 29, 2020
    Posts:
    7
    Please see the code here to reproduce this issue: https://github.com/JeremyFrederick/EntityGraphicsBug

    The code calls GetEntity in a baker to save an entity that is not in the original scene. When that entity is instantiated, the Graphics and Graphic Device pool grows exponentially and eventually crashes. This can be reproduced in the code in the repository that is linked.
     
    Thygrrr likes this.
  2. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    700
    @jrf407 what's the crash / stack trace you get with this? You absolutely rock! I'll have to run this for a while.

    @richardkettlewell this could be be (or be related to) the elusive but annoying crash or memory leak I've been seeing in 2022.2.10f1 and earlier with ECS.

    The runtime memory leak didn't have any entity instantiating going on, but it happening during baking could be relateing to what causes the other crashes.

    Possibly related cases:
    https://unity3d.atlassian.net/servicedesk/customer/portal/2/IN-35737
    https://unity3d.atlassian.net/servicedesk/customer/portal/2/IN-34770
    https://unity3d.atlassian.net/servicedesk/customer/portal/2/IN-34365
    https://unity3d.atlassian.net/servicedesk/customer/portal/2/IN-34398
    https://unity3d.atlassian.net/servicedesk/customer/portal/2/IN-35830

    Edit: I can reproduce the erratic behaviour in 2022.2.10f1 I've been able to use as little as 95 MB with the "bug", most frequent is something in the 1-2 GB range, sometimes 10 GB in one go, and in my subsequent post I show a way to "re-prime" it, which then leads to cumulative leakage.

    I can totally see this crashing quite often with more than 1 subscene, which is what keeps happening to me.

    Edit2: Yeah this will occasionally consume all available memory over time in a normal editor or play mode usage pattern, confirmed.

    (profiler screenshot of an example run of Jeremy's project showing 6.9 GB of graphics driver memory used; regular case is around 1 to 2 GB, I can get it to as little as 95 MB in some runs, all other things the same, and the upper bound is likely infinite until crash)
    upload_2023-3-22_11-55-37.png
     
    Last edited: Mar 22, 2023
  3. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    700
    upload_2023-3-22_12-8-20.png

    By unloading and reloading (or just reimporting) the subscene during Play Mode, you can "prime" the bug again, and pressing "1" on the keyboard has a likelihood of consuming another chunk of memory, and I'll cumulatively try to get it to crash so I can compare stack traces.

    HOWEVER; this bug also "re-primes" itself if the EditorApplication loses focus, and then more memory gets gobbled up with a certain chance after the next time it becomes active (just focus is enough, no need to minimize). It also "spirals" after focus regained sometimes (may need to press "1")

    This was exactly what I was observing and why I reported so many crashes related to, but not exclusively connected to, coming back to the Editor from Rider or a Web Browser or other app, suspecting it had something to do with Compilation (but some crashes just happened without me doing anything)

    upload_2023-3-22_12-10-39.png

    Edit: yes, this is the bug that has been haunting me for most of March, I was already thinking I was imagining things because I had only a sporadic repro, and only in ECS projects.

    upload_2023-3-22_12-14-13.png

    I'm so grateful for this repro, thank you so much!
     
    Last edited: Mar 22, 2023
  4. wechat_os_Qy09AtTzCAF9FWIJIvuFRFhfs

    wechat_os_Qy09AtTzCAF9FWIJIvuFRFhfs

    Joined:
    Jun 17, 2021
    Posts:
    5
    I got the same problem in Unity 2022.2.11 and 2022.2.12.
    Entities: 1.0.0 pre.47
    Entity Graphics: 1.0.0 pre.44
    The Memory use raised up as soon as I spawn entities in Android device and finally cause a crash.
    20230323-175232.jpeg
     
    Thygrrr likes this.
  5. Xentarok

    Xentarok

    Joined:
    Apr 3, 2013
    Posts:
    17
    Maaan, I've been going crazy over a memory leak since upgrading my project from Unity 2021 and Entities 0.51 to 2022 and 1.0, glad it seems I'm not insane - for me it doesn't matter if I disable the subscene with the prefab authoring or if I'm in editor or build, Unity will just keep piling memory untill it explodes.
     
    StefanWo and Thygrrr like this.
  6. wechat_os_Qy09AtTzCAF9FWIJIvuFRFhfs

    wechat_os_Qy09AtTzCAF9FWIJIvuFRFhfs

    Joined:
    Jun 17, 2021
    Posts:
    5
    I tested.
    This problem still exist in Entities 1.0.0 pre.65
     
  7. JussiKnuuttila

    JussiKnuuttila

    Unity Technologies

    Joined:
    Jun 7, 2019
    Posts:
    351
    I have spent time investigating this issue and found a possible cause. We will try to develop a fix as soon as possible.
     
    Xentarok and MartinTilo like this.
  8. JussiKnuuttila

    JussiKnuuttila

    Unity Technologies

    Joined:
    Jun 7, 2019
    Posts:
    351
    This issue should be fixed in the next release of Entities Graphics.
     
  9. JussiKnuuttila

    JussiKnuuttila

    Unity Technologies

    Joined:
    Jun 7, 2019
    Posts:
    351
    If this is a blocking issue, you should be able work around the issue with either of the following:
    • Have at least one renderable entity present in the ECS World (the bug is caused by code getting incorrectly skipped when there are no renderable entities)
    • Make a local code modification to the
      UpdateEntitiesGraphicsBatches
      method in
      EntitiesGraphicsSystem.cs
      : Remove the line with the
      if
      statement, so the code inside the if statement gets executed every time the method is called instead of just when the condition is true.
     
  10. wechat_os_Qy09AtTzCAF9FWIJIvuFRFhfs

    wechat_os_Qy09AtTzCAF9FWIJIvuFRFhfs

    Joined:
    Jun 17, 2021
    Posts:
    5
    We tried the first one and it worked.