Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Difficult to reproduce "Oops!" crash when loading scene during loading screen

Discussion in 'Windows' started by Lisk, Dec 11, 2018.

  1. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Me and several engineers have been working on solving a game crash bug for about 15 hours this week, so I thought to make a thread.

    The game crashes with an "Oops! The game crashed!" message during the loading screen at a specific time (90.1% complete), which is when the scene is loaded via SceneManager.LoadSceneAsync(sceneNumber, LoadSceneMode.Additive)

    My best way of reproducing the crash has been:
    - Use Windows 10 or Mac OSX (the crash does not happen on Windows 8)
    - Reinstall the game (the crash happens much more commonly on a fresh install of the game)
    - Spam ALT+TAB while the game is in the loading screen (the crash does not happen if you don't spam ALT+TAB. It does not happen, or very rarely happens, if you ALT+TAB just a few times or just have the game minimized. Spamming ALT+TAB seems to be the only semi-reliable way of reproducing the crash).
    - The game crashes about 50% of the time. If I change any of the above steps, I have not been able to reproduce the crash at all. (edit: it varies, sometimes I'll go 10 times in a row without crashing, sometimes I'll go 5 times in a row crashing)
    - Changing the LoadScene call to a synchronous call does not change the situation - it still crashes.
    - Another thing I tried was copy & pasting my environment props 10x, thinking that it would increase the chance of crashing, but it didn't seem to affect it at all.

    I have plenty of crash dumps. Attached are a few of them.

    The stacktrace is consistently the same:
    Code (CSharp):
    1.      Legion TD 2.exe!memcpy() Line 390    Unknown
    2.      Legion TD 2.exe!ReadFileCache(CacheReaderBase & cacher, void * data, unsigned __int64 position, unsigned __int64 size) Line 23    C++
    3.      Legion TD 2.exe!SerializedFile::ReadHeader() Line 615    C++
    4.      Legion TD 2.exe!SerializedFile::InitializeRead(const std::basic_string<char,std::char_traits<char>,stl_allocator<char,66,16> > & path, ResourceImageGroup & resourceImage, unsigned int cacheSize, unsigned int cacheCount, TransferInstructionFlags options, unsigned int readOffset, unsigned int readEndOffset) Line 319    C++
    5.      Legion TD 2.exe!PersistentManager::GetStreamNameSpaceInternal(int nameSpaceID) Line 1989    C++
    6.      Legion TD 2.exe!PersistentManager::GetLoadErrorMessageFromPath(const std::basic_string<char,std::char_traits<char>,stl_allocator<char,66,16> > & path) Line 240    C++
    7. >    Legion TD 2.exe!LoadSceneOperation::Perform() Line 108    C++
    8.      Legion TD 2.exe!PreloadManager::Run() Line 367    C++
    9.      Legion TD 2.exe!PreloadManager::Run(void * managerPtr) Line 205    C++
    10.      Legion TD 2.exe!Thread::RunThreadWrapper(void * ptr) Line 48    C++
    11.      kernel32.dll!BaseThreadInitThunk()    Unknown
    12.      ntdll.dll!RtlUserThreadStart()    Unknown
    13.  
    It seems like it has something to do with caching, given that ReadFileCache is in the stacktrace and the bug seems to only reproduce on first runthrough. Otherwise, I'm not sure what to do.

    I have encountered the crash on several hardware configurations. Both on laptops and desktops. At least one on a GTX 1080, so it is doubtful that GPU memory capacity is an issue here.

    Does anyone have tips on how to further debug this? The scene I am loading is fairly simple - it's a bunch of trees, point lights, and water. No scripts, actually. All of the game scripts are already loaded way earlier - I am just additively trying to load in the environment props.
     

    Attached Files:

    Last edited: Dec 12, 2018
  2. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,646
    Hi, from what I can see in the dump file, it's a crash when loading assets. We actually had a report of a very similar issue recently (just a couple of weeks ago). We fixed it, but I'm not sure which release the fix will land in yet (as the fix was just made). However, you seems to be on a super old Unity version. What are the chances you'll be able to upgrade?
     
  3. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Thanks for the reply, Tautvydas-Zilys! The chances are high, but I won't be able to do it until early-mid Q1 2019. Every time I upgraded the Unity version in the past, various plugins broke or people experienced rare crashes in-game, so I don't want to perform such an upgrade right before the holidays (when I take a vacation).

    Upgrading Unity is definitely something I've considered as a possible solution, but it's good to hear it from an official source.

    Do you know if there are any possible workarounds in the meantime? For example, do you think reducing the size of the scene (fewer game objects) might help at all?
     
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,646
    I wasn't the one to fix the bug so I don't know exactly what triggers it, but I'll talk with the person who did it.
     
    Lisk likes this.
  5. JEONSULIN

    JEONSULIN

    Joined:
    Dec 29, 2016
    Posts:
    4
    We are experiencing very similar issue as well perhaps the same issue. We get same crash when loading a scene and yes we are using SceneManager.LoadSceneAsync as well.

    ========== OUTPUTING STACK TRACE ==================
    0x00007FF72C8E3660 (TUp2) memcpy
    0x00007FF72BF45C1C (TUp2) ReadFileCache
    0x00007FF72BF40D4D (TUp2) SerializedFile::ReadHeader
    0x00007FF72BF417AE (TUp2) SerializedFile::InitializeRead
    0x00007FF72BF4214C (TUp2) PersistentManager::GetStreamNameSpaceInternal
    0x00007FF72BF42490 (TUp2) PersistentManager::GetSerializedFileInternalIfObjectAvailable
    0x00007FF72BF42E2A (TUp2) PersistentManager::ReadObjectThreaded
    0x00007FF72BF1A047 (TUp2) LoadSceneOperation::perform
    0x00007FF72BF196E1 (TUp2) PreloadManager::Run
    0x00007FF72BF19A49 (TUp2) PreloadManager::Run
    0x00007FF72BFFFCBC (TUp2) Thread::RunThreadWrapper
    0x00007FF809093034 (KERNEL32) BaseThreadInitThunk
    0x00007FF80A851471 (ntdll) RtlUserThreadStart
    ========== END OF STACKTRACE ===========

    ========== OUTPUTING STACK TRACE ==================
    0x00007FF6AA4E3620 (TUp2) memcpy
    0x00007FF6A9B45C1C (TUp2) ReadFileCache
    0x00007FF6A9B40D4D (TUp2) SerializedFile::ReadHeader
    0x00007FF6A9B417AE (TUp2) SerializedFile::InitializeRead
    0x00007FF6A9B4214C (TUp2) PersistentManager::GetStreamNameSpaceInternal
    0x00007FF6A9B429D0 (TUp2) PersistentManager::GetSerializedTypesAndIdentifiers
    0x00007FF6A95E861A (TUp2) AssetBundleManager::populatePreloadData
    0x00007FF6A95EFFDD (TUp2) AssetBundleManager::SortPreloadObjects
    0x00007FF6A9B1A171 (TUp2) LoadSceneOperation::perform
    0x00007FF6A9B196E1 (TUp2) PreloadManager::Run
    0x00007FF6A9B19A49 (TUp2) PreloadManager::Run
    0x00007FF6A9BFFCBC (TUp2) Thread::RunThreadWrapper
    0x00007FFF61BE3034 (KERNEL32) BaseThreadInitThunk
    0x00007FFF62401471 (ntdll) RtlUserThreadStart
    ========== END OF STACKTRACE ===========

    and I'am guessing If it does not crash, it makes memory break somehow because we're getting NullReference Exception in the code where we were using it for more than a year without any problem. Maybe it is not relevant but We were using 2017.1.0f3 version before and upgraded to 2017.1.0p5 recently and suddenly we got this crash so we thought 2017.1.0p5 was the problem and downgraded again to 2017.1.0f3. However the crash still exist in 2017.1.0f3. What's weird is that we never get this crash before we upgraded to 2017.1.0p5.
     
    Lisk likes this.
  6. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Thanks, looking forward to your response! We're pretty pressed for time to try to get a fix out before holidays.

    In the meantime, can you at least tell me what version the bug report you're referring to was detected on? If it was o Unity 2018, then it's probably not worth me upgrading right now, since I'd hate to introduce potentially more bugs if that one is a different one than the one I am experiencing.
     
  7. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,646
    Unfortunately it doesn't seem like it's possible to make a workaround for this... Here's what the person who fixed it said:
    The fix will land to 2017.4, 2018.3 and 2019.1+.
     
  8. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Thanks for the reply and checking with the engineer who made the fix.

    I still think there has to be some way to workaround it, something like simplifying the scene we are currently using. We have two different maps, a 2v2 map and a 4v4 map. The crash only happens when using the 4v4 map. I tested extensively to confirm that it is the scene loading (and not any gameplay logic differences) that causes the crash. I also created a test map with no trees, and it no longer crashes.

    Is the "large file" the comment refers to, referring to an individual asset? (like an individual model or texture or other binary file on disk), or does it refer to the cumulative scene size? AKA would reducing texture sizes potentially fix it vs. deleting a bunch of trees?
     
  9. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,646
    It refers to a particular file on disk, which is either a scene or "shared assets" file, which is shared between multiple scenes.

    I have an idea, although I'm not sure if it will work. If your game is 64-bit only, you could try writing a plugin that would reserve the lower address space so that it couldn't be used by Unity. Since the bug is triggered when the file is loaded into a low memory address, reserving bottom 4 GB using VirtualAlloc (using MEM_RESERVE flag, so the memory doesn't actually get allocated since you don't want to increase your process memory usage). This should theoretically work around it. However, you'd need to do it very early in the process life time.
     
    Lisk likes this.
  10. JEONSULIN

    JEONSULIN

    Joined:
    Dec 29, 2016
    Posts:
    4
  11. Lisk

    Lisk

    Joined:
    Oct 23, 2013
    Posts:
    99
    Thanks for the additional info. On our end, we don't even use any asset bundles if that's helpful to know.

    I'll look into the VirtualAlloc workaround, but I probably can't prioritize it because we support 32 bit OSes as well as Mac OSX.

    The workaround I'll probably end up going with is just removing a lot of trees.
     
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,646
    I guess splitting them into different files like asset bundles would help too.