Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Deadlock in AssetBundleManager

Discussion in 'Asset Bundles' started by georgerh, Nov 11, 2021.

  1. georgerh

    georgerh

    Joined:
    Feb 28, 2020
    Posts:
    72
    Some players of our game are running into what seems to be a deadlock in AssetBundleManager:

    Code (CSharp):
    1. >    ntdll.dll!NtWaitForSingleObject()    Unknown
    2.      KERNELBASE.dll!WaitForSingleObjectEx()    Unknown
    3.      UnityPlayer.dll!Baselib_SystemSemaphore_Acquire()    Unknown
    4.      UnityPlayer.dll!AssetBundleManager::AddAssetBundleLoadAssetOperation(class AssetBundleLoadAssetOperation *)    Unknown
    5.      UnityPlayer.dll!AssetBundleLoadAssetOperation::AssetBundleLoadAssetOperation(struct MemLabelId,class AssetBundle &,class core::basic_string<char,class core::StringStorageDefault<char> > const &,class ScriptingSystemTypeObjectPtr,bool)    Unknown
    6.      UnityPlayer.dll!AssetBundleLoadAssetOperation::LoadAsset(class AssetBundle &,class core::basic_string<char,class core::StringStorageDefault<char> > const &,class ScriptingSystemTypeObjectPtr,bool)    Unknown
    7.      UnityPlayer.dll!AssetBundle::LoadAssetAsync_Internal(class core::basic_string<char,class core::StringStorageDefault<char> >,class ScriptingSystemTypeObjectPtr,class ScriptingExceptionPtr *)    Unknown
    8.      UnityPlayer.dll!AssetBundle_CUSTOM_LoadAssetAsync_Internal(class ScriptingBackendNativeObjectPtrOpaque *,class ScriptingBackendNativeStringPtrOpaque *,class ScriptingBackendNativeObjectPtrOpaque *)    Unknown
    9.      [Frames may be missing, no binary loaded for GameAssembly.dll]  
    10.      GameAssembly.dll!00007ffbd9481a38()    Unknown
    Not sure what other thread is holding the lock but could be this one

    Code (CSharp):
    1. >    ntdll.dll!NtWaitForAlertByThreadId()    Unknown
    2.      ntdll.dll!RtlpWaitOnAddressWithTimeout()    Unknown
    3.      ntdll.dll!RtlpWaitOnAddress()    Unknown
    4.      ntdll.dll!RtlpWaitOnCriticalSection()    Unknown
    5.      ntdll.dll!RtlpEnterCriticalSectionContended()    Unknown
    6.      ntdll.dll!RtlEnterCriticalSection()    Unknown
    7.      UnityPlayer.dll!PersistentManager::GetSerializedTypesAndIdentifiers(struct dynamic_array<int,0> const &,struct dynamic_array<class Unity::Type const *,0> &,struct dynamic_array<struct SerializedObjectIdentifier,0> &)    Unknown
    8.      UnityPlayer.dll!SortPreloadObjects(struct dynamic_array<int,0> &,bool)    Unknown
    9.      UnityPlayer.dll!AssetBundleManager::CollectPreloadDataDependencies(class PPtr<class AssetBundle> &,struct dynamic_array<int,0> &,bool)    Unknown
    10.      UnityPlayer.dll!AssetBundleLoadAssetOperation::Perform(void)    Unknown
    11.      UnityPlayer.dll!PreloadManager::ProcessSingleOperation(void)    Unknown
    12.      UnityPlayer.dll!PreloadManager::Run(void)    Unknown
    13.      UnityPlayer.dll!PreloadManager::Run(void *)    Unknown
    14.      UnityPlayer.dll!Thread::RunThreadWrapper(void *)    Unknown
    15.      kernel32.dll!BaseThreadInitThunk()    Unknown
    16.      ntdll.dll!RtlUserThreadStart()    Unknown
    Unfortunately, we are unable to reproduce this locally but quite a few players are affected. Does anybody have an idea what we could do about this? We are on Unity 2019.4.15f1.
     
  2. georgerh

    georgerh

    Joined:
    Feb 28, 2020
    Posts:
    72
    Is it normal that co-routines can run during a synchronous Bundle.LoadAsset call? This can cause some unexpected bugs because code is interrupted that shouldn't be interruptible.

    My guess is that Bundle.LoadAsset -> Some Coroutine -> Bundle.LoadAsset causes a deadlock but I still have to prove that.

    Code (CSharp):
    1.      [Inline Frame] GameAssembly.dll!InterfaceFuncInvoker0<bool>::Invoke(unsigned int declaringInterface, Il2CppClass *) Line 195    C++
    2.      GameAssembly.dll!SetupCoroutine_InvokeMoveNext_m9106BA4E8AE0E794B17F184F1021A53F1D071F31(Il2CppObject * ___enumerator0, __int64 ___returnValueAddress1, const MethodInfo * method) Line 27734    C++
    3.      GameAssembly.dll!RuntimeInvoker_FalseVoid_t22962CB4C05B1D89B55A6E1139F0E87A90987017_RuntimeObject_Nullable_1_t0D03270832B3FFDDC0E7C2D89D4A0EA25376A1EB(void(*)() methodPointer, const MethodInfo * methodMetadata, void * obj, void * * args) Line 146437    C++
    4.      GameAssembly.dll!il2cpp::vm::Runtime::Invoke(const MethodInfo * method, void * obj, void * * params, Il2CppException * * exc) Line 545    C++
    5.      UnityPlayer.dll!scripting_method_invoke(class ScriptingMethodPtr,class ScriptingObjectPtr,struct ScriptingArguments &,class ScriptingExceptionPtr *,bool)    Unknown
    6.      UnityPlayer.dll!ScriptingInvocation::Invoke(class ScriptingExceptionPtr *,bool)    Unknown
    7.      UnityPlayer.dll!Coroutine::Run(bool *)    Unknown
    8.      UnityPlayer.dll!AsyncOperation::InvokeCoroutine(void)    Unknown
    9.      UnityPlayer.dll!UnityWebRequestAsyncOperation::InvokeCoroutine(void)    Unknown
    10.      UnityPlayer.dll!UnityWebRequestProto<class UnityWebRequestTransport,class AtomicRefCounter,class RedirectHelper,class ResponseHelper,class DownloadHandler,class UploadHandler,class CertificateHandler,class HeaderHelper,class AsyncOperation>::Job_InvokeCoroutine(class AsyncOperation *)    Unknown
    11.      UnityPlayer.dll!BackgroundJobQueue::ExecuteMainThreadJobs(void)    Unknown
    12.      UnityPlayer.dll!Texture::VerifyFileTextureUploadCompletion(void)    Unknown
    13.      UnityPlayer.dll!Texture2D::VerifyFileTextureUploadCompletion(void)    Unknown
    14.      UnityPlayer.dll!Texture2D::AwakeFromLoad(enum AwakeFromLoadMode)    Unknown
    15.      UnityPlayer.dll!AwakeFromLoadQueue::InvokePersistentManagerAwake(struct AwakeFromLoadQueue::Item *,unsigned int,enum AwakeFromLoadMode)    Unknown
    16.      UnityPlayer.dll!AwakeFromLoadQueue::PersistentManagerAwakeFromLoad(void)    Unknown
    17.      UnityPlayer.dll!PersistentManager::IntegrateAllThreadedObjects(void)    Unknown
    18.      UnityPlayer.dll!PersistentManager::LoadAndIntegrateAllPreallocatedObjects(enum PersistentManager::LockFlags)    Unknown
    19.      UnityPlayer.dll!PersistentManager::LoadObjects(int const *,int)    Unknown
    20.      UnityPlayer.dll!ProcessAssetBundleEntries()    Unknown
    21.      UnityPlayer.dll!LoadNamedObjectFromAssetBundle(class AssetBundle &,class core::basic_string<char,class core::StringStorageDefault<char> > const &,class ScriptingSystemTypeObjectPtr const &)    Unknown
    22.      UnityPlayer.dll!AssetBundle::LoadAsset_Internal(class core::basic_string<char,class core::StringStorageDefault<char> >,class ScriptingSystemTypeObjectPtr,class ScriptingExceptionPtr *)    Unknown
    23.      UnityPlayer.dll!AssetBundle_CUSTOM_LoadAsset_Internal(class ScriptingBackendNativeObjectPtrOpaque *,class ScriptingBackendNativeStringPtrOpaque *,class ScriptingBackendNativeObjectPtrOpaque *)    Unknown
    24.      GameAssembly.dll!AssetBundle_LoadAsset_TisRuntimeObject_m3209F473EEC424474B1053E66A1F09574719CB0B_gshared(AssetBundle_tCE287BAB693894C3991CDD308B92A8C9C4BD4C78 * __this, String_t * ___name0, const MethodInfo * method) Line 66687    C++
    25.  
     
  3. Id0n

    Id0n

    Joined:
    Jun 25, 2020
    Posts:
    3
    We have similair problem. During sync load from asset bundle unity invokes UnityWebRequests completion callbacks that haven't been called yet. Async operations competions during synchronous operation can cause any number of unexpected bugs.

    It is called side effect and should be fixed asap.
     
  4. georgerh

    georgerh

    Joined:
    Feb 28, 2020
    Posts:
    72