Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug Crashes during CI builds in accelerator

Discussion in 'Unity Accelerator' started by unity_EA3DE52BED9CD2E0DD40, Jun 16, 2023.

  1. unity_EA3DE52BED9CD2E0DD40

    unity_EA3DE52BED9CD2E0DD40

    Joined:
    Oct 25, 2022
    Posts:
    5
    Hi!

    We are having a lot of issues with accelerator crashing only our iOS builds, any advice how to proceed?
    It is a large project with versions
    2021.3.21f1 (1b156197d683)
    unity-accelerator-app-v1.0.941+g6b39b61

    We have multiple game projects using the same cache server, and the issue exists in only one of the projects.
    I am attaching some of the stack traces we had from yesterday. The reproduction rate in our CI is about 1 in 4 builds would fail.


    TcpProtobufClient - Transport error errorcode: 58, details: Can't send after socket shutdown
    3117AcceleratorClientConnectionCallback - disconnected - 10.10.2.225:10080
    3118TcpProtobufClient - Transport error errorcode: 58, details: Can't send after socket shutdown
    3119AcceleratorClientConnectionCallback - disconnected - 10.10.2.225:10080
    3120Failed to upload metadata for artifact(static dependencies=dfc07b4d058c3668b2a6f4db93fd3e4c) to cacheserver
    3121Failed to upload metadata for artifact(static dependencies=22a9f725a56982ea6d73a0adc3affa5b) to cacheserver
    3122Assertion failed on expression: 'it != m_SendMap.end()'
    3123
    3124=================================================================
    3125 Native Crash Reporting
    3126=================================================================
    3127Got a segv while executing native code. This usually indicates
    3128a fatal error in the mono runtime or one of the native libraries
    3129used by your application.
    3130=================================================================
    3131
    3132=================================================================
    3133 Native stacktrace:
    3134=================================================================
    3135 0x149f47710 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_dump_native_crash_info
    3136 0x149f09c90 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_handle_native_crash
    3137 0x149e924a8 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_sigsegv_signal_handler_debug
    3138 0x19c3602a4 - /usr/lib/system/libsystem_platform.dylib : _sigtramp
    3139 0x104518c58 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN18TcpProtobufSession8EndBatchEj
    3140 0x1045086d8 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4core16mutable_functionIFvvEE9DispatchTIZN9IOService10PostHelperIZN17AcceleratorClient19ScheduleNextRequestEvE4$_15EEvOT_NS4_25WrapCallableInFunctionTagEEUlvE_E8DispatchEv
    3141 0x10455858c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail18completion_handlerIN4core16mutable_functionIFvvEEENS_10io_context19basic_executor_typeINSt3__19allocatorIvEELj0EEEE11do_completeEPvPNS0_19scheduler_operationERKNS8_10error_codeEm
    3142 0x10453f69c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKNSt3__110error_codeE
    3143 0x10453f1f0 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler3runERNSt3__110error_codeE
    3144 0x104543498 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService3RunEb
    3145 0x104542b14 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService4Impl14ThreadFunctionEPv
    3146 0x102fd8488 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN6Thread16RunThreadWrapperEPv
    3147 0x19c33206c - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
    3148 0x19c32ce2c - /usr/lib/system/libsystem_pthread.dylib : thread_start
    3149
    3150=================================================================
    3151 Telemetry Dumper:
    3152=================================================================
    3153Pkilling 0x6141554688x from 0x6171963392x
    3154Pkilling 0x12754956288x from 0x6171963392x
    3155Pkilling 0x12759248896x from 0x6171963392x
    3156Pkilling 0x12763541504x from 0x6171963392x
    3157Pkilling 0x6170816512x from 0x6171963392x
    3158Pkilling 0x12664991744x from 0x6171963392x
    3159Pkilling 0x8449385728x from 0x6171963392x
    3160Pkilling 0x12752809984x from 0x6171963392x
    3161Pkilling 0x12870856704x from 0x6171963392x
    3162Pkilling 0x12757102592x from 0x6171963392x
    3163Pkilling 0x12761395200x from 0x6171963392x
    3164Pkilling 0x12628946944x from 0x6171963392x
    3165Pkilling 0x6168670208x from 0x6171963392x
    3166Pkilling 0x12765687808x from 0x6171963392x
    3167Obtained 20 stack frames.
    3168#0 0x0000014a091054 in mono_get_hazardous_pointer
    3169#1 0x00000149fe8618 in mono_jit_info_table_find_internal
    3170#2 0x00000149f06a74 in mini_jit_info_table_find_ext
    3171#3 0x00000149f05bf0 in mono_summarize_unmanaged_stack
    3172#4 0x0000014a03c704 in mono_threads_summarize_native_self
    3173#5 0x0000014a03ca60 in mono_threads_summarize_execute_internal
    3174#6 0x0000014a03d050 in mono_threads_summarize
    3175#7 0x00000149f47958 in mono_dump_native_crash_info
    3176#8 0x00000149f09c90 in mono_handle_native_crash
    3177#9 0x00000149e924a8 in mono_sigsegv_signal_handler_debug
    3178#10 0x0000019c3602a4 in _sigtramp
    3179#11 0x4533000104518c58 in TcpProtobufSession::EndBatch(unsigned int)
    3180#12 0x000001045086d8 in core::mutable_function<void ()>::DispatchT<void IOService::postHelper<AcceleratorClient::ScheduleNextRequest()::$_15>(AcceleratorClient::ScheduleNextRequest()::$_15&&, IOService::WrapCallableInFunctionTag)::'lambda'()>::Dispatch()
    3181#13 0x0000010455858c in asio::detail::completion_handler<core::mutable_function<void ()>, asio::io_context::basic_executor_type<std::__1::allocator<void>, 0u> >::do_complete(void*, asio::detail::scheduler_operation*, std::__1::error_code const&, unsigned long)
    3182#14 0x0000010453f69c in asio::detail::scheduler::do_run_one(asio::detail::conditionally_enabled_mutex::scoped_lock&, asio::detail::scheduler_thread_info&, std::__1::error_code const&)
    3183#15 0x0000010453f1f0 in asio::detail::scheduler::run(std::__1::error_code&)
    3184#16 0x00000104543498 in IOService::Run(bool)
    3185#17 0x00000104542b14 in IOService::Impl::ThreadFunction(void*)
    3186#18 0x00000102fd8488 in Thread::RunThreadWrapper(void*)
    3187#19 0x0000019c33206c in _pthread_start
    3188Mono process hang detected, sending kill signal to pid 95707


    TcpProtobufClient - Transport error errorcode: 58, details: Can't send after socket shutdown
    2533AcceleratorClientConnectionCallback - disconnected - 10.10.2.225:10080
    2534TcpProtobufClient - Transport error errorcode: 58, details: Can't send after socket shutdown
    2535AcceleratorClientConnectionCallback - disconnected - 10.10.2.225:10080
    2536Assertion failed on expression: 'it != m_SendMap.end()'
    2537
    2538=================================================================
    2539 Native Crash Reporting
    2540=================================================================
    2541Got a bus while executing native code. This usually indicates
    2542a fatal error in the mono runtime or one of the native libraries
    2543used by your application.
    2544=================================================================
    2545
    2546=================================================================
    2547 Native stacktrace:
    2548=================================================================
    2549 0x1500c3710 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_dump_native_crash_info
    2550 0x150085c90 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_handle_native_crash
    2551 0x15000e4a8 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_sigsegv_signal_handler_debug
    2552 0x1844c6a84 - /usr/lib/system/libsystem_platform.dylib : _sigtramp
    2553 0x107530d0c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZNK5unity6google8protobuf11MessageLite23SerializePartialToArrayEPvi
    2554 0x107530d0c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZNK5unity6google8protobuf11MessageLite23SerializePartialToArrayEPvi
    2555 0x1064e4580 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN18TcpProtobufSession16SendMessageAsyncERKN3Rpc7SegmentEj
    2556 0x1064e415c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN17TcpProtobufClient23SendMessageBatchedAsyncEjRKN3Rpc7SegmentEON4core8functionIFvS3_EEE
    2557 0x1064c798c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN17AcceleratorClient11InternalGetERKN4core12basic_stringIcNS0_20StringStorageDefaultIcEEEES6_R7IWriterONS0_8functionIFvbEEEj
    2558 0x1064d462c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4core16mutable_functionIFvvEE9DispatchTIZN9IOService10PostHelperIZN17AcceleratorClient19ScheduleNextRequestEvE4$_15EEvOT_NS4_25WrapCallableInFunctionTagEEUlvE_E8DispatchEv
    2559 0x10652458c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail18completion_handlerIN4core16mutable_functionIFvvEEENS_10io_context19basic_executor_typeINSt3__19allocatorIvEELj0EEEE11do_completeEPvPNS0_19scheduler_operationERKNS8_10error_codeEm
    2560 0x10650b69c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKNSt3__110error_codeE
    2561 0x10650b1f0 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler3runERNSt3__110error_codeE
    2562 0x10650f498 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService3RunEb
    2563 0x10650eb14 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService4Impl14ThreadFunctionEPv
    2564 0x104fa4488 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN6Thread16RunThreadWrapperEPv
    2565 0x184497fa8 - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
    2566 0x184492da0 - /usr/lib/system/libsystem_pthread.dylib : thread_start
    2567
    2568=================================================================
    2569 Telemetry Dumper:
    2570=================================================================
    2571Pkilling 0x6403813376x from 0x6407106560x
    2572Pkilling 0x12585037824x from 0x6407106560x
    2573Pkilling 0x12595769344x from 0x6407106560x
    2574Pkilling 0x12571947008x from 0x6407106560x
    2575Pkilling 0x8045697856x from 0x6407106560x
    2576Pkilling 0x6405959680x from 0x6407106560x
    2577Pkilling 0x12587184128x from 0x6407106560x
    2578Pkilling 0x12574093312x from 0x6407106560x
    2579Pkilling 0x12589330432x from 0x6407106560x
    2580Pkilling 0x12454391808x from 0x6407106560x
    2581Pkilling 0x12591476736x from 0x6407106560x
    2582Pkilling 0x6108442624x from 0x6407106560x
    2583Pkilling 0x12700676096x from 0x6407106560x
    2584Pkilling 0x12593623040x from 0x6407106560x
    2585Obtained 24 stack frames.
    2586#0 0x0000015020d054 in mono_get_hazardous_pointer
    2587#1 0x00000150164618 in mono_jit_info_table_find_internal
    2588#2 0x00000150082a74 in mini_jit_info_table_find_ext
    2589#3 0x00000150081bf0 in mono_summarize_unmanaged_stack
    2590#4 0x000001501b8704 in mono_threads_summarize_native_self
    2591#5 0x000001501b8a60 in mono_threads_summarize_execute_internal
    2592#6 0x000001501b9050 in mono_threads_summarize
    2593#7 0x000001500c3958 in mono_dump_native_crash_info
    2594#8 0x00000150085c90 in mono_handle_native_crash
    2595#9 0x0000015000e4a8 in mono_sigsegv_signal_handler_debug
    2596#10 0x000001844c6a84 in _sigtramp
    2597#11 0x6169800107530d0c in unity::google::protobuf::MessageLite::SerializePartialToArray(void*, int) const
    2598#12 0x00000107530d0c in unity::google::protobuf::MessageLite::SerializePartialToArray(void*, int) const
    2599#13 0x000001064e4580 in TcpProtobufSession::SendMessageAsync(Rpc::Segment const&, unsigned int)
    2600#14 0x000001064e415c in TcpProtobufClient::SendMessageBatchedAsync(unsigned int, Rpc::Segment const&, core::function<void (Rpc::Segment const&)>&&)
    2601#15 0x000001064c798c in AcceleratorClient::InternalGet(core::basic_string<char, core::StringStorageDefault<char>> const&, core::basic_string<char, core::StringStorageDefault<char>> const&, IWriter&, core::function<void (bool)>&&, unsigned int)
    2602#16 0x000001064d462c in core::mutable_function<void ()>::DispatchT<void IOService::postHelper<AcceleratorClient::ScheduleNextRequest()::$_15>(AcceleratorClient::ScheduleNextRequest()::$_15&&, IOService::WrapCallableInFunctionTag)::'lambda'()>::Dispatch()
    2603#17 0x0000010652458c in asio::detail::completion_handler<core::mutable_function<void ()>, asio::io_context::basic_executor_type<std::__1::allocator<void>, 0u>>::do_complete(void*, asio::detail::scheduler_operation*, std::__1::error_code const&, unsigned long)
    2604#18 0x0000010650b69c in asio::detail::scheduler::do_run_one(asio::detail::conditionally_enabled_mutex::scoped_lock&, asio::detail::scheduler_thread_info&, std::__1::error_code const&)
    2605#19 0x0000010650b1f0 in asio::detail::scheduler::run(std::__1::error_code&)
    2606#20 0x0000010650f498 in IOService::Run(bool)
    2607#21 0x0000010650eb14 in IOService::Impl::ThreadFunction(void*)
    2608#22 0x00000104fa4488 in Thread::RunThreadWrapper(void*)
    2609#23 0x00000184497fa8 in _pthread_start
    2610Mono process hang detected, sending kill signal to pid 70229

    Querying for cacheable assets in Cache Server:
    31057 899a2ea689d50b62ba7dd9d89e2b849a:Assets/Prefabs/Areas/Map.prefab
    31058TcpProtobufClient - Transport error errorcode: 60, details: Operation timed out
    31059AcceleratorClientConnectionCallback - disconnected - 10.10.2.225:10080
    31060Failed to upload artifact(artifact id=66d610ec036d4496052a83c3adcd5ed2, static dependencies=b9c88bb21589bc1f656cfcf37ad12d24, content hash=53ba52a4b21dc094947b9dc7bb14a8ce) to cacheserver
    31061Failed to upload artifact(artifact id=0ae00dbcb67403c67c46f64e925d0e65, static dependencies=d7ab85528ca90dd0cbcecdc1aaf018ce, content hash=53ba52a4b21dc094947b9dc7bb14a8ce) to cacheserver
    31062Assertion failed on expression: 'it != m_SendMap.end()'
    31063
    31064=================================================================
    31065 Native Crash Reporting
    31066=================================================================
    31067Got a segv while executing native code. This usually indicates
    31068a fatal error in the mono runtime or one of the native libraries
    31069used by your application.
    31070=================================================================
    31071
    31072=================================================================
    31073 Native stacktrace:
    31074=================================================================
    31075 0x2d6547710 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_dump_native_crash_info
    31076 0x2d6509c90 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_handle_native_crash
    31077 0x2d64924a8 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_sigsegv_signal_handler_debug
    31078 0x1844c6a84 - /usr/lib/system/libsystem_platform.dylib : _sigtramp
    31079 0x1046b0514 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN18TcpProtobufSession16SendMessageAsyncERKN3Rpc7SegmentEj
    31080 0x1046b015c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN17TcpProtobufClient23SendMessageBatchedAsyncEjRKN3Rpc7SegmentEON4core8functionIFvS3_EEE
    31081 0x10469398c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN17AcceleratorClient11InternalGetERKN4core12basic_stringIcNS0_20StringStorageDefaultIcEEEES6_R7IWriterONS0_8functionIFvbEEEj
    31082 0x1046a062c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4core16mutable_functionIFvvEE9DispatchTIZN9IOService10PostHelperIZN17AcceleratorClient19ScheduleNextRequestEvE4$_15EEvOT_NS4_25WrapCallableInFunctionTagEEUlvE_E8DispatchEv
    31083 0x1046f058c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail18completion_handlerIN4core16mutable_functionIFvvEEENS_10io_context19basic_executor_typeINSt3__19allocatorIvEELj0EEEE11do_completeEPvPNS0_19scheduler_operationERKNS8_10error_codeEm
    31084 0x1046d769c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKNSt3__110error_codeE
    31085 0x1046d71f0 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler3runERNSt3__110error_codeE
    31086 0x1046db498 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService3RunEb
    31087 0x1046dab14 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService4Impl14ThreadFunctionEPv
    31088 0x103170488 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN6Thread16RunThreadWrapperEPv
    31089 0x184497fa8 - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
    31090 0x184492da0 - /usr/lib/system/libsystem_pthread.dylib : thread_start
    31091
    31092=================================================================
    31093 Telemetry Dumper:
    31094=================================================================
    31095Pkilling 0x6413004800x from 0x12197376000x
    31096Pkilling 0x13081800704x from 0x12197376000x
    31097Pkilling 0x12966998016x from 0x12197376000x
    31098Pkilling 0x12971290624x from 0x12197376000x
    31099Pkilling 0x12975583232x from 0x12197376000x
    31100Pkilling 0x12770373632x from 0x12197376000x
    31101Pkilling 0x12979875840x from 0x12197376000x
    31102Pkilling 0x12196229120x from 0x12197376000x
    31103Pkilling 0x12522647552x from 0x12197376000x
    31104Pkilling 0x12969144320x from 0x12197376000x
    31105Pkilling 0x12973436928x from 0x12197376000x
    31106Pkilling 0x12977729536x from 0x12197376000x
    31107Pkilling 0x12982022144x from 0x12197376000x
    31108Pkilling 0x12194082816x from 0x12197376000x
    31109Pkilling 0x8045697856x from 0x12197376000x
    31110Obtained 22 stack frames.
    31111#0 0x000002d6691054 in mono_get_hazardous_pointer
    31112#1 0x000002d65e8618 in mono_jit_info_table_find_internal
    31113#2 0x000002d6506a74 in mini_jit_info_table_find_ext
    31114#3 0x000002d6505bf0 in mono_summarize_unmanaged_stack
    31115#4 0x000002d663c704 in mono_threads_summarize_native_self
    31116#5 0x000002d663ca60 in mono_threads_summarize_execute_internal
    31117#6 0x000002d663d050 in mono_threads_summarize
    31118#7 0x000002d6547958 in mono_dump_native_crash_info
    31119#8 0x000002d6509c90 in mono_handle_native_crash
    31120#9 0x000002d64924a8 in mono_sigsegv_signal_handler_debug
    31121#10 0x000001844c6a84 in _sigtramp
    31122#11 0xde1a0001046b0514 in TcpProtobufSession::SendMessageAsync(Rpc::Segment const&, unsigned int)
    31123#12 0x000001046b015c in TcpProtobufClient::SendMessageBatchedAsync(unsigned int, Rpc::Segment const&, core::function<void (Rpc::Segment const&)>&&)
    31124#13 0x0000010469398c in AcceleratorClient::InternalGet(core::basic_string<char, core::StringStorageDefault<char>> const&, core::basic_string<char, core::StringStorageDefault<char>> const&, IWriter&, core::function<void (bool)>&&, unsigned int)
    31125#14 0x000001046a062c in core::mutable_function<void ()>::DispatchT<void IOService::postHelper<AcceleratorClient::ScheduleNextRequest()::$_15>(AcceleratorClient::ScheduleNextRequest()::$_15&&, IOService::WrapCallableInFunctionTag)::'lambda'()>::Dispatch()
    31126#15 0x000001046f058c in asio::detail::completion_handler<core::mutable_function<void ()>, asio::io_context::basic_executor_type<std::__1::allocator<void>, 0u>>::do_complete(void*, asio::detail::scheduler_operation*, std::__1::error_code const&, unsigned long)
    31127#16 0x000001046d769c in asio::detail::scheduler::do_run_one(asio::detail::conditionally_enabled_mutex::scoped_lock&, asio::detail::scheduler_thread_info&, std::__1::error_code const&)
    31128#17 0x000001046d71f0 in asio::detail::scheduler::run(std::__1::error_code&)
    31129#18 0x000001046db498 in IOService::Run(bool)
    31130#19 0x000001046dab14 in IOService::Impl::ThreadFunction(void*)
    31131#20 0x00000103170488 in Thread::RunThreadWrapper(void*)
    31132#21 0x00000184497fa8 in _pthread_start
    31133Mono process hang detected, sending kill signal to pid 75606


    Querying for cacheable assets in Cache Server:
    31132 899a2ea689d50b62ba7dd9d89e2b849a:Assets/Prefabs/Areas/Map.prefab
    31133TcpProtobufClient - Transport error errorcode: 60, details: Operation timed out
    31134AcceleratorClientConnectionCallback - disconnected - 10.10.2.225:10080
    31135Failed to upload artifact(artifact id=e8388e2d81a7a855e7fd7d1c52ed15f6, static dependencies=cc66e5eb3c97d32b8388e560dee9e181, content hash=53ba52a4b21dc094947b9dc7bb14a8ce) to cacheserver
    31136Failed to upload artifact(artifact id=cc8289d67f48614d0fd09291d81633f5, static dependencies=6823d9cbbe3417ef5f711314b3c9ee6c, content hash=53ba52a4b21dc094947b9dc7bb14a8ce) to cacheserver
    31137Assertion failed on expression: 'it != m_SendMap.end()'
    31138
    31139=================================================================
    31140 Native Crash Reporting
    31141=================================================================
    31142Got a segv while executing native code. This usually indicates
    31143a fatal error in the mono runtime or one of the native libraries
    31144used by your application.
    31145=================================================================
    31146
    31147=================================================================
    31148 Native stacktrace:
    31149=================================================================
    31150 0x13e147710 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_dump_native_crash_info
    31151 0x13e109c90 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_handle_native_crash
    31152 0x13e0924a8 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/Frameworks/MonoBleedingEdge/MonoEmbedRuntime/osx/libmonobdwgc-2.0.dylib : mono_sigsegv_signal_handler_debug
    31153 0x1a3f842a4 - /usr/lib/system/libsystem_platform.dylib : _sigtramp
    31154 0x104a60514 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN18TcpProtobufSession16SendMessageAsyncERKN3Rpc7SegmentEj
    31155 0x104a6015c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN17TcpProtobufClient23SendMessageBatchedAsyncEjRKN3Rpc7SegmentEON4core8functionIFvS3_EEE
    31156 0x104a4398c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN17AcceleratorClient11InternalGetERKN4core12basic_stringIcNS0_20StringStorageDefaultIcEEEES6_R7IWriterONS0_8functionIFvbEEEj
    31157 0x104a5062c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4core16mutable_functionIFvvEE9DispatchTIZN9IOService10PostHelperIZN17AcceleratorClient19ScheduleNextRequestEvE4$_15EEvOT_NS4_25WrapCallableInFunctionTagEEUlvE_E8DispatchEv
    31158 0x104aa058c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail18completion_handlerIN4core16mutable_functionIFvvEEENS_10io_context19basic_executor_typeINSt3__19allocatorIvEELj0EEEE11do_completeEPvPNS0_19scheduler_operationERKNS8_10error_codeEm
    31159 0x104a8769c - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKNSt3__110error_codeE
    31160 0x104a871f0 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN4asio6detail9scheduler3runERNSt3__110error_codeE
    31161 0x104a8b498 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService3RunEb
    31162 0x104a8ab14 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN9IOService4Impl14ThreadFunctionEPv
    31163 0x103520488 - /Applications/Unity/Hub/Editor/2021.3.21f1/Unity.app/Contents/MacOS/Unity : _ZN6Thread16RunThreadWrapperEPv
    31164 0x1a3f5606c - /usr/lib/system/libsystem_pthread.dylib : _pthread_start
    31165 0x1a3f50e2c - /usr/lib/system/libsystem_pthread.dylib : thread_start
    31166
    31167=================================================================
    31168 Telemetry Dumper:
    31169=================================================================
    31170Pkilling 0x8579556608x from 0x6233534464x
    31171Pkilling 0x11773440000x from 0x6233534464x
    31172Pkilling 0x6232387584x from 0x6233534464x
    31173Pkilling 0x11777732608x from 0x6233534464x
    31174Pkilling 0x11782025216x from 0x6233534464x
    31175Pkilling 0x11786317824x from 0x6233534464x
    31176Pkilling 0x6141751296x from 0x6233534464x
    31177Pkilling 0x11810746368x from 0x6233534464x
    31178Pkilling 0x6230241280x from 0x6233534464x
    31179Pkilling 0x11775586304x from 0x6233534464x
    31180Pkilling 0x11779878912x from 0x6233534464x
    31181Pkilling 0x11784171520x from 0x6233534464x
    31182Pkilling 0x11788464128x from 0x6233534464x
    31183Pkilling 0x11348242432x from 0x6233534464x
    31184Obtained 22 stack frames.
    31185#0 0x0000013e291054 in mono_get_hazardous_pointer
    31186#1 0x0000013e1e8618 in mono_jit_info_table_find_internal
    31187#2 0x0000013e106a74 in mini_jit_info_table_find_ext
    31188#3 0x0000013e105bf0 in mono_summarize_unmanaged_stack
    31189#4 0x0000013e23c704 in mono_threads_summarize_native_self
    31190#5 0x0000013e23ca60 in mono_threads_summarize_execute_internal
    31191#6 0x0000013e23d050 in mono_threads_summarize
    31192#7 0x0000013e147958 in mono_dump_native_crash_info
    31193#8 0x0000013e109c90 in mono_handle_native_crash
    31194#9 0x0000013e0924a8 in mono_sigsegv_signal_handler_debug
    31195#10 0x000001a3f842a4 in _sigtramp
    31196#11 0x9876000104a60514 in TcpProtobufSession::SendMessageAsync(Rpc::Segment const&, unsigned int)
    31197#12 0x00000104a6015c in TcpProtobufClient::SendMessageBatchedAsync(unsigned int, Rpc::Segment const&, core::function<void (Rpc::Segment const&)>&&)
    31198#13 0x00000104a4398c in AcceleratorClient::InternalGet(core::basic_string<char, core::StringStorageDefault<char> > const&, core::basic_string<char, core::StringStorageDefault<char> > const&, IWriter&, core::function<void (bool)>&&, unsigned int)
    31199#14 0x00000104a5062c in core::mutable_function<void ()>::DispatchT<void IOService::postHelper<AcceleratorClient::ScheduleNextRequest()::$_15>(AcceleratorClient::ScheduleNextRequest()::$_15&&, IOService::WrapCallableInFunctionTag)::'lambda'()>::Dispatch()
    31200#15 0x00000104aa058c in asio::detail::completion_handler<core::mutable_function<void ()>, asio::io_context::basic_executor_type<std::__1::allocator<void>, 0u> >::do_complete(void*, asio::detail::scheduler_operation*, std::__1::error_code const&, unsigned long)
    31201#16 0x00000104a8769c in asio::detail::scheduler::do_run_one(asio::detail::conditionally_enabled_mutex::scoped_lock&, asio::detail::scheduler_thread_info&, std::__1::error_code const&)
    31202#17 0x00000104a871f0 in asio::detail::scheduler::run(std::__1::error_code&)
    31203#18 0x00000104a8b498 in IOService::Run(bool)
    31204#19 0x00000104a8ab14 in IOService::Impl::ThreadFunction(void*)
    31205#20 0x00000103520488 in Thread::RunThreadWrapper(void*)
    31206#21 0x000001a3f5606c in _pthread_start
    31207Mono process hang detected, sending kill signal to pid 25390