Bug APIUpdater hard fails in batch mode causing Unity to fail to build

Keywords: batchmode, api_updater

Context:

We have an editor module that downloads dependencies into our projects.
When we kick start a batch mode build, if a dependency is missing, the first pass compilation usually fails when an assembly is missing and our dependency manager kicks in and download the missing dependencies. This then triggers a new recompilation / APIUpdate process, and everything is fine and dandy.

Problem:

When upgrading one of our projects to 2020.3.26f1 from 2019.4.31f1, the APIUpdater hard fails in batch mode during the first compilation pass.

If we open the project in non batch mode, the failure doesn't happen.

Code (CSharp):

1. COMMAND LINE ARGUMENTS:
2. /Applications/Unity_2020.3.26f1/Unity.app/Contents/MacOS/Unity
3. -logFile
4. /Users/agent007/workspace/PROJECT/u3d.log
5. -projectPath
6. /Users/agent007/workspace/PROJECT
7. -quit
8. -batchmode
9. -accept-apiupdate
10. -buildTarget
11. android
12. -executeMethod
13. WWTK.SimpleBuild.PerformAndroidBuild
14. -username
15. user@domain.com
16. -password
17. (hidden)
18.  
19. [...]
20. [ScriptCompilation] Recompiling all scripts because: InitialRefresh: Cached user assemblies were invalid
21. [...]
22. - Starting compile Library/ScriptAssemblies/Assembly-CSharp-firstpass.dll
23. - Finished compile Library/ScriptAssemblies/Assembly-CSharp-firstpass.dll in 0.502862 seconds
24. - Starting compile Library/ScriptAssemblies/Assembly-CSharp.dll
25. - Starting compile Library/ScriptAssemblies/Assembly-CSharp-Editor-firstpass.dll
26. - Finished compile Library/ScriptAssemblies/Assembly-CSharp-Editor-firstpass.dll in 0.367607 seconds
27. [...]
28. -----Compiler Commandline Arguments:
29. Filename: /Applications/Unity_2020.3.26f1/Unity.app/Contents/Tools/RoslynScripts/unity_csc.sh
30. Arguments: /noconfig @Temp/UnityTempFile-898de5e6343b94e4f9afaec6272e72e5
31. Responsefile: Temp/UnityTempFile-898de5e6343b94e4f9afaec6272e72e5 Contents:
32.  
33. -----CompilerOutput:-stdout--exitcode: 1--compilationhadfailure: True--outfile: Temp/Assembly-CSharp.dll
34. Assets/XXX/XXX/XXXX.cs(3,17): error CS0234: The type or namespace name 'YYY' does not exist in the namespace 'XXXX' (are you missing an assembly reference?)
35. [...]
36. [Script Updater] Unable to check errors listed in `Temp/ApiUpdater_CheckMovedFrom8e36420c1b5d74e0ba480c65f12b1314`. File contents:
37. 5027174928
38. 7136607203321344161
39. /Users/agent007/workspace/PROJECT/Assets/XXX/XXX/XXXX.cs
40. 3
41. 17
42. XXXX.YYY
43. 0
44. 0
45. 5027175144
46. 7136607203321344161
47. [...]
48. -----------------------
49.  
50. Temp/UnityTempFile-2ada0f263f421480db1bd65d992b2243 (types):
51. Search Paths:20
52. /Applications/Unity_2020.3.26f1/Unity.app/Contents/Managed
53. /Applications/Unity_2020.3.26f1/Unity.app/Contents/Managed/UnityEngine
54. /Users/agent007/workspace/PROJECT/Assets
55. [...]
56. -----------------------
57.  
58. Program exited with code = 1
59. Program Output:
60.  
61. Unhandled Exception:
62. System.IO.FileNotFoundException: Cannot resolve dependency to assembly because it has not been preloaded. When using the ReflectionOnly APIs, dependent assemblies must be pre-loaded or loaded on demand through the ReflectionOnlyAssemblyResolve event.
63. File name: 'System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
64.   at (wrapper managed-to-native) System.MonoCustomAttrs.GetCustomAttributesDataInternal(System.Reflection.ICustomAttributeProvider)
65.   at System.MonoCustomAttrs.GetCustomAttributesData (System.Reflection.ICustomAttributeProvider obj) [0x0000e] in <695d1cc93cca45069c528c15c9fdd749>:0
66.   at System.Reflection.CustomAttributeData.GetCustomAttributesInternal (System.RuntimeType target) [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
67.   at System.RuntimeType.GetCustomAttributesData () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
68.   at System.Reflection.MemberInfo.get_CustomAttributes () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
69.   at APIUpdater.NonObsoleteApiUpdaterDetector.ExtraInfoParser+<>c__DisplayClass3_1.<LoadTypesWithMovedFromAttributeAsync>b__3 (System.Type t) [0x00009] in <6358200448ef46218e7be4a1b6be469f>:0
70.   at System.Linq.Enumerable+WhereArrayIterator`1[TSource].ToList () [0x00019] in <351e49e2a5bf4fd6beabb458ce2255f3>:0
71.   at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0001f] in <351e49e2a5bf4fd6beabb458ce2255f3>:0
72.   at APIUpdater.NonObsoleteApiUpdaterDetector.ExtraInfoParser+<ParseTypesWithMovedFromAttributeAsync>d__2.MoveNext () [0x0010c] in <6358200448ef46218e7be4a1b6be469f>:0
73. --- End of stack trace from previous location where exception was thrown ---
74.   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <695d1cc93cca45069c528c15c9fdd749>:0
75.   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <695d1cc93cca45069c528c15c9fdd749>:0
76.   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <695d1cc93cca45069c528c15c9fdd749>:0
77.   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <695d1cc93cca45069c528c15c9fdd749>:0
78.   at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
79.   at APIUpdater.NonObsoleteApiUpdaterDetector.ExtraInfoParser+<ParseAsync>d__0.MoveNext () [0x000ba] in <6358200448ef46218e7be4a1b6be469f>:0
80. --- End of stack trace from previous location where exception was thrown ---
81.   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <695d1cc93cca45069c528c15c9fdd749>:0
82.   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <695d1cc93cca45069c528c15c9fdd749>:0
83.   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <695d1cc93cca45069c528c15c9fdd749>:0
84.   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <695d1cc93cca45069c528c15c9fdd749>:0
85.   at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
86.   at APIUpdater.NonObsoleteApiUpdaterDetector.Program+<Main>d__0.MoveNext () [0x003af] in <6358200448ef46218e7be4a1b6be469f>:0
87. --- End of stack trace from previous location where exception was thrown ---
88.   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <695d1cc93cca45069c528c15c9fdd749>:0
89.   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <695d1cc93cca45069c528c15c9fdd749>:0
90.   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <695d1cc93cca45069c528c15c9fdd749>:0
91.   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <695d1cc93cca45069c528c15c9fdd749>:0
92.   at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
93.   at APIUpdater.NonObsoleteApiUpdaterDetector.Program.<Main> (System.String[] args) [0x0000c] in <6358200448ef46218e7be4a1b6be469f>:0
94. [ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Cannot resolve dependency to assembly because it has not been preloaded. When using the ReflectionOnly APIs, dependent assemblies must be pre-loaded or loaded on demand through the ReflectionOnlyAssemblyResolve event.
95. File name: 'System.Xml, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
96.   at (wrapper managed-to-native) System.MonoCustomAttrs.GetCustomAttributesDataInternal(System.Reflection.ICustomAttributeProvider)
97.   at System.MonoCustomAttrs.GetCustomAttributesData (System.Reflection.ICustomAttributeProvider obj) [0x0000e] in <695d1cc93cca45069c528c15c9fdd749>:0
98.   at System.Reflection.CustomAttributeData.GetCustomAttributesInternal (System.RuntimeType target) [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
99.   at System.RuntimeType.GetCustomAttributesData () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
100.   at System.Reflection.MemberInfo.get_CustomAttributes () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
101.   at APIUpdater.NonObsoleteApiUpdaterDetector.ExtraInfoParser+<>c__DisplayClass3_1.<LoadTypesWithMovedFromAttributeAsync>b__3 (System.Type t) [0x00009] in <6358200448ef46218e7be4a1b6be469f>:0
102.   at System.Linq.Enumerable+WhereArrayIterator`1[TSource].ToList () [0x00019] in <351e49e2a5bf4fd6beabb458ce2255f3>:0
103.   at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0001f] in <351e49e2a5bf4fd6beabb458ce2255f3>:0
104.   at APIUpdater.NonObsoleteApiUpdaterDetector.ExtraInfoParser+<ParseTypesWithMovedFromAttributeAsync>d__2.MoveNext () [0x0010c] in <6358200448ef46218e7be4a1b6be469f>:0
105. --- End of stack trace from previous location where exception was thrown ---
106.   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <695d1cc93cca45069c528c15c9fdd749>:0
107.   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <695d1cc93cca45069c528c15c9fdd749>:0
108.   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <695d1cc93cca45069c528c15c9fdd749>:0
109.   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <695d1cc93cca45069c528c15c9fdd749>:0
110.   at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
111.   at APIUpdater.NonObsoleteApiUpdaterDetector.ExtraInfoParser+<ParseAsync>d__0.MoveNext () [0x000ba] in <6358200448ef46218e7be4a1b6be469f>:0
112. --- End of stack trace from previous location where exception was thrown ---
113.   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <695d1cc93cca45069c528c15c9fdd749>:0
114.   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <695d1cc93cca45069c528c15c9fdd749>:0
115.   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <695d1cc93cca45069c528c15c9fdd749>:0
116.   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <695d1cc93cca45069c528c15c9fdd749>:0
117.   at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
118.   at APIUpdater.NonObsoleteApiUpdaterDetector.Program+<Main>d__0.MoveNext () [0x003af] in <6358200448ef46218e7be4a1b6be469f>:0
119. --- End of stack trace from previous location where exception was thrown ---
120.   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <695d1cc93cca45069c528c15c9fdd749>:0
121.   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <695d1cc93cca45069c528c15c9fdd749>:0
122.   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <695d1cc93cca45069c528c15c9fdd749>:0
123.   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <695d1cc93cca45069c528c15c9fdd749>:0
124.   at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <695d1cc93cca45069c528c15c9fdd749>:0
125.   at APIUpdater.NonObsoleteApiUpdaterDetector.Program.<Main> (System.String[] args) [0x0000c] in <6358200448ef46218e7be4a1b6be469f>:0
126. UnityEngine.StackTraceUtility:ExtractStackTrace () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/StackTrace.cs:37)
127. UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
128. UnityEngine.Logger:Log (UnityEngine.LogType,object)
129. UnityEngine.Debug:Log (object)
130. UnityEditor.Scripting.Compilers.APIUpdaterHelper:FilterReferenceToTypeWithChangedNamespaceErrors (string) (at /Users/bokken/buildslave/unity/build/Editor/Mono/Scripting/Compilers/APIUpdaterHelper.cs:108)
131.  
132. (Filename: /Users/bokken/buildslave/unity/build/Editor/Mono/Scripting/Compilers/APIUpdaterHelper.cs Line: 108)

Analysis

The APIUpdater definitively complains about missing references and assemblies.

We searched for the code of the APIUpdaterHelper but the UnityCsReference doesn't seem to contain the exact version included in 2020.3.26f1

The closest we found with this class is
https://github.com/Unity-Technologi...o/Scripting/Compilers/APIUpdaterHelper.cs#L27

but there's no code referencing the line 108 where it fails.

Questions

• why is the APIUpdater hard failing here?
• is there a known version of 2020 LTS that works with our context?
• if not, is there a work-around to this issue?

 

Hi there jerome-lacoste. We have same error. Did you fix this ?

 

Hi, Adriano!
We did just like you said. Here is the log file
https://dropmefiles.com/TA2lH

we have the same problem while opening and while building our project

Unity version = 2020.3.34f1
docker image = unityci/editor:ubuntu-2020.3.34f1-android-1
os version = Ubuntu 18.04.6 LTS (Bionic Beaver)
docker version = 20.10.18

 

Hi, Adriano!
I hope you can help us.
All information is provided in the post above.
If you need some additional info, please contact me

 

Hello, Adriano
Thanks for your answer
we've created a small pet project in wich we try to add pieces of our global project in order to get the bug, but we've succeded yet
Can you please tell us what this method (Assembly.ReflectionOnlyLoadFrom()) does? Maybe we'll find the solution this way
We need a build for linux x64 from you in order to try without using this method (better it would be branched from 2020.3.34f1 version)

Pavel