Search Unity

IL2CPP: compiler limit: token overflowed internal buffer

Discussion in 'Windows' started by EusebiuMarcuUCB, Jul 15, 2019.

  1. EusebiuMarcuUCB

    EusebiuMarcuUCB

    Joined:
    Sep 20, 2018
    Posts:
    9
    I am building an UWP and after the VS solution is generated, I get this error when building it.

    Code (CSharp):
    1.  1>C:\New folder\Il2CppOutputProject\Source\il2cppOutput\Il2CppTypeDefinitions.c(10319): fatal error C1064: compiler limit: token overflowed internal buffer
    Any ideas?

    Code (CSharp):
    1.  1>Building GameAssembly.dll with MsvcWinRtToolChain
    2. 1>    Msvc Install Version: 15.0
    3. 1>    Msvc Install SDK Directory: C:\Program Files (x86)\Windows Kits\10
    4. 1>    Msvc Linker Path: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\bin\HostX64\ARM\link.exe
    5. 1>    Msvc Compiler Path: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\bin\HostX64\ARM\cl.exe
    6. 1>
    7. 1>    Output directory: C:\New folder\build\bin\ARM\Debug
    8. 1>    Cache directory: C:\New folder\build\obj\il2cppOutputProject\ARM\Debug
    9. 1>il2cpp.exe didn't catch exception: Unity.IL2CPP.Building.BuilderFailedException: Il2CppTypeDefinitions.c
    10. 1>C:\New folder\Il2CppOutputProject\Source\il2cppOutput\Il2CppTypeDefinitions.c(10319): fatal error C1064: compiler limit: token overflowed internal buffer
    11. 1>
    12. 1>Invocation was: Executable: "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\bin\HostX64\ARM\cl.exe"
    13. 1>
    14. 1>Unhandled Exception: Unity.IL2CPP.Building.BuilderFailedException: Il2CppTypeDefinitions.c
    15. 1>C:\New folder\Il2CppOutputProject\Source\il2cppOutput\Il2CppTypeDefinitions.c(10319): fatal error C1064: compiler limit: token overflowed internal buffer
    16. 1>
    17. 1>Invocation was: Executable: "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\bin\HostX64\ARM\cl.exe"
    18. 1>Arguments: "C:\New folder\Il2CppOutputProject\Source\il2cppOutput\Il2CppTypeDefinitions.c" /nologo /c /bigobj /W3 /Zi /EHs /GR- /Gy /utf-8 /wd4102 /wd4800 /wd4056 /wd4190 /wd4723 /wd4467 /wd4503 /wd4996 /Od /Zc:inline /RTC1 /MDd /DWINDOWS_UWP /DUNITY_UWP /DUNITY_WSA_10_0 /DUNITY_WSA /DUNITY_WINRT /DPLATFORM_WINRT /DNET_4_0 /DUNITY_AOT /DGC_NOT_DLL /DRUNTIME_IL2CPP /DIL2CPP_DEBUG=1 /D_WIN32 /DWIN32 /DWIN32_THREADS /D_WINDOWS /DWINDOWS /D_UNICODE /DUNICODE /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /D_WINSOCK_DEPRECATED_NO_WARNINGS /DNOMINMAX /D_DEBUG /DDEBUG /DIL2CPP_DEBUG /D__arm__ /DWINDOWS_SDK_BUILD_VERSION=18362 /D__WRL_NO_DEFAULT_LIB__ /DWINAPI_FAMILY=WINAPI_FAMILY_APP /I"C:\New folder\Il2CppOutputProject\IL2CPP\libil2cpp" /I"C:\New folder\Il2CppOutputProject\IL2CPP\libil2cpp" /I"C:\New folder\Il2CppOutputProject\IL2CPP\external\bdwgc\include" /I"C:\New folder\Il2CppOutputProject\Source" /I"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\include" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\shared" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\winrt" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt" /Fo"C:\New folder\build\obj\il2cppOutputProject\ARM\Debug\09B17862D6E792746F864293F89B66F0.obj" /Fd"C:\New folder\build\obj\il2cppOutputProject\ARM\Debug\09B17862D6E792746F864293F89B66F0.pdb"
    19. 1>EnvArg key: PATH value: C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x86;C:\Program Files (x86)\Windows Kits\10\bin\x86;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\bin\HostX64\x64
    20. 1>
    21. 1>   at Unity.IL2CPP.Building.CppProgramBuilder.BuildAllCppFiles(IEnumerable`1 sourceFilesToCompile, IBuildStatisticsCollector statisticsCollector)
    22. 1>   at Unity.IL2CPP.Building.CppProgramBuilder.Build(IBuildStatistics& statistics)
    23. 1>   at il2cpp.Program.DoRun(String[] args)
    24. 1>   at il2cpp.Program.Run(String[] args)
    25. 1>   at il2cpp.Program.Main(String[] args)
    26. 1>Arguments: "C:\New folder\Il2CppOutputProject\Source\il2cppOutput\Il2CppTypeDefinitions.c" /nologo /c /bigobj /W3 /Zi /EHs /GR- /Gy /utf-8 /wd4102 /wd4800 /wd4056 /wd4190 /wd4723 /wd4467 /wd4503 /wd4996 /Od /Zc:inline /RTC1 /MDd /DWINDOWS_UWP /DUNITY_UWP /DUNITY_WSA_10_0 /DUNITY_WSA /DUNITY_WINRT /DPLATFORM_WINRT /DNET_4_0 /DUNITY_AOT /DGC_NOT_DLL /DRUNTIME_IL2CPP /DIL2CPP_DEBUG=1 /D_WIN32 /DWIN32 /DWIN32_THREADS /D_WINDOWS /DWINDOWS /D_UNICODE /DUNICODE /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /D_WINSOCK_DEPRECATED_NO_WARNINGS /DNOMINMAX /D_DEBUG /DDEBUG /DIL2CPP_DEBUG /D__arm__ /DWINDOWS_SDK_BUILD_VERSION=18362 /D__WRL_NO_DEFAULT_LIB__ /DWINAPI_FAMILY=WINAPI_FAMILY_APP /I"C:\New folder\Il2CppOutputProject\IL2CPP\libil2cpp" /I"C:\New folder\Il2CppOutputProject\IL2CPP\libil2cpp" /I"C:\New folder\Il2CppOutputProject\IL2CPP\external\bdwgc\include" /I"C:\New folder\Il2CppOutputProject\Source" /I"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\include" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\shared" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\um" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\winrt" /I"C:\Program Files (x86)\Windows Kits\10\Include\10.0.18362.0\ucrt" /Fo"C:\New folder\build\obj\il2cppOutputProject\ARM\Debug\09B17862D6E792746F864293F89B66F0.obj" /Fd"C:\New folder\build\obj\il2cppOutputProject\ARM\Debug\09B17862D6E792746F864293F89B66F0.pdb"
    27. 1>EnvArg key: PATH value: C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x86;C:\Program Files (x86)\Windows Kits\10\bin\x86;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.21.27702\bin\HostX64\x64
    28. 1>
    29. 1>   at Unity.IL2CPP.Building.CppProgramBuilder.BuildAllCppFiles(IEnumerable`1 sourceFilesToCompile, IBuildStatisticsCollector statisticsCollector)
    30. 1>   at Unity.IL2CPP.Building.CppProgramBuilder.Build(IBuildStatistics& statistics)
    31. 1>   at il2cpp.Program.DoRun(String[] args)
    32. 1>   at il2cpp.Program.Run(String[] args)
    33. 1>   at il2cpp.Program.Main(String[] args)
    34. 1>C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Microsoft\VC\v160\Microsoft.MakeFile.Targets(44,5): error MSB3073: The command ""C:\New folder\Il2CppOutputProject\\IL2CPP\build\il2cpp.exe" --libil2cpp-static --compile-cpp -architecture=ARMv7 -configuration=Debug -platform=winrt -outputpath="C:\New folder\\build\bin\ARM\Debug\GameAssembly.dll" --data-folder="C:\New folder\\build\bin\ARM\Debug\\" -cachedirectory="C:\New folder\\build\obj\il2cppOutputProject\ARM\Debug\\" -generatedcppdir="C:\New folder\Il2CppOutputProject\\Source"  --profiler-report --additional-defines=WINDOWS_UWP --additional-defines=UNITY_UWP --additional-defines=UNITY_WSA_10_0 --additional-defines=UNITY_WSA --additional-defines=UNITY_WINRT --additional-defines=PLATFORM_WINRT -dotnetprofile=unityaot -verbose --map-file-parser="C:\New folder\Il2CppOutputProject\\IL2CPP\MapFileParser\MapFileParser.exe"" exited with code -532462766.
    35. 1>Done building project "Il2CppOutputProject.vcxproj" -- FAILED.
    36. ========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
    37.  
     
  2. timke

    timke

    Unity Technologies

    Joined:
    Nov 30, 2017
    Posts:
    65
  3. EusebiuMarcuUCB

    EusebiuMarcuUCB

    Joined:
    Sep 20, 2018
    Posts:
    9
    extern const Il2CppGenericInst GenInst_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_0_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_1_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_2_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_3_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_4_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_5_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_6_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_7_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_8_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_9_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_10_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_11_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_12_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_13_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_14_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_15_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_16_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_17_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_18_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_19_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_20_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_21_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_22_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_23_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_24_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_25_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_26_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_27_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_28_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_29_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_30_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_31_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_32_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_33_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_34_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_35_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_36_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_37_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_38_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_39_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_40_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_41_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_42_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_43_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_44_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_45_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_46_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_47_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_48_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_49_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_50_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_51_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_52_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_53_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_54_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_55_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_56_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_57_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_58_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_59_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_60_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_61_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_62_0_0_0_MutableTuple_128_t87D1AF9AF348AB161116CA5729C01D43093F45F9_gp_63_0_0_0;

    Can you guide me in finding the identifier from C#?
     
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    I think there's a type called "MutableTuple" that has 128 generic parameters. Do you have anything like that in your code?
     
  5. EusebiuMarcuUCB

    EusebiuMarcuUCB

    Joined:
    Sep 20, 2018
    Posts:
    9
  6. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,364
    Is there any chance you are able to modify that code and comment out that particular class?

    Also, can you report a bug to us with a repro project attached so we could fix IL2CPP to not generate such long identifier names?
     
  7. EusebiuMarcuUCB

    EusebiuMarcuUCB

    Joined:
    Sep 20, 2018
    Posts:
    9
    Also the 64 has also the same issue. So, I fallback to 32 and now the build does not have this problem anymore.

    To reproduce the issue is quite simple - I will try to make a bug report:
    - create a new project
    - add IronPython libs to it (mainly Microsoft.Dynamic.dl)
    - make use of some function in IronPython
    - Build it for UWP
    - Build VS solution.