2019.4.18 - CLI build fail (Editor works) - Emscripten [Error 6] The handle is invalid

Hello,

I hope I did not miss it in search

We have a problem with building our upcoming games (2 of them) using Command Line Interface. When using the very same build command from editor everything works fine

We tried it on 2 different PCs with 2019.4.16 and 2019.4.18 and they receive the same error. This one is from TeamCity build chain, but it looks the same (sans paths) on my local machine. Second game results in exactly identical issue and again builds just fine from editor itself. Android Builds also build just fine from CLI, it's just WebGL that is causing headache. New/empty WebGL projects build fine. So it's not an issue with machines or Unity installation itself

I am removing parts of the pathnames for clarity

[UNITY] -> Unity Install Location \ WebGLSupport\BuildTools

--- code follows (super long lines, was breaking code tags) ---

Filename: "[UNITY]\Emscripten_Win\python\2.7.5.3_64bit\python.exe"
Arguments: -E "[UNITY]\Emscripten\emcc"
@"D:\AgentUnity1\work\37c533f1543ab8a2\Assets\..\Temp\emcc_arguments.resp"
index: -1
Failed running "[UNITY]\Emscripten_Win\python\2.7.5.3_64bit\python.exe" -E
"C[UNITY]\Emscripten\emcc"
@"D:\AgentUnity1\work\37c533f1543ab8a2\Assets\..\Temp\emcc_arguments.resp"

stdout:
stderr:Traceback (most recent call last): File "[UNITY]\Emscripten\emcc.py", line 3063, in <module> sys.exit(run()) File "[UNITY]\Emscripten\emcc.py", line 1632, in run extra_files_to_link += system_libs.calculate([f for _, f in sorted(temp_files)] + extra_files_to_link, in_temp, stdout_=None, stderr_=None, forced=forced_stdlibs) File "[UNITY]\Emscripten\tools\system_libs.py", line 520, in calculate symbolses = shared.Building.parallel_llvm_nm(list(map(os.path.abspath, temp_files)))
File "[UNITY]\Emscripten\tools\shared.py", line 1844, in parallel_llvm_nm object_contents = pool.map(g_llvm_nm_uncached, files)
File "[UNITY]\Emscripten_Win\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 250, in map return self.map_async(func, iterable, chunksize).get()
File "[UNITY]\Emscripten_Win\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 554, in get raise self._valueWindowsError: [Error 6] The handle is invalid

---

Overall: [Error 6] The handle is invalid

The command used was rather simple, just project path, execute method and the standard "-quit -nographics -batchmode -buildTarget WebGL"

We spent quite some time on Google looking for some suggestions, but things we found (like removing jslibs from project) did not help

Full log is >8 MB and contains things that we cant show due to NDA, but here's a redacted part starting with Emscripten linking process:
https://imdev.pl/share/6c84886a-8ae3-491d-b8ad-ea23c8f26710/buildlog_webgl_dev.txt

We will be very helpful for any help here

 

@jukka_j
I added those but same error unfortunately And no extra debug info. Unless Emscripten keeps it logs somewhere else that is

2019.4.18f1 - cutting full folder names

---

Filename: "BuildTools\Emscripten_Win\python\2.7.5.3_64bit\python.exe"
Arguments: -E "BuildTools\Emscripten\emcc" @"D:\Work\PROJECT\Assets\..\Temp\emcc_arguments.resp"
index: -1
Failed running "\BuildTools\Emscripten_Win\python\2.7.5.3_64bit\python.exe" -E "\BuildTools\Emscripten\emcc" @"D:\Work\PROJECT\Assets\..\Temp\emcc_arguments.resp"

Emscripten_Win\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 554, in get raise self._valueWindowsError: [Error 6] The handle is invalid

---

Im not sure why its calling multiprocessing part of library (or maybe it doesnt matter?) but the line 554 relates to error raising from:

#
# Class whose instances are returned by `Pool.apply_async()`
#

class ApplyResult(object):

PS: it works just fine in 2020.2 but we have significant in-game issues and bugs there, so cant upgrade easily

 

Thanks @jukka_j

Unfortunately not much I can do. I spent the whole day yesterday just binary-search-building through commits history and it led me absolutely nowhere

I did get one commit to build and it was from right before our 2019.3->2019.4 transition. No further commits built properly, so I went back to the one that worked and... It failed with Handle Error 6. Restarted my PC, re-downloaded that commit, cleaned up library and everything. Will not build ever again

We are at a complete 100% loss, been looking at it since early December and tried basically every single thing...

 

Thanks @jukka_j

I tried multiple options and collected just the error logs in the linked zip file

Before that:
- reinstalled Unity, 2019.4.20f1
- deleted project + downloaded and imported from scratch

There are:
- baseline - with no system environment variables
- just EM_POPEN_WORKAROUND=1 - which was suggested in some places

- EM_POPEN_WORKAROUND=1 + EMCC_CORES=1
- EM_POPEN_WORKAROUND=1 + EMCC_CORES=1 + EMCC_DEBUG=1
- EMCC_CORES=1
- Adding DEBUG caused a weird "PYTHON not defined" error so I added python.exe location from Unity Install to environment variables but this changed nothing so I skipped this log

I restarted Windows before each and every test

Handle Error 6... always

https://imdev.pl/share/a51d56ff-8bb9-4d34-a3fc-8a20710056c2/ErrorLogs.zip

 

Damn...

Well, since I spent further 2 days working on it... Might as well share

I tried 3 things:
- EMCC_CORES
- EMCC_DEBUG
- direct change in emscripten.py
Like I said in last post - those did not help

But after this I started looking closely at the log and the very last thing that was linked was Live2D animation plugin, with their webgl .bc library. So I removed it (breaking the project in the process) and... it compiled fine

Now the fun part - we have another project in development that is using Live2d and builds with no issues. It's Live2d plugin library is a bit newer so I copied it and...

It built. Almost opened the champagne but decided to give it one more try... FAIL

And that's when we gave up to enjoy the weekend

On Monday morning I discovered that editor will not make builds either, mostly due to changes to Environment Variables and python scripts. So I removed ENV Variables, restored Python files and... NADA, FAIL

Completely reinstalling 2019.4.20 did fix it fortunately but I'm now too scared and too scarred to touch it

PS My early tests in 2020.2 showed the game was building fine there but... There was enough broken bits that we didnt follow with it (and 2020.1 breaks the way WebGL index and template files are handled, so we would have a lot of extra work with it as well. Unstable build + those changes + release window for the game = nope, not gonna risk it now)

 

Just a quick update @jukka_j - nuclear option worked and Mac was able to build it fine. Thus we assume that so should Linux, so we will move the build process to a VM on our server. Feels safer than updating a Live game to 2020.2

 

@jukka_j ok, we decided to bite the bullet and move to Unity 2020.3.3 only to... get bitten by this abomination of an error

Is there any chance this fix make it to main Unity anytime sooner?

 

@jukka_j any chance you could look here? do you maybe have a working but "not tested" Emscripten that we could replace manually just to unlock the build process? This is insanely frustrating issue

 

Thanks for remembering about this

I really wish I could, but the projects are... NDAd, large and contain paid plugins. I will see what happens if I just try to build an empty test project, but since there are no errors I assume it will work without problems

Like I said before - based on the full output log I traced that the very last thing that goes to the linker is Live2d plugin. So I will also try to include in the test project and see what happens

 

@jukka_j hah. Found it

The cause is Live2d as I suspected. I tried building just an empty Unity project with their Unity plugin and nothing else - and it broke in the very same way as above. I contacted them but who knows how much time it will take them to reply, so maybe you have some idea?

It looks like during the CLI build (but not during editor build) Live2d opens... another process that... does nothing it seems, just prints the plugin version


But it IS a new process and [ERROR 6] is directly related to handling multiple processes. So I'd blame it

Unfortunately I don't see anything within their plugin that spawns that process - so it's probably part of the plugin's library

Is there any way of blocking spawning additional processes during the build process? Maybe that would be enough... dirty hack to save us buying and setting up additional non-Windows build machine...

Here's the test project - again, just empty Unity + Live2d sample
https://imdev.pl/share/d36110cb-a13c-43e0-b51e-05cc8e05f374/Live2dCLITest.zip

And it can be built using the command (adapt the paths of course)
d:\Work\Unity\2020.3.3f1\Editor\Unity.exe -quit -nographics -batchmode -buildTarget WebGL -projectPath d:\Work\Live2dCLITest -logFile D:\BuildWebGL.txt -executeMethod BuildSystem.Build_WebGL

Thanks for help

 

@jukka_j damn, thanks anyway. Yes, I wrote them but it's a Japanese company and I don't expect to see much help from them

 

@jukka_j it took some time but finally got a reply from there. Might be of interest to you so Im posting it here:

So as I understand we're basically out of options other than Linux/Mac until... Unity 2021.2, right?

 

Thank you. This is weird indeed. However based on our limited tests this issue happens 100% for this specific plugin. We tried it on a few different machines and different Unity versions - always with same result

But other than that... We are clueless. And since the plugin doesn't have full source available (there's a .bc library there) we can't really dig into it

Btw - tried building it on 2021.2 alpha (not that the game worked properly, but heck, why not try?) and... the build in CLI crashed in an even more interesting way. Editor build worked fine of course..

Created GICache directory at C:/Users/MikeSz/AppData/LocalLow/Unity/Caches/GiCache. Took: 0.034s, timestamps: [260.236 - 260.270]
gi::BakeBackendSwitch: switching bake backend from 3 to 1.
[00:00:00] Precompute started.
Received signal SIGSEGV
Stack trace:
RtlLookupFunctionEntry returned NULL function. Aborting stack walk.
<Missing stacktrace information>

We're cursed

 

Indeed, I did not report it as a bug as I wasnt sure if this is a problem on Unity or plugin side. But the project was attached above, pasting URL to it once more below. Should I report it somehow?

https://imdev.pl/share/d36110cb-a13c-43e0-b51e-05cc8e05f374/Live2dCLITest.zip

That error was in our live project, cant share it But it's a fully 2D project, there's no GI, no lighting at all. Why would it fail on GI... Is there any way to kill GI completely so I can verify it builds properly? EDIT: Just tried again. created new Lighting settings file, unchecked GI and everything related to lighting. And got the same CLI crash as before

Oh, and Live2D Plugin DID build in 2021.2.0a17 without a hiccup

So overall - pre-2021.2 our games will not build due to an issue with Live2D+Emscripten, 2021.2 will not build due to GI error (and we don't use GI). But regardless, I don't think alpha version of Unity is ready for primetime and certain widely used plugins (e.g. Doozy UI) do not support it at all and go into infinite loop being unable to create some required assets