Bug Extreme build performance regression with com.android.tools.build:gradle >= 3.6.x

I have had to upgrade com.android.tools.build:gradle, in order to build AABs (which needs 4.0.x) and to support ARCore (which needs 3.6.x) and noticed that certain gradle build steps are nearly 40 times slower than with 3.4.x.

I tracked the regression to the jump from 3.4.x to 3.6.x and it is the same on every version after that.

Here are some Gradle Build Scans for the same build done with different versions of com.android.tools.build:gradle, both done after cleaning the project.

3.4.3 - Total build time of 14.783s
3.6.4 - Total build time of 6m 49.483s (!!!!)

The offending steps are:
:launcher:mergeDebugJavaResource 6m 41.434s com.android.build.gradle.internal.tasks.MergeJavaResourceTask
:launcher:desugarDebugFileDependencies 6m 36.926s com.android.build.gradle.internal.tasks.DexFileDependenciesTask
:launcher:mergeDebugNativeLibs 6m 35.912s com.android.build.gradle.internal.tasks.MergeNativeLibsTask
:launcherrocessDebugResources 6m 34.827s com.android.build.gradle.internal.res.LinkApplicationAndroidResourcesTask

Any idea what I could possibly do to solve or at least improve this? It massively increases my iteration times.

 

I've done a little bit of hunting to see what on earth is being passed as an argument to aapt2.exe to take nearly 7 minutes to complete. To my surprise, I found that it passes the longest regular expression I have ever seen in my life as an parameter to --no-compress-regex.

The regex appears to be a character by character match of every asset in the build, which in our case is a lot of assets. The regex is 189,189 characters long.

Heavily truncated example:

Code (CSharp):

1.  
2. --no-compress-regex (((a|A)(a|A)/(a|A)(n|N)(d|D)(r|R)(o|O)(i|I)(d|D)/(b|B)(a|A)(s|S)(e|E)(a|A)(n|N)(i|I)(m|M)(a|A)(t|T)(i|I)(o|O)(n|N)(s|S)_(a|A)(s|S)(s|S)(e|E)(t|T)(s|S)_(a|A)(l|L)(l|L)\.(b|B)(u|U)(n|N)(d|D)(l|L)(e|E)$)|((a|A)(a|A)/(a|A)(n|N)(d|D)(r|R)(o|O)(i|I)(d|D)/(c|C)(u|U)(r|R)(r|R)(e|E)(n|N)(c|C)(y|Y)(i|I)(c|C)(o|O)(n|N)(s|S)_(a|A)(s|S)(s|S)(e|E)(t|T)(s|S)_(a|A)(l|L)(l|L)_(d|D)0(f|F)18(e|E)(e|E)
3.  

What on earth is going on!?

 

In this case it's launcher/build.gradle, but that is indeed the issue, that is precisely what it is doing. It would appear that every asset bundle in streaming assets is ending up in aaptOptions.noCompress and the build tools explode that to that monster regex. Replacing that whole list with '.bundle' brings the build time down to nothing again. This leads to two obvious questions: Why was it fast before 3.6.x and why is Unity putting every file in there individually as opposed to just .bundle.

Edit: Actually, dumping the aapt2 args that 3.4.x was spitting out, I can see why it used to be fast, it used to send each one to an individual -0 argument. 3.6.x compiles a monster regex out of the whole thing itself.

Edit 2: Found the offending entry in the changelog:

• aaptOptions.noCompress is no longer case sensitive on all platforms (for both APK and bundles) and respects paths that use uppercase characters.

That is an... interesting... way of implementing that.

 

Sure, will do on Monday.

 

Thanks for the heads up. I tried to make a repro against 2021.1.21 and the behaviour has been quite different, so this would explain that. I haven't tested with my actual project (it doesn't build in 2021) but I have copied the directory structure of my StreamingAssets across and it does condense everything to .bundle, which is my current workaround on 2019.4.31 anyway, bringing build times from 7 minutes down to the 15 seconds it was on com.android.tools.build:gradle <= 3.4.x. It is obvious that aapt2 was ending up spending literal minutes evaluating that regex.

One other thing that is interesting is that if I force 2021.1.21 to generate an asset list as before (by seeding with files that don't have an extension) the issue is slightly different than it was on the project generated by 2019.4.31, instead of the build getting ridiculously slow there is a threshold of files past which aapt2 just immediately crashes.

 

-ish. From 3.6.0 onwards, the Gradle plugin generates a single monstrous regex for everything provided to aaptOptions.noCompress and passes it into -no-compress-regex rather than a series of -0 entries. So, when building from 2019.4.31 (which is missing your fix) it ends up generating an absolutely ridiculous regex for the file list and slows the build down to a crawl, even with fewer elements than 620. When trying to repro, I did get to a number where it was just outright crashing rather than being slow, which sounds similar to the original issue. Either way, your fix should sort both issues out.