How to reduce import times when upgrading Unity versions? 4hrs and counting

Hello ya'll,

I am currently upgrading from 2019.3.13 to 2019.3.14 and am quite surprised how long it is taking up finish the initial re-importing of assets. The project was already using Asset Database v2 and is of moderate size (10gb). 4 hours into the import and it isn't even halfway through

Hardware probably isn't an issue as it is an up-to-date setup, so I am wondering if there is anything that can be done to avoid this kind of delay. Are there things that can be copied over, or some kind of editor script that can be included to prevent/skip what seems like an entirely unnecessary import process? Why does upgrading even require re-importing everything? Any insight would be much appreciated.

 

Importing is usually of the things that always slows down my workflow significantly. I just downloaded something from the package manager and currently waiting for a full re-import of all assets, not sure why.

 

The unity unity Accellerator server can save some time in cases not all asses changed / the importer version of unity has not changed.
If the imported asset would be the same as before then the cache server would provide the already imported asset back to you.

https://docs.unity3d.com/Manual/UnityAccelerator.html

For new assets and new unity version this does not help.

 

The same full re-import is happening when upgrading from 2019.3.14f1 to 2019.3.15f. It looks like the texture compression tools are the same, the meta-data doesn't change, but the artifact db had a "mostly redundant" version bump*. "Mostly redundant", since equivalent artifacts are identical but for a "0x35" instead of a "0x34" at byte position 0x1c
It's a bit sad that you have to wait a few hours for this...
... which reminds me: isn't it about time Unity introduced multi-threaded import? It's almost unbearable watching image files getting imported one after another, not even fully utilizing a single core. When importing *.blend files, Unity is starting a new Blender process for each file (!), so it's sad but reasonable to assume they'll start an "etccompress" process for each image file. Just start one for each core, and be done with it.
The new asset database was supposed to "pave the path to faster importing", but this exceedingly simple "optimization" hasn't been implemented, yet. In 2020. With HEDT solutions featuring 64 cores...

*) hopefully it's not completely redundant, as in: someone at Unity has decided to just bump that version by default, whether there are actual changes or not.

 

If the single core isn't being fully utilized, isn't the process I/O bound anyway? How would more cores help then?

 

It's hardly I/O bound, just not in a tight processing loop. Consider the OP's case - 10 GB worth of assets taking north of 4 hours, or about 0.7 MB/sec: that's a non issue for even ancient hardware.

 

After 12 hours my upgrade was ~25% complete so I gave up. I had the same issue with .13 -> .14 and I ended up creating a new .14 project and rebuilding my game. But now I have the same issue with .14 -> .15. I did not see this issue with previous upgrades. The project is using Asset Pipeline Mode Version 2 and there are 7,177 FBX files so at 10 seconds each that's 19 hours.

In my case it looks like the issue is FBX related. Each FBX import seems just over 10 seconds with a few taking longer. It also looks like the import crashes because I see CrashedAssetImports.txt with new files in it, but the process overwrites that file because I rarely see more than five so I can't be sure it's every FBX file. Other assets and FBX animation don't seem impacted.

As an example from Editor.log:

Start importing Assets/Ultimate Poly Fantasy/FBX/Stone_Bricks_and_Blocks_1A1.fbx using Guid(9024685f39207e345a6cef65c35407d3) Importer(-1,00000000000000000000000000000000)
Done importing asset: 'Assets/Ultimate Poly Fantasy/FBX/Stone_Bricks_and_Blocks_1A1.fbx' (target hash: 'b6abace492f6da0253684d7b2f48c6e3') in 76.646248 second​

The file is 101KB and imports in Blender in 0.03125 seconds. My guess is that there's a 10 second timeout involved in the import process and the FBX import process isn't ending correctly so it waits for 10 seconds and then gives up.

 

Damn that's gotta be some kind of bug. you make an empty project with just that asset and try to upgrade it can you reproduce that import time? That would be a great minimal project to submit a bug report with!

 

It's unlikely to be I/O bound if it's running as single thread because the main function of the import is to parse and convert the file which are both CPU heavy operations. It's a small file read, process the file, then a small file write into the cache. There's also some meta data changes. I/O should be negligible.

 

I have the same issue, I always upgraded my projects from one minor version of Unity to another and it usually took a few minutes. But for .14 and .15 it re-imports everything like when you delete the library folder. And that's incredibly long for medium/large projects. I have lots of sound files which is the part that takes the longest in my case.

Why did this change? I didn't have the issue for any 2019.3.X version before .14 and .15 and I didn't change a single parameter in my project