Search Unity

  1. How can our website serve you better? Give us your feedback. Take our survey and let us know.
    Dismiss Notice

Unity Asset Importing speedups in 2021.2 - call for feedback

Discussion in '2021.2 Beta' started by Aras, Jun 10, 2021.

  1. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,739
    Hi everyone, continuing our focus on performance and fast iteration, we’ve added some asset import improvements to 2021.2 alpha lately. These include texture and model import optimizations, a new option to import textures and meshes in parallel, and various other scalability optimizations.

    Here’s a test importing all the textures, models, prefabs and scenes from Book of the Dead: Environment project.
    Clipboard01.png

    This contains 2.25GB of source asset data (346 textures, 133 models, 214 prefabs, 6 scenes). Timings are on Windows, AMD ThreadRipper 1950X (16 threads), SSD storage machine. We’re comparing Unity 2020.3.10, 2021.1.9 and the current 2021.2 alpha (a19).

    Model (fbx, obj, sketchup, ...) importing in particular got optimized quite a bit in 2021.2. On average it's maybe "twice as fast", but we've also seen some pathological cases where a model that used to take hours to import, now imports in several minutes.

    Note that parallel asset importing is off by default for now. When turned on, at the moment it only works for texture & model imports, all other asset types are imported in a single process as before. The setting is in Project Settings -> Editor -> Refresh section. In this example, we see enabled parallel asset import with 4 asset import worker processes.

    If, for iteration time purposes, we also use the “Fast” texture compressor option in Build Settings (this does not change any texture sizes or formats, just spends less effort trying to come up with the best possible compressed texture pixels; see this thread for details), all the assets import in 38 seconds. This is a 3.8x speedup compared to the 2021.1 version!
    Clipboard05.png

    Switching the same project to Android (ASTC texture format) platform for the first time takes 87 seconds, using both parallel asset import with 4 asset import work processes and the Force Fast Compressor option. This speeds up ASTC texture compression dramatically (6.6x speedup!). However the textures will contain slightly more compression artifacts, so you don’t want to have this setting on for your final build.

    Scaling to Large Projects
    In our test projects with a large number of assets (like “a million”), we’ve managed to cut the editor startup time by around 50%, compared to previous versions. This is due to various optimizations to asset database processing and data structures. For example, opening a project with 900 thousand asset files, when everything is up to date, took over 3 minutes before, and now takes just over a minute.
    pasted image 0.png

    Opening of scenes with very large & flat hierarchies (lots of objects at the root level) is also much faster now.


    Feedback needed!


    It would be useful to have more data on how these improvements work for your 2021.2 alpha projects so far, or if you’re running into any issues. We are therefore asking you to share your Editor performance benchmarks! Please share your numbers in the thread below or send me a DM.

    How do you get to those numbers? You can take a look at the editor log file to see import times of individual assets or the whole import (it looks like “Asset Pipeline Refresh: Total: 139.7 seconds”). Among the several command line arguments you can supply upon opening Unity, the -profiler-enable command line argument allows you to profile the Editor during launch. Using this argument can help you see what happens during startup and what takes time.

    We’ve also created this Editor Log Parser that will output a CSV file with all the numbers, so that we can put some of those easy to read graphs together. This is especially useful when there is no library and the full project is reimported.

    Here are the numbers that we’re most interested in, in the simplest possible format:
    Startup
    Unity 2019.4: xx seconds
    Unity 2021.2: xx seconds

    Full project import:
    Unity 2019.4: xx seconds
    Unity 2021.2: xx seconds
     
    Last edited: Jun 10, 2021
  2. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,495
    Full project import would be delete library -> open? Or right click -> reimport?
     
  3. ImpossibleRobert

    ImpossibleRobert

    Joined:
    Oct 10, 2013
    Posts:
    386
    These are such important improvements. You cannot imagine how happy that makes me. Thanks a lot Aras and team!
     
    sl1nk3_ubi likes this.
  4. dongch007

    dongch007

    Joined:
    Jan 8, 2014
    Posts:
    22
    Will startup time optimize backport to 2020?
    Start a project in HHD also quite slow.....
     
  5. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,739
    Either, as long as comparison is the same between versions.
     
  6. GoGoGadget

    GoGoGadget

    Joined:
    Sep 23, 2013
    Posts:
    762
    As someone with a threadripper who has been struggling with how slow the asset importer is on 2020, this is a great improvement. WP Aras & team. Will definitely be giving it a crack soon, and will put numbers here.
     
  7. DoctorShinobi

    DoctorShinobi

    Joined:
    Oct 5, 2012
    Posts:
    169
    This is very exciting. I'm especially happy to see the editor becoming more multicore friendly. Good job!
     
  8. mahdi_jeddi

    mahdi_jeddi

    Joined:
    Jul 18, 2016
    Posts:
    188
    How can I get the startup time? I'm trying to see if it's on the Editor.log, but there are way too many timings there, and I'm not sure which one is the one you want. Is this it?
    [Project] Loading completed in 14.133 seconds
     
  9. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,495
    Tried the SimpleEditorLogParser, but it's giving me "Log is not in a supported format. Please implement new parsing for new format present in this log file." for Unity 2020.3.0 (current project version).

    But from just reading the log, startup times are

    2020.3.0f1:
    Approximately 15.5 seconds

    2021.2.0a20:
    Approximately 14 seconds

    BUT; I had to comment out some some code that happened at startup in 2020.3, since it caused errors on startup in 20201.2 (reflection code that interacted with things that have been removed). Specifically, I had two things that hooked up to DragAndDropService in order to handle some dragging and dropping. I don't know why it broke, but I don't have the time to update it now. I also had a custom UnityEditor.OverlayWindow that I booted at startup (apparently, as it was throwing exceptions on boot!). Those have been replaced now, so I had to comment that out as well.

    I don't know how much cost running that code had, but it should probably take something like 1.5 milliseconds, not 1.5 seconds, so there's a good chance there's an improvement.

    Reimport all time is next, I guess.
     
  10. Unity_Javier

    Unity_Javier

    Unity Technologies

    Joined:
    Mar 7, 2018
    Posts:
    89
    You can launch the editor with the command line argument -profiler-enable
    (i.e Unity.exe -projectpath path_to_project -profiler-enable)
    That will capture the timing for the first frame and put it in the profiler that you can bring up :)
     
    mahdi_jeddi likes this.
  11. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,495
    It seems like Assets->Reimport all is down from 2052 seconds to 994 seconds. Going to check deleted library as well
     
  12. ImpossibleRobert

    ImpossibleRobert

    Joined:
    Oct 10, 2013
    Posts:
    386
    OMG this is amazing. Here my start-up numbers:

    2021.1.11: 3m04s
    2021.2.20: 0m36s

    Assets folder: 102.448 files, 4544 folders, 43.3Gb
     
  13. mahdi_jeddi

    mahdi_jeddi

    Joined:
    Jul 18, 2016
    Posts:
    188
    I couldn't use the profiler, because it would just try to import the profiler data for hours, so I ended up using the Editor Log for the timings. I also had to redo the imports a few times, because it would give me out of memory exception on 2020.3. The time of 2021.2 was in the middle of the editor log, so I don't know how accurate it is. I opened the project a few times until the startup time settled on a number (The first one right after an import was a little bit higher).

    The project is mostly PNG files that would be loaded into sprites (~65K png files, ~ 9.5 GB). 2021.2 is set to 6 worker threads:

    Startup:
    2020.3.11f1: 13.135
    2021.2.0a20: 10.869 20% faster

    Import:
    2020.3.11f1: 5,964 seconds (1:39 hours)
    2021.2.0a20: 2618 seconds (43 minutes) 2.27x faster

    This is on a 8700K 6 Cores, 32 GB of RAM and SSD. During the import the PC was completely on its own only doing this. I sometimes looked at the CPU/Disk usage. It was usually around 25% CPU usage and 140 MB/s disk usage.
     
    Last edited: Jun 11, 2021
    richardkettlewell likes this.
  14. mahdi_jeddi

    mahdi_jeddi

    Joined:
    Jul 18, 2016
    Posts:
    188
    Ok, I made a mistake. The numbers for the 2021.2 are for the default import with no worker count set, which is amazing. I'm gonna try again with 6 workers to see how it changes it.
     
  15. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,739
    FYI in our measurements it typically makes sense to use about 25% of workers as your CPU has threads -- the reason being that e.g. importing one texture is mostly multithreaded already, and importing meshes (since 2021.2) is partially multithreaded too. So if you create N=CPU-threads workers which all have N threads doing work, it's a bit of CPU over-subscription.
     
  16. mahdi_jeddi

    mahdi_jeddi

    Joined:
    Jul 18, 2016
    Posts:
    188
    With 6 workers it's now taking 100% CPU usage and >300MB/s disk usage. I'm still not super sure about the accuracy of these numbers as I still get out of memory errors at the end of the imports. I'll try to see if I can fix those.

    1569 seconds (26 minutes) : 3.8x faster than 2020.3 1.6x faster than 2020.2 without worker threads
     
  17. mahdi_jeddi

    mahdi_jeddi

    Joined:
    Jul 18, 2016
    Posts:
    188
    I guessed that it would be the case. I just wanted to go wild and see if I can get some more juice out of it. 3.8x still seems logical as the original was going around 25%.

    The importing of packages is also multi-threaded now? I saw it goes over 70% when they're being processed.

    Also, what's the difference between In-Process and Out of Process workers? Does it matter for performance or is it just as a test option?
     
  18. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,739
    Packages are no different from any regular assets, as far as importing is concerned.

    Hmm you might be hitting https://issuetracker.unity3d.com/is...mory-error-when-importing-size-heavy-textures which I was just fixing this week (workaround for now would be to switch editor non-DX11 graphics API).
     
  19. mahdi_jeddi

    mahdi_jeddi

    Joined:
    Jul 18, 2016
    Posts:
    188
    That's an amazing news, because we had this issue for a long time. Our workaround always was to just open the project again until it reimports everything. The same thing happens when packing the sprite atlases.
     
  20. Carpe-Denius

    Carpe-Denius

    Joined:
    May 17, 2013
    Posts:
    830
    I can't give exact figures, but my (rather large) project import went from "about a week" in 2020 to "about two days" in 2021.1 and "7 hours" in 2021.2a21; but in 2021.2b1 the import takes a little bit longer and never finishes due to an out of memory error. I don't use dx11, though.
     
    phobos2077 and richardkettlewell like this.
  21. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,739
    Would be good to know what exactly the out of memory error says, and also when that happens (the asset being imported should be the last thing printed in the editor log file). If it happens consistently during import of that asset, that sounds like a bug and would love a bug report with that asset!
     
  22. Carpe-Denius

    Carpe-Denius

    Joined:
    May 17, 2013
    Posts:
    830
    My project is a little bit to big to send in and I always feel bad when the bug reporter tells me I don't send in enough information, but if you want to, I can search through the crash logs.

    It doesn't happen with a specific asset, though. It always starts with
    Code (CSharp):
    1. d3d12: upload buffer was full! Waited for COPY queue for 5.151 ms.
    several hundred times.
    After that, I'll change to

    Code (CSharp):
    1. d3d12 : CreateCommittedResource 'BufferD3D12::CreateBufferResource() Buffer' (157200 x 1) format 0 failed (887a0005).
    2.     Device Remove Reason (HRESULT): 0x887a0007
    3.     Device Memory Stats:
    4.         Local AvailableForReservation: 3749 MB, Budget: 7198 MB, CurrentUsage: 7170 MB, CurrentReservation: 64 MB
    5.         Non-local AvailableForReservation: 4850 MB, Budget: 9444 MB, CurrentUsage: 9465 MB, CurrentReservation: 0 MB
    6. d3d12: CreateResource for buffer failed.
    7.  
    8. d3d12: failed to create a buffer of size 157200
    9. BufferD3D12 : Unexpected null accessor on Buffer
    again several hundred times with different sizes (always ????? x 1, a one pixel wide texture?)

    After those it'll close with:

    Code (CSharp):
    1. Could not allocate memory: System out of memory!
    2. Trying to allocate: 4803120B with 16 alignment. MemoryLabel: Serialization
    3. Allocation happened at: Line:69 in C:\buildslave\unity\build\Runtime\Utilities\dynamic_array.h
    4. Memory overview
    5.  
    6. [ ALLOC_TEMP_TLS ] used: 44462896B | peak: 0B | reserved: 22347776B
    7. [ ALLOC_MEMORYPROFILER ] used: 28778184B | peak: 28778184B | reserved: 33562624B
    8. [ ALLOC_DEFAULT ] used: 1147040159B | peak: 0B | reserved: 1344184320B
    9. [ ALLOC_TEMP_JOB_1_FRAME ] used: 0B | peak: 0B | reserved: 2097152B
    10. [ ALLOC_TEMP_JOB_2_FRAMES ] used: 0B | peak: 0B | reserved: 2097152B
    11. [ ALLOC_TEMP_JOB_4_FRAMES (JobTemp) ] used: 0B | peak: 0B | reserved: 134217728B
    12. [ ALLOC_TEMP_JOB_ASYNC (Background) ] used: 0B | peak: 0B | reserved: 14680064B
    13. [ ALLOC_GFX ] used: 12514140332B | peak: 0B | reserved: 12534845440B
    14. [ ALLOC_CACHEOBJECTS ] used: 5146131464B | peak: 0B | reserved: 5191888896B
    15. [ ALLOC_TYPETREE ] used: 1328609320B | peak: 0B | reserved: 1369440256B
    16. [ ALLOC_PROFILER ] used: 757288B | peak: 850200B | reserved: 16777216B
    17. [ ALLOC_PROFILER_EDITOR ] used: 0B | peak: 0B | reserved: 0B
    18. Could not allocate memory: System out of memory!
    19. Trying to allocate: 4803120B with 16 alignment. MemoryLabel: Serialization
    20. Allocation happened at: Line:69 in C:\buildslave\unity\build\Runtime\Utilities\dynamic_array.h
    21. Memory overview
    22.  
    23. [ ALLOC_TEMP_TLS ] used: 44462896B | peak: 0B | reserved: 22347776B
    24. [ ALLOC_MEMORYPROFILER ] used: 28778184B | peak: 28778184B | reserved: 33562624B
    25. [ ALLOC_DEFAULT ] used: 1147040159B | peak: 0B | reserved: 1344184320B
    26. [ ALLOC_TEMP_JOB_1_FRAME ] used: 0B | peak: 0B | reserved: 2097152B
    27. [ ALLOC_TEMP_JOB_2_FRAMES ] used: 0B | peak: 0B | reserved: 2097152B
    28. [ ALLOC_TEMP_JOB_4_FRAMES (JobTemp) ] used: 0B | peak: 0B | reserved: 134217728B
    29. [ ALLOC_TEMP_JOB_ASYNC (Background) ] used: 0B | peak: 0B | reserved: 14680064B
    30. [ ALLOC_GFX ] used: 12514140332B | peak: 0B | reserved: 12534845440B
    31. [ ALLOC_CACHEOBJECTS ] used: 5146131464B | peak: 0B | reserved: 5191888896B
    32. [ ALLOC_TYPETREE ] used: 1328609320B | peak: 0B | reserved: 1369440256B
    33. [ ALLOC_PROFILER ] used: 757288B | peak: 850200B | reserved: 16777216B
    34. [ ALLOC_PROFILER_EDITOR ] used: 0B | peak: 0B | reserved: 0B
    35. UnityEngine.StackTraceUtility:ExtractStackTrace ()
    36. UnityEditor.ModelImporterPostProcessor:OnPostprocessAllAssets (string[],string[],string[],string[])
    37. System.Reflection.RuntimeMethodInfo:Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo)
    38. System.Reflection.MethodBase:Invoke (object,object[])
    39. UnityEditor.AssetPostprocessingInternal:InvokeMethod (System.Reflection.MethodInfo,object[])
    40. UnityEditor.AssetPostprocessingInternal:PostprocessAllAssets (string[],string[],string[],string[],string[])
    41.  
    42. [C:\buildslave\unity\build\Runtime\Allocator\MemoryManager.cpp line 1624]
    43.  
    If I can help without uploading a 600gb project folder, I'm open to anything.

    Additionally, there is a problem with the MaterialPostProcessor in hdrp (all materials get reimported after update and fill up memory), but I can manually change the cs file and force unity to save assets every 100 or so materials, but this crash happens before the material reimport.
     
    phobos2077 likes this.
  23. Ruchir

    Ruchir

    Joined:
    May 26, 2015
    Posts:
    550
    I'm also wondering the same, is there a difference?
     
  24. jdrewsen

    jdrewsen

    Unity Technologies

    Joined:
    Mar 28, 2011
    Posts:
    172
    In-Process will run all imports inside the main unity editor process one by one.
    Out of process will run compatible (Textures/Models) importers in worker processes in parallel, and incompatible ones in-process. This is generally a speedup and the direction we are going for all importers ie. making theme all compatible.

    We realize the wording is a bit confusing and it will be changed before final release.
     
    mahdi_jeddi and Ruchir like this.
  25. LeonhardP

    LeonhardP

    Unity Technologies

    Joined:
    Jul 4, 2016
    Posts:
    2,598
    The fix will be available in b3.
     
  26. MP-ul

    MP-ul

    Joined:
    Jan 25, 2014
    Posts:
    176
    Related to the importer in the 2021.2 I made an animation recently and while i was trying to import the model in unity guess what, the model was not visible Look.
    upload_2021-7-17_21-28-31.png

    In unity 2019 it just works, look!.
    upload_2021-7-17_21-26-5.png
    So how am I supposed to test animations and features on the new version when it can't import an FBX correctly?!.
     
    Ruchir likes this.
  27. Aras

    Aras

    Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,739
    If some asset has imported fine in any previous version of Unity, but does not import correctly now, that's a regression that we should fix. If you filed a bug report with said model (and the import settings, i.e. the .meta file) attached, that would be most excellent.
     
    richardkettlewell and Ruchir like this.
  28. MP-ul

    MP-ul

    Joined:
    Jan 25, 2014
    Posts:
    176
    1351441 I hope the 20 minutes of waiting are worth. This is how the animation should look.

     
    Last edited: Jul 18, 2021
    ImpossibleRobert likes this.
  29. richardkettlewell

    richardkettlewell

    Unity Technologies

    Joined:
    Sep 9, 2015
    Posts:
    1,798
    without wanting to derail the thread too much, this *might* be the new “remove constant scale curves” option. You could try disabling it, see if it helps. If so, the fix is already on the way.
     
  30. MP-ul

    MP-ul

    Joined:
    Jan 25, 2014
    Posts:
    176
    Well that did do something, but the model is still broken. upload_2021-7-18_22-59-54.png
     
  31. MP-ul

    MP-ul

    Joined:
    Jan 25, 2014
    Posts:
    176
    @richardkettlewell @Aras Let me know if you have a fix for this. I started doing good progress on something and it would be a shame if i have to stop because of 1 little thing.
     
unityunity