Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice
  3. Join us on November 16th, 2023, between 1 pm and 9 pm CET for Ask the Experts Online on Discord and on Unity Discussions.
    Dismiss Notice
  4. Dismiss Notice

Feedback Multi-threaded asset import

Discussion in 'Editor & General Support' started by jonaslarsson, Apr 5, 2020.

?

Do we need multi-threaded asset import?

  1. Yes

    129 vote(s)
    100.0%
  2. No, asset importing is fast enough

    0 vote(s)
    0.0%
  3. No, you are using Unity the wrong way

    0 vote(s)
    0.0%
  1. jonaslarsson

    jonaslarsson

    Joined:
    Aug 17, 2019
    Posts:
    1
    Asset importing is exceedingly slow, for my newly started project it's taking well over an hour after adding a few asset packs (2 seconds per fbx file).

    At the same time, only one CPU core is in use!

    I think we desperately need multi-threading for asset imports as I've been spending more time this week on waiting for asset importing when changing project settings, than I have been working on the project.

    I guess the work-around right now is to avoid adding asset packs from the unity store. Just manually pull in the minimum amount of assets needed. But this removes a lot of the convenience since it's difficult to get an overview of what is in an asset pack from Windows Explorer.
     
  2. Tortuap

    Tortuap

    Joined:
    Dec 4, 2013
    Posts:
    124
    To partially address this issue, I have a dedicated projects in which I'm extracting Assets pack I'm interested in. This way I don't clutter my game project, and also I can import new stuff in background, while continuing to work on my game project.

    In practice, I have multiple dedicated projects : one for musics, one for sounds, one for City themes, one for Space themes, etc. rather than a single huge project containing all my assets packs. I separated them because opening a project also takes a while, proportionally to the number of files in it.
     
    ModLunar and Morbeavus like this.
  3. kristoof

    kristoof

    Joined:
    Aug 26, 2013
    Posts:
    83
    Is it single-threaded tho?
    On my 5900X it looks like it's hitting all the treads.
    Maybe not as good utilization, around 20-30% usually (and it's still somewhat slow) but it looks like this already has some groundwork done.
    Would be great to get some comments on this from UT.
    Edit: 2019.4 LTS
     

    Attached Files:

  4. SampsaPlaysome

    SampsaPlaysome

    Joined:
    Oct 20, 2019
    Posts:
    29
    The import is embarassingly single-threaded at times. I get like 20% overral utilization when importing on 5950X. I think that the problem is essentially every asset is imported in single thread, but the importer itself can run multithreaded (like etc compression). But why it doesn't run multiples of those at the same time - no idea.

    upload_2021-4-1_10-13-52.png
     
  5. bricefr

    bricefr

    Joined:
    May 3, 2015
    Posts:
    61
    Same thing under Linux, only 1 core is working while importing several hundreds of FBX. Kinda frustrating with a 32 cores machine :p Almost as slow as UE4 import process :p
     
  6. notagame

    notagame

    Joined:
    Sep 17, 2017
    Posts:
    17
    Yeah this is the death by a thousand paper cuts. Unity just keeps on giving with these utterly ridiculous design choices that never get fixed. I have a 64 core CPU that is idling at 3% while unity keeps importing for a couple hours. It's ludicrous. Meanwhile, Unreal Engine 4 utilizes 100% of the 64 cores when it loads a project and its done within a minute or two.
     
  7. levi9000

    levi9000

    Joined:
    Jan 25, 2014
    Posts:
    8
    Yes please. The more impressive my made with unity game gets, the more time I lose waiting for Unity. I have to reimport everything at least twice a month when the asset database cache gets corrupted (and then unity starts 30-200 second stutters). It would be nice if reimporting didn't mean go do something else for a half hour.

    Is there a place to cast votes for feature requests or do we just post into the empty forum void?
     
  8. notagame

    notagame

    Joined:
    Sep 17, 2017
    Posts:
    17
    Yeah it's pityful. It's like Unity as a company didn't get the memo in 2006 when CPUs started to have more than one core. I have a threadripper with 128 cores and wait about an hour or two at 3% usage while unity opens the project from a fresh git pull :D. It's hilarious. Must be really difficult to spin up that PNG decoder in two threads instead of one.

    Same for WebGL export. I went as far as disabling Brotli and using a python (get that... PYTHON) script to compress everything in parallel. Unity takes about 10 minutes to compress my WebGL build, while my python script takes about 20 seconds.

    Meanwhile with UE4 practically all 128 cores are utilized whenever there is ANY kind of work to do.
     
    mischa2k likes this.
  9. A_Savvidis

    A_Savvidis

    Joined:
    Jul 21, 2016
    Posts:
    97
    For anyone reading this. We have parallel importing after version 2021.2. See here