Search Unity

[Open Source] Rule Based Unity Addressable Asset Importer

Discussion in 'Addressables' started by Favo-Yang, Jun 28, 2019.

  1. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    A simple rule based addressable asset importer. It marks assets as addressable, by applying to files having a path matching the rule pattern.

    https://github.com/favoyang/unity-addressable-importer

    Features
    • Path pattern supports both wildcard (*, ?) and regex
    • Specify a static group or dynamic group.
    • Add or replace labels
    • Address replacement (raw path, simplified, regex based replacement)
    • Install as upm package
    Feel free to modify it based on your demand.
     
    Last edited: Aug 4, 2019
  2. unity_bill

    unity_bill

    Joined:
    Apr 11, 2017
    Posts:
    1,053
    I love it! thanks for sharing with everyone.
     
    Favo-Yang likes this.
  3. danilonishimura

    danilonishimura

    Joined:
    Jul 13, 2010
    Posts:
    70
    Really appreciated! It's a really nice approach. One thing I added after forking and modifying the project is a ruleset for assigning a custom group name, including path folders.

    Given a path "Assets/Level Assets/Props/Wooden Barrel/Barrel01.prefab"

    %PATH% returns "Assets/Level Assets/Props/Wooden Barrel/Barrel01.prefab"

    %PATH%[0] returns "Assets"
    %PATH%[1] returns "Level Assets"
    %PATH%[-1] returns "Barrel01.prefab" (cyclic)
    %PATH%[-2] returns "Wooden Barrel"

    So "%PATH%[-2]" would create an Asset Group called "Wooden Barrel" containing all barrels inside it.
    Beware that if by any reason, there is another folder also called Wooden Barrel elsewhere in the project, and the same kind of ruleset is applied, when that ruleset imports an asset, it will be sent to the same asset group.

    Also, I added an option in the ruleset to create group if it doesn't exist, and also to delete if there's no asset in it.

    Of course, if no variable is used, it behaves like what you have today.
     
    Last edited: Jul 30, 2019
    Whatever560 likes this.
  4. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    Awsome, would you mind create a PR? I will find a time to merge it, maybe slightly delayed due to the crunch time of another project.
     
    unity_bill and danilonishimura like this.
  5. danilonishimura

    danilonishimura

    Joined:
    Jul 13, 2010
    Posts:
    70
    Sure! No problem. Just have to make a backport to your original code due to structural changes on my version.
     
    unity_bill likes this.
  6. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    @danilonishimura I've merged your PR with a few changes
    - Group creation default off.
    - Changed %PATH%[0] syntax to ${PATH[0]} to keep consistent with regex replacement. Now it possible to use both ${PATH[index]} and ${group} in groupName/addressReplacement.
     
    danilonishimura likes this.
  7. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    Version 0.2.1

    Changes
    - Added address/group replacement. Working with regex rule to build the address/group based on information captured from the path.
    - Added path elements extraction for address/group replacement (i.e. `${PATH[0]}`)
    - Added option to automatically create groups if not exist, default False.
    - Added option to remove empty groups except the default group, default False.
    - Added option to define if labels from ruleset are added or replace the current ones.
    - Added save, documentation buttons.
    - Improved documentation.

    Thanks @AlkisFortuneFish @danilonishimura for the contributions.
     
    unity_bill likes this.
  8. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    Version 0.2.2

    Changes
    - Fixed simplified behavior.
    - Added unit tests.
     
  9. hanniiel

    hanniiel

    Joined:
    Jun 14, 2013
    Posts:
    39
    I've been following this repo while ago it's really amazing. <3
     
  10. danilonishimura

    danilonishimura

    Joined:
    Jul 13, 2010
    Posts:
    70
    @Favo-Yang Manually re-importing assets from many directories proved to be a bit tedious, so I made piece of code that triggers the Addressable import by selecting a folder and clicking "AddressablesImporter : Check sub folders".
    Will create a PR soon.
     
    Last edited: Aug 5, 2019
    Favo-Yang likes this.
  11. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    972
    Oh, cool, I had a hacky version of that I could not be bothered to tidy up and submit, cheers for doing it properly! :)
     
    danilonishimura likes this.
  12. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    Version 0.3.0

    Changes
    - Added context menu for faster asset import/update.
     
    unity_bill and danilonishimura like this.
  13. Colin_MacLeod

    Colin_MacLeod

    Joined:
    Feb 11, 2014
    Posts:
    332
    @Favo-Yang Love your importer - it's a wonderful thing.

    We currently include the contents of AddressableAssetsData in version control, and it occasionally gets out of sync.

    Not really sure of the specific sequence, but I'd really like to generate the addressables fresh in a build script. Then, I'd exclude the whole lot from git, apart from AddressableImportSettings.

    Do you have any experience doing this and/or any pointers?
     
  14. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    @Colin_MacLeod thanks for using the library.

    Your idea may be achieve-able, but you need understand the concept of group,schemas and templates well. So you can selective what commit to VCS and leave the dynamic part (group) to the importer. The importer has a group template feature may help.

    However consider all the works but little benefit, I would probably write a commit hook to force me commit the AddressableAssetsData folder. Depends on why "it occasionally gets out of sync", you may have a good reason to implement it.
     
  15. Colin_MacLeod

    Colin_MacLeod

    Joined:
    Feb 11, 2014
    Posts:
    332
    Yeah, I'm really not sure what's causing the issue, that's true. Seems to happen when 2 people create assets that map to addressables.

    Regarding version control, the only thing I seem to need to keep is the AddressableImportSettings instance. It seems I can regenerate everything from that, by importing the appropriate folders?
     
  16. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    If you only need default group, then probably yes, you can import default addressable folders, and leave the rest to the importer.

    If you need more or changed any settings, then you need commit other stuffs to VCS as well, but ignore Assets/AddressableAssetsData/AssetGroups/*. I never tried this approach, so it may have more issues.
     
  17. Colin_MacLeod

    Colin_MacLeod

    Joined:
    Feb 11, 2014
    Posts:
    332
    Not sure I fully understand @Favo-Yang. I am using the Group template but I am not creating any addressables that aren't matched by the AddressableImportSettings.

    It seems, to create the script I really just need to know the command to create default addressables (the button that appears on the addressables groups tab when you first open it), and the command to start the import for a given folder.
     
  18. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    The method is
    Code (CSharp):
    1. AddressableAssetSettingsDefaultObject.Settings = AddressableAssetSettings.Create(AddressableAssetSettingsDefaultObject.kDefaultConfigFolder, AddressableAssetSettingsDefaultObject.kDefaultConfigAssetName, true, true);
    You may soon meet two issues however,
    - what if you want to change some schemas or group settings, to share with others, you may want to keep related files to VCS.
    - what if you want to create dynamic groups using custom group template? you need put the template file to VCS, and template reference schemas... end up with lots related stuffs into VCS.

    That's why I said, you may want to only ignore Assets/AddressableAssetsData/AssetGroups/* from VCS. That's the only thing can be dynamic recreated and maintained by the importer.
     
  19. Colin_MacLeod

    Colin_MacLeod

    Joined:
    Feb 11, 2014
    Posts:
    332
    Really sorry, I think I am missing a key point here. Thank you so much for taking the time to clarify this with me.

    To try to make it clearer, let me show you one of our examples. I am using the group template - one of our rules creates groups for each avatar prefab type, like this:
    upload_2019-12-3_11-57-12.png

    But I know that if I delete everything out of the AddressableAssetDataFolder, recreate the addressables and then reimport, these groups are dynamically created for me, all based on the info run AddressableImportSettings. So I'm not seeing which bit would need me to keep the AssetGroups folder?
     
  20. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    You're not using custom group template, but the default one. In such case it's fine.

    upload_2019-12-4_2-50-32.png

    However, there's lots stuff you can configure on a group, bascailly everything under the content packing & loading, which it is actually a schema. The importer can not maintain those for you.

    upload_2019-12-4_2-52-36.png
     
    Colin_MacLeod likes this.
  21. Colin_MacLeod

    Colin_MacLeod

    Joined:
    Feb 11, 2014
    Posts:
    332
    Ah! Light goes on. That makes sense. Thank you so much for taking the time!
     
  22. MikeHergaarden

    MikeHergaarden

    Joined:
    Mar 9, 2008
    Posts:
    1,027
    Thanks, this makes assignment much easier! Using it in our first game with Addressables.
     
    danilonishimura and Favo-Yang like this.
  23. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    v0.4.2
    • Updated context menu path to create settings.
    • Added alternative way to install the package via openupm.
     
  24. madeinjamming

    madeinjamming

    Joined:
    Dec 12, 2013
    Posts:
    3
    Hey! Your tool looks cool!
    I am trying to make it work but seems I am doing smth wrong.

    I need only one feature from the tool (famous last words), to remove files extensions.

    I created "AddressableImportSettings" and select "Address Simplified" checkbox, then pressed "Save Project"
    but the images still have extensions

    Could u help me to set up the tool?


    upload_2021-3-27_10-5-1.png


    Code (CSharp):
    1. public class NewBehaviourScript : MonoBehaviour
    2. {
    3.     public Image _logi;
    4.     public Image _logb;
    5.    
    6.     void Start()
    7.     {
    8.         var bg = Addressables.LoadAssetAsync<Sprite>("Assets/RunTimeAssets/LoadingScreen/bg");
    9.         _logb.sprite = bg.WaitForCompletion();
    10.  
    11.         var logo = Addressables.LoadAssetAsync<Sprite>("Assets/RunTimeAssets/LoadingScreen/logo");
    12.         _logi.sprite = logo.WaitForCompletion();
    13.     }
    14. }
     
  25. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    You need to set up the path field to construct a rule, please follow Define Rules.

    To apply changed rule on imported assets, please follow Quick Assets Re-import.
     
  26. wesmag

    wesmag

    Joined:
    Mar 25, 2020
    Posts:
    34
    This is a very cool tool, and I was looking at doing something similar (albeit, very simplified). I was wondering though: We want to import assets from specified directories. i.e we only every include directories into the addressables groups, not files themselves, so we have something like:

    upload_2021-8-12_11-22-15.png

    And there is a rule thusly:

    upload_2021-8-12_11-22-55.png

    From what I can see, this won't pick up files that are duplicated or added as it is matching against the path. This, however, works with the Right Click -> Check Folders functionality as it will match perfectly against the folder and add the relevant files. OnPostprocessAllAssets is still kicked off when a file is added though, and I was thinking: would it be possible to check a select folder group (that is in the groups window perhaps) when this happens and add the new files, or delete the removed files? BTW, I did manage to add a rule where files are pick up, but they were not added to the folder groups but as a single file with the key as a complete path to the file.

    Or perhaps there is a better way of doing this?
     
  27. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    Hi @wesmag,

    I'm not sure if I understand you correctly. The importer didn't change any behaviour of the Addressables system. It just makes entries for you automatically. AFAIK, if you adding a folder itself as the entry in a group, the folder entry will make sure everything in the folder is applied to Addressabels. However, the UI didn't always get refreshed. i.e. after you make a folder entry, keep the Addressables group window open, add a new file to that folder in the project window, the Addressables group window won't be updated. If you close the group window and open it again, the newly added file will be there.
     
    phobos2077 likes this.
  28. Whatever560

    Whatever560

    Joined:
    Jan 5, 2016
    Posts:
    513
    I didn't look at it yet. Is path capture numbering reversed or reversable ? The main idea would be for it to kinda work like former unity resources folder.
    Ex :
    Assets/Some/Where/Cubes/CubeA.fbx
    Assets/Some/Where/Over/The/Rainbow/Cubes/CubeB.fbx

    "${PATH[^1]}/${PATH[^0]}" would give "Cubes/CubeA.fbx" and "Cubes/CubeB.fbx"

    Thanks so much for the tool.

    [Edit] : https://github.com/favoyang/unity-a...ion~/AddressableImporter.md#group-replacement

    You guys are perfect. If I get it right "${PATH[-1]}/${name}" would work ?
     
  29. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    I assume you're talking about the address replacement? (because taking the filename into a group name didn't seem reasonable).

    Then address replacement only works for regex match type, where you can use a ${PATH[index]} reference, an unnamed group reference $1, $2, ... or a named group reference ${name}. It doesn't support filename or extname (maybe a feature request?)

    But a little regex shall work

    Path: Assets/Some/.*/(?<filename>.+)\.(?<extname>.+)
    Address Replacement: ${PATH[-1]}/${filename}.${extname}

    upload_2022-10-31_17-26-15.png
     
    Whatever560 likes this.
  30. Whatever560

    Whatever560

    Joined:
    Jan 5, 2016
    Posts:
    513
    Thanks, I don't always have the best choice of words. Indeed we're talking about the adress and not the file hierarchy and name. The example you've shown is 100% what I wanted to achieve.
     
  31. nehvaleem

    nehvaleem

    Joined:
    Dec 13, 2012
    Posts:
    436
    First of all - thanks for your effort as it is a wonderful tool.

    I've got one question - is there a way to setup this tool to automatically filter assets by asset type, for example, scriptableobject subclass (like when searching the project with t:className). I would like to import all SOs with a given type as addressables. I can probably do it with an asset postprocessor, but still curious if something like this is possible or planned with addressable asset importer.
     
  32. Favo-Yang

    Favo-Yang

    Joined:
    Apr 4, 2011
    Posts:
    464
    There isn't a specific hook available for post-processing ScriptableObjects using AssetPostprocessor. However, you can achieve a similar effect by utilizing the OnPostprocessAllAssets method. By using AssetDatabase.GetMainAssetTypeAtPath, you can check the asset type and filter a specific subtype on the loaded object using AssetDatabase.LoadAssetAtPath.

    Please note that this approach may have performance implications as it can be resource-intensive (LoadAssetAtPath). Therefore, for this importer that relies solely on path detection, I would advise against implementing such a solution. Instead, consider using descriptive naming conventions for your ScriptableObjects, such as appending the class name as a suffix (e.g., "-classname.so").