Feedback Feature request: PackTogetherByPath

This is partly a feature request and partly a "could you integrate something I have already done internally". The use-case is the following:

We have assets that are items that a player can buy and equip. Each item is described by a data file. A lot of items are variants of the same mesh, sharing most assets but overriding some. An simplified asset structure would be:

Shirts/Shirt 001/Shirt 001 Red.asset
Shirts/Shirt 001/Shirt 001 Green.asset
Shirts/Shirt 001/Shirt 001 Blue.asset
Shirts/Shirt 001/Shirt 001.mesh
Shirts/Shirt 001/Shirt 001_Red_Albedo.png
Shirts/Shirt 001/Shirt 001_Green_Albedo.png
Shirts/Shirt 001/Shirt 001_Blue_Albedo.png
Shirts/Shirt 001/Shirt 001_Normal.png
Shirts/Shirt 001/Shirt 001_MetallicSmoothness.png
Shirts/Shirt 001/Shirt 001_AO.png

The three data assets are addressable and their group should be set to PackSeparately and individually downloadable. Now, if each one pulls its dependencies, we end up with duplication of all common assets. The version of our game currently live just allows this duplication. This, however, results in nearly 800% increase of data size, extremely long asset bundle build times of 1.5 hour cached, 4-6 hours uncached, and a poorer user experience, as downloads are bloated unnecessarily.

Now, we would like to extract the common dependencies out of of there and into their own group, which
CheckBundleDupeDependencies does beautifully already. However, none of the existing packing modes make any sense for that group. Using PackSeparately results in ridiculously excessive numbers of asset bundles and PackTogether defeats the purpose of splitting into bundles to start with.

We *could* do something funky with labels and PackTogetherByLabel but what seemed to be much simpler was adding a PackTogetherByPath mode. This is basically a clone of PackTogetherByLabel, but instead of grouping on the label set of the assets, it groups on the path. I had to do some annoying juggling with the asset bundle names (taking the 3 last path segments) as we ended up hitting max path length in the cache but fundamentally it just works and seems to be the easiest solution for this use-case, which I would imagine is fairly common.

The resulting bundles are:

Wearables_Assets/Shirts/Shirt 001/Shirt 001 Red.bundle
-- Shirts/Shirt 001/Shirt 001 Red.asset
-- Shirts/Shirt 001/Shirt 001_Red_Albedo.png
Wearables_Assets/Shirts/Shirt 001/Shirt 001 Green.bundle
-- Shirts/Shirt 001/Shirt 001 Green.asset
-- Shirts/Shirt 001/Shirt 001_Green_Albedo.png
Wearables_Assets/Shirts/Shirt 001/Shirt 001 Blue.bundle
-- Shirts/Shirt 001/Shirt 001 Blue.asset
-- Shirts/Shirt 001/Shirt 001_Blue_Albedo.png

DuplicateAssetIsolation_Assets/Shirts/Shirt 001/Shirt 001.bundle
-- Shirts/Shirt 001/Shirt 001.mesh
-- Shirts/Shirt 001/Shirt 001_Normal.png
-- Shirts/Shirt 001/Shirt 001_MetallicSmoothness.png
-- Shirts/Shirt 001/Shirt 001_AO.png

This is far more reasonable.

PS: I was about to report that PackTogetherByLabel bug, inadvertently found it by copying it, but looks like it's sorted in the latest release!

 

I'm working on this feature in a open source project by @Favo-Yang.
https://forum.unity.com/threads/open-source-rule-based-unity-addressable-asset-importer.702383/

You can download it as it is now to make yourself familiar, and get the update soon (hopefully today). As it is internally, it doesn't have all the features you described, but I'll make sure to try cover these aspects in the future.

 

Thanks, but that is not actually the same for a few reasons:

I evaluated splitting each asset in its own group and had a script doing that already. It is completely unmanageable and heavy handed in our case, because we have thousands of assets and would end up with thousands of groups.

The closest I could get, with my own fork of the @Favo-Yang's importer, is extracting information from the path fragments and adding it to labels that could then be used with PackTogetherWithLabels, but that ended up being quite problematic too, because the real structure of the assets is far more complicated than what I showed there. The only real way to extract dependencies is through actual extraction of dependencies. The assets we're talking about are UMA assets and the combinations of various features used varies a lot and hence the dependency graph is both large and very varied.

PS: Amusingly enough, I have my own modifications I need to PR to @Favo-Yang's importer at some point, I've added address generation based on regex capture groups, it was crying for it considering it already uses regex. You get an extremely powerful data extraction and text replacement service for free with Regex.Replace().

For example, my path rule for the main recipe assets is:
Assets/UMA_Assets/Outfits/(?<prefix>(?<category>[^/]+)/(.*/)*)(?<asset>.*_Recipe.*\.asset)
This then feeds the address with:
Wearables/${category}/${asset}