Search Unity

  1. Are you interested in providing feedback directly to Unity teams? Sign up to become a member of Unity Pulse, our new product feedback and research community.
    Dismiss Notice

Feedback Automatic duplicate asset resolving

Discussion in 'Addressables' started by phobos2077, Jul 20, 2021.

  1. phobos2077

    phobos2077

    Joined:
    Feb 10, 2018
    Posts:
    320
    So we have a project with a lot of art assets (textures, animations, etc.), not all of them are being used in the final build, so naturally we want Unity to resolve the dependency tree from the scenes and root assets we load explicitly down to the individual shaders, animations, and sprites being used.

    Another requirement is we need to split our content into bundles based on when it's needed (when loading the menu scene, when going into action, etc.). But here comes the problem - duplicate assets.

    Let's say there are Asset A which references Asset B which references Asset C (so A->B->C). Now we have a bundle 1 which has asset A explicitly referenced via Addressables location, and also we have bundle 2 which has asset B explicitly. Now as you may know, this will result in asset C being duplicated in both bundles 1 and 2.

    The solution currently being offered means going to Addressables Analyze and pressing Analyze then Fix on the "Check Duplicate Bundle Dependencies". Which will put asset C explicitly into its own group. This has issues:
    1. This is manual action that needs to be done before every build. This is not an option for CI pipeline, so we would have to dig into source code and try & run this operation via script, then copy the result assets from the "Duplicate Asset Isolation" group that it creates into our own pre-configured group.
    2. It places all duplicate assets into one group, which is also not ideal for certain configurations. We might have thousands of duplicates due to how asset groups are used.
    Another solution might be to explicitly place entire folders of art assets into separate groups so that we don't have to use Analyze before every build, but it's also not a good option because it will not check if asset is referenced anywhere at all and ends up bloating the final build with unused assets.

    So this begs the question.. why doesn't Addressables automatically resolves asset duplicates during build? It might have used the obvious solution of placing the duplicated asset into the bundle with the shortest reference path. So that means if I added a Scene with an FBX into one bundle, and FBX itself into another bundle, it's obvious where I would want to store the art that FBX uses...

    Is there any such solution planned? Seems like another feature that's needed for practically 99% of projects.
     
    sameng likes this.
  2. TreyK-47

    TreyK-47

    Unity Technologies

    Joined:
    Oct 22, 2019
    Posts:
    1,588
    I'll flag this with the team for some insight!
     
    phobos2077 likes this.
  3. sameng

    sameng

    Joined:
    Oct 1, 2014
    Posts:
    60
    Great suggestion.

    I would love to have this as well, it would greatly improve the addressables build workflow.

    I think the solution presented would work really well for my project and be a standout benefit of addressables over a normal assetbundle workflow.
     
    phobos2077 likes this.
  4. andymilsom

    andymilsom

    Unity Technologies

    Joined:
    Mar 2, 2016
    Posts:
    110
    Thank you for the feedback. We have experimented with this in the past, and continue to look at ways to improve the workflow, feedback is always appreciated to understand your needs.

    In this regard there are particular issues that are tricky to resolve, and don't really have a good way to do so.

    A majority of projects have lots of groups. Let's say we have two groups that have a shared dependency. The question now is, how to decide on what group's settings to place the dependency in. One Group could be local, and another remote and we don't have a very reliable way to know this. If the remote group was chosen, then when loading the local group it can un-expectantly go download the dependency from using remote settings. This could cause runtime issues with what and where you are loading from not being strongly defined.

    Another is over-excessive bundle updates. If Assets move around regularly between bundles, it can cause a negative experience, both for hosting/download bandwidth and the users having to update often.

    One possible solution is to manage duplicates within the same Group, where there is Pack Separately mode (And no other Groups also reference that dependency). As you can be confident it has the same settings.

    Though at this time, we have no way that we could make this a default behaviour. It looks like in your case, and other. It will be beneficial to have some behaviour. An analyse script could be run (including in CI) which does what you are asking, placing dependencies into "dependency" groups, based on how they are used, and not just all in one group.
    I have made a task for us to look at adding a sample to our samples. Or modify the existing rule if we find it is better overall.
     
  5. phobos2077

    phobos2077

    Joined:
    Feb 10, 2018
    Posts:
    320
    How about adding automatic dependency resolution as an option that defaults to the current behavior? In our case we don't have two groups with the same explicit entries, duplicates only happen because of implicit dependencies, so just following the shortest path rule will be perfect. I'm sure it will be beneficial for many projects.

    Having to add duplicate dependencies explicitly into serialized group data seems like a hack, to be honest. Ideally you'd want to setup your project with a given structure (based on folders, etc) and then never touch Asset Groups again until you need significant structural changes in the project. I can't imagine asking artists or designers to go and add something manually to Asset Groups... this is just not going to work on bigger project.
     
  6. andymilsom

    andymilsom

    Unity Technologies

    Joined:
    Mar 2, 2016
    Posts:
    110
    We will continue to look at workflow and improvements, but I cannot give any confirmation or timeline on this.

    Including a Group is the only way right now. This can be automated in your build process, done as a prebuild step and removed after building and not have any interaction or even visibility to your designers.
     
    phobos2077 likes this.
unityunity