Search Unity

Help Wanted How does Accelerator decide when to rebuild assets, when to use cache?

Discussion in 'Unity Accelerator' started by SlavDok, Feb 13, 2020.

  1. SlavDok

    SlavDok

    Joined:
    Feb 13, 2020
    Posts:
    2
    Newbie here, maybe this had already been explained back in Cache Server days.
    But how does the Accelerator decide when to rebuild assets, and when to use the available cache?

    Let's say I have Subversion Trunk checked out among 2 developers, both connecting to the same Accelerator.
    Dev 1 checks out project, imports assets, and this populates Accelerator cache.
    Dev 2 checks out project, and pulls assets from Accelerator cache.
    So far all is good.

    Now, Dev 2 switches to a different branch, and starts makes changes to assets.
    When he builds, he will upload "branch-version" of assets to Accelerator cache.

    Then, Dev 3 checkout projects.... How is Accelerator able to distinguish between the Assets from different branches? Unity is not aware which SVN branch it's in. Project name is same, asset name is same.

    Should I somehow forbid developers from uploading to Accelerator server? Only allow CI builds to populate Accelerator cache?

    How do I enable Accelerator cache for read-only for Developers?
     
  2. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    149
    First item to highlight is that the logic is part of the new Asset Import Pipeline, not the Accelerator. Some details about it are described in our docs (but more information will be coming). The Accelerator is a proxy and caching agent that the pipeline coordinates with to save the imported version of the assets, often referred to as "artifacts".

    Regarding your example, as noted in the section titled, "The source Asset Database," you'll find that part of the key used when looking for an artifact includes the hash of the file's contents. Artifacts may be used when working on the different branch only if they—and their dependencies!—remain unchanged. Therefore, you get the benefit of not having to reimport those assets Dev 2 isn't actively working on, and no concern over adversely impacting Dev 3 from still utilizing the artifacts that are generated from the "mainline" branch.

    Additionally, if you want to enforce full separation between branches, we have an option in the Edit → Project Settings... → Editor → Cache Server section labeled Namespace prefix. In your example, Dev 2 could set this value to be "mybranch" and guarantee all their artifacts will never be used or accessed by Dev 1 or Dev 3. The downside, of course, is they also lose the reusability of those artifacts that have not changed and will be forced to go through the import process.

    Regarding a read-only option, there isn't a way to enforce it from the Accelerator (yet!). However, you can use the same project settings section I mention above and uncheck the Upload element. This will mean that artifacts are only ever retrieved from the Accelerator when working in the project and never saved to the Accelerator. Then your CI/CD system can use the -cacheServerEnableUpload true option I describe in this post to allow your automation runs to populate the Accelerator.
     
    alexvector and SlavDok like this.
  3. SlavDok

    SlavDok

    Joined:
    Feb 13, 2020
    Posts:
    2
    Perfect, thank you for the detailed explanation! :)

    A little side question: if the assets' hashes are unique across branches (if they changed), why would someone choose to download but not upload to Accelerator server? Just bandwidth and disk space considerations?

    Thank you
     
  4. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    149
    The asset hashes are not unique across branches. They are only unique when their content is changed. This is usually safe to rely on in most cases (especially given the pipeline's new ability to use dependency analysis to also recognize when something related should require a new import). However, there are still cases that could result in overwriting an artifact stored in the Accelerator improperly. For example, if you happen to be using ScriptedImporter with caching enabled and the results are not deterministic or if a change was made and the developer forgot to bump the importer's version. In this type of environment, some developers like the safety of knowing only official artifacts are stored in the Accelerator.
     
    SlavDok likes this.
unityunity