Search Unity

Will support auto-clean with restricted storage size?

Discussion in 'Unity Accelerator' started by Kichang-Kim, Sep 12, 2019.

  1. Kichang-Kim

    Kichang-Kim

    Joined:
    Oct 19, 2010
    Posts:
    331
    Hi. I want to know whether Unity Accelerator for AssetPipline V2 will support auto-clean feature, like unity-cache-server-cleanup.

    I used unity-cache-server-cleanup as daemon and it cleanup periodically exceeded caches, hope that Accelerator support it too.
     
  2. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    91
    Excellent point, @Kichang-Kim, and the Accelerator definitely takes care of disk storage management automatically. There is no other tool needed to manage space. In the future we anticipate the ability to have more knobs you can use to declare limits around this operation.
     
    Lars-Steenhoff and Kichang-Kim like this.
  3. Kichang-Kim

    Kichang-Kim

    Joined:
    Oct 19, 2010
    Posts:
    331
    @bradunity I checked latest Unity Accelerator and found that there are many configurable options in unity-accelerator.cfg file.

    I think that CacheMinFreeBytes, CacheMinFreePercent,
    CacheEvictUntilFreeBytes, CacheEvictUntilFreePercent are related to auto-clean feature, but I can't sure what exactly means. Can you explain what exactly do these parameters?
     
  4. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    91
    @Kichang-Kim, yep, they certainly do. For now, we're not ready to officially support all the options in that configuration while we work through the beta release cycle to determine if these are the right set or if we need to potentially work or if we need to make changes. I would recommend leaving them with their default values while the product is in beta. When we have documentation for these options, we'll post a new forum post, and I'll try to also update this thread.

    Currently, the Accelerator's eviction policy will begin evicting items from the cache when the volume hosting the configured storage directory reaches 5% or 1 GiB of free space remaining (whichever is reached first). Evictions will continue until the volume's usage has at least 5 GiB or 10% free space available.

    We'd also love to hear thoughts from yourself and any other interested community folks around what you'd like to see regarding managing items in the cache. Some questions we've been mulling are...
    • Should some items be "stronger" than others? For example, would you find it valuable to have results from your official build system have "precedence" over those built by normal developers using the Unity Editor?
      • Would you like to manage this "strength" by who or what was running the build? For example, would you like to apply a "weight" to the items based on authentication to influence the eviction policy?
    • Should some items live "forever" and never be evicted? For example, perhaps you'd like to never evict those items that were built as part of a specific release, or other types of criteria.
      • Would you like to also have an option to replicate these types of items to a more permanent storage facility for later retrieval if the cache were lost? If so, what types of storage would you consider important (e.g. local object stores, S3, Google Cloud Storage, etc.)?
    • What other kinds of eviction policies would you value?
     
  5. Kichang-Kim

    Kichang-Kim

    Joined:
    Oct 19, 2010
    Posts:
    331
    Thanks for clarification. Here are my feedbacks:

    1. My cache server is running on build server which has other process, like jenkins for client/assetbundle build. Then some of them shares same storage. So I think that "usage" based eviction policy is need. ex) Keep cache not to exceed 200GB. (like old cache server policy).

    2. I don't need permanent cache right now, but officially supported back-up feature is welcome. (If I lost my cache, re-importing from scratch consume half~one day.)

    3. My project use lots of lightmaps and textures and it is changed daily. Actually, most asses are changed frequently. So simple LRU policy is sufficient. But I think that "processing-time" based policy also may be good choice. Because lightweight assets are not important and don't need to be cache.
     
    gregoryh_unity likes this.