Will support auto-clean with restricted storage size?

Hi. I want to know whether Unity Accelerator for AssetPipline V2 will support auto-clean feature, like unity-cache-server-cleanup.

I used unity-cache-server-cleanup as daemon and it cleanup periodically exceeded caches, hope that Accelerator support it too.

 

Excellent point, @Kichang-Kim, and the Accelerator definitely takes care of disk storage management automatically. There is no other tool needed to manage space. In the future we anticipate the ability to have more knobs you can use to declare limits around this operation.

 

@bradunity I checked latest Unity Accelerator and found that there are many configurable options in unity-accelerator.cfg file.

I think that CacheMinFreeBytes, CacheMinFreePercent,
CacheEvictUntilFreeBytes, CacheEvictUntilFreePercent are related to auto-clean feature, but I can't sure what exactly means. Can you explain what exactly do these parameters?

 

@Kichang-Kim, yep, they certainly do. For now, we're not ready to officially support all the options in that configuration while we work through the beta release cycle to determine if these are the right set or if we need to potentially work or if we need to make changes. I would recommend leaving them with their default values while the product is in beta. When we have documentation for these options, we'll post a new forum post, and I'll try to also update this thread.

Currently, the Accelerator's eviction policy will begin evicting items from the cache when the volume hosting the configured storage directory reaches 5% or 1 GiB of free space remaining (whichever is reached first). Evictions will continue until the volume's usage has at least 5 GiB or 10% free space available.

We'd also love to hear thoughts from yourself and any other interested community folks around what you'd like to see regarding managing items in the cache. Some questions we've been mulling are...

• Should some items be "stronger" than others? For example, would you find it valuable to have results from your official build system have "precedence" over those built by normal developers using the Unity Editor?
  • Would you like to manage this "strength" by who or what was running the build? For example, would you like to apply a "weight" to the items based on authentication to influence the eviction policy?
• Should some items live "forever" and never be evicted? For example, perhaps you'd like to never evict those items that were built as part of a specific release, or other types of criteria.
  • Would you like to also have an option to replicate these types of items to a more permanent storage facility for later retrieval if the cache were lost? If so, what types of storage would you consider important (e.g. local object stores, S3, Google Cloud Storage, etc.)?
• What other kinds of eviction policies would you value?

 

Thanks for clarification. Here are my feedbacks:

1. My cache server is running on build server which has other process, like jenkins for client/assetbundle build. Then some of them shares same storage. So I think that "usage" based eviction policy is need. ex) Keep cache not to exceed 200GB. (like old cache server policy).

2. I don't need permanent cache right now, but officially supported back-up feature is welcome. (If I lost my cache, re-importing from scratch consume half~one day.)

3. My project use lots of lightmaps and textures and it is changed daily. Actually, most asses are changed frequently. So simple LRU policy is sufficient. But I think that "processing-time" based policy also may be good choice. Because lightweight assets are not important and don't need to be cache.

 

I just want to set a maximum in GB, independent of how much percentage diskspace I have left.
( minus 1 GiB of free space remaining )

 

We have added this feature in the release announced today. You can now provide quotas for how much data the Accelerator is allowed to use by using the following new configuration options and restarting the Accelerator:

• MaxUsedBytes and MaxUsedPercent trigger eviction when storage used by the Accelerator exceeds either of these settings.
• EvictUntilUsedBytes and EvictUntilUsedPercent are eviction thresholds: eviction ends when both of these conditions are met. When MinFree fields are also set, eviction is triggered by the most sensitive setting. Likewise eviction is stopped when all of the EvictUntil values are satisfied.
• Use a value of -1 to ignore any property monitoring (MaxBytes or MinFree).

 

@bradunity Thanks for update! but I checked my old unity-accelerator.cfg and found that there are CacheXXX parameters.

Did you mean that CacheMaxUsedBytes? or MaxUsedBytes? also, how to create latest version of default config file?

Thanks.

 

@Kichang-Kim, I'm really sorry. It appears we had a miscommunication in our release process last week and this new feature missed that release and, instead, will be rolled out this week (on Thursday) assuming QA passes cleanly.

The configuration file is automatically updated when the new Accelerator is installed and run. If you want to create a default version of the file, the easiest method is to run it from a different directory like this: unity-accelerator
register adbv2. Afterward, you should find a new unity-accelerator.cfg in that directory with the default values.

 

@Kichang-Kim we pushed the new version out today, as I mentioned above. You are correct in that the setting is
CacheMaxUsedBytes and to help folks out, we've added a new tool to document the configuration options. Try out
unity-accelerator tool config-help to see the full list.

 

I confirmed that there are new options for max used bytes eviction. Thanks.

 

@bradunity Hi, I found that something strange behaviour. When I set "-1" for all of cache limit and eviction policy except MaxUsedBytes and EviecUntilUsedBytes, some properties are automatically restored to default value after a while.

 

Thanks for highlighting that behavior, @Kichang-Kim. It won't affect your settings as it's simply providing the default values based on the size of available storage. We have opened a ticket to make these behave like the newly added Max options to prevent future confusion when folks use -1 as a value for the older Cache options.