Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

What's the difference from the Cache Server.

Discussion in 'Unity Accelerator' started by chrisk, Sep 11, 2019.

  1. chrisk

    chrisk

    Joined:
    Jan 23, 2009
    Posts:
    700
    Regarding the asset importing, what's the difference between the Cache Server and the Accelerator?
    Thanks.
     
    Peter77 likes this.
  2. marie-unity

    marie-unity

    Unity Technologies

    Joined:
    Aug 6, 2019
    Posts:
    65
    The Cache Server supports Asset Pipeline v1 only. The Accelerator supports Asset Pipeline V2 (available in 2019.3). They’ll coexist for the foreseeable future. You can actually use the Accelerator wizard to install the legacy cache server.
     
    chrisk likes this.
  3. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    800
    Please, oh please, make a big fat table at the top of the blog post to state that.
    Asset Pipeline v1 (<=2019.2): uses Cache Server
    Asset Pipeline v2 (>=2019.3): uses Unity Accelerator
     
    elechen and joshcamas like this.
  4. JurjenBiewenga

    JurjenBiewenga

    Joined:
    Mar 28, 2018
    Posts:
    24
    Looking at the installer it appears that Accelerator also supports V1
     
  5. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    800
    Yeah, it looks like that, but actually it just installs Cache Server if you select that option if I got that right from the blog post / comments below.
     
  6. StaceyH

    StaceyH

    Unity Technologies

    Joined:
    Apr 11, 2018
    Posts:
    44
    Great questions - I've created some initial Q&A for the FAQ on the welcome thread, which will be updated soon. In the meantime, I thought two of them might also be useful to post on this thread. I'm hoping that this helps clarify, but if not let me know and I can adjust.

    What is the difference between the Unity Accelerator and the Cache Server?
    The most important difference is that the original cache server only supports Asset Pipeline v1. The Unity Accelerator adds caching support for both Asset Pipeline v2 (coming soon) and source code via the Collaborate service (available now).

    Another great benefit of the Unity Accelerator is that when using it with Asset Pipeline v2 and/or Collaborate service, it features automatic disk storage management. This means that it will periodically clean up unneeded assets and/or source. With the original Cache Server, teams often had to use additional tooling to manage disk space.

    A third difference is that the Unity Accelerator also exposes metrics that provide you with useful information such as bytes transferred, editors connected and more. They are hosted as Prometheus metric values as /metrics, which you can query from the local network.

    I noticed that you can install the Legacy Cache Server in the Unity Accelerator installation wizard. Does this mean it has new functionality?
    We have not added any new functionality to the legacy cache server. The decision was made to include it in the Unity Accelerator installation wizard so that teams only needed to go to a single place to manage installations.
     
  7. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    800
    Here's some more questions:

    Q: Which version of Cache Server is used in the Accelerator Installer?
    For Cache Server, it was recommended that only the Cache Server version released together with the specific version of Unity Editor was used (that's why each Unity release comes with a separate Cache Server download).

    Q: Does each Unity Accelerator instance only support a specific Unity version or will it be able to work with multiple simultaneous Unity installs/projects (starting with 2019.3)? (same link as above)

    Q: Have the shortcomings of Cache Server & v1 been resolved in Accelerator & v2? See the Unity docs that basically say "don't use Cache Server on a Windows machine". Also, see this paragraph for further issues that it had.

    Q: Does Unity Accelerator have options? Technical documentation besides that quickstart PDF? See this link for technical documentation and configuration options of Cache Server.
     
    Last edited: Sep 19, 2019
    bradunity likes this.
  8. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    Here's what I've learned/gathered (apologies for the delay), @fherbst :
    1. Which version of Cache Server is used in the Accelerator Installer?
      • The installer uses npm to get the latest published version of the V1 Cache Server at the time of installation.
      • The thread you linked to doesn't seem to indicate any recommendation regarding a correlation between the version of Cache Server and the version of the Editors.
        • What Stephen mentions is that we recommend using a different V1 Cache Server for each Project (to avoid collisions). The download links from the Unity download archive are all directed to the Unity Cache Server Github repository to ensure folks are using the latest published version which includes important bug fixes.
        • When using the new Asset Import Pipeline (V2), the Accelerator can handle caching for many projects supported by one Accelerator.
    2. Does each Unity Accelerator instance only support a specific Unity version?
      • No, the Accelerator will remain backward compatible, supporting all versions of the V2 Asset Import Pipeline starting in 2019.3 and later. It will automatically upgrade itself as needed to make this seamless. As mentioned in #1 above, it can simultaneously support multiple projects.
    3. Have the shortcomings of Cache Server & v1 be[en] resolved in Accelerator & v2?
      • Yes, the new Asset Import Pipeline and the Accelerator have been designed and built with older problems in mind to avoid known issues we've encountered in the past. For example, the new Asset Import Pipeline is much more deterministic and tracks dependencies to understand when a dependent asset must be reimported. There are many other improvements that will be detailed in future documentation for the new pipeline. I'll update this thread when they are publicly available.
      • The Accelerator performs just as well on Windows as Mac OS X or Linux, provided the minimum operating system versions (https://docs.unity3d.com/2019.3/Documentation/Manual/UnityAccelerator.html).
    4. Does Unity Accelerator have options?
      • Yes, the Accelerator has options, but we've elected to wait on providing full documentation around their use during the early stage of beta as they can make it harder for us to help troubleshoot if any unforeseen issues arise with the default setup and are subject to change.
      • Some of the more interesting options that we have are the ability to enable more debugging logs and to explicitly choose ports to listen on. I'll update this thread when we publish details on all the various options in the unity-accelerator.cfg file found in the storage directory chosen during installation. But if there's anything you'd like to try, let me know and I'd be happy to walk you through what it does and how to tweak it. Also, during this Beta testing, we'd love to hear about things missing that you need or would like to have control over.
     
    Last edited: Sep 19, 2019
    fherbst likes this.
  9. joshcamas

    joshcamas

    Joined:
    Jun 16, 2017
    Posts:
    1,242
    So we don't need to use collaborate to use the accelerator, right?
     
  10. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    joshcamas likes this.
  11. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,286
    Will this also provide background compression of textures so the editor is not blocked when compressing textures?
     
  12. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    Hi!

    Our team is starting to look at this new tool, we were wondering if Accelerator is using the same command line argument to set the ip address ?
    At this moment, cache server is using "-CacheServerIPAddress <ip : port>", should we assume it will be the same argument used to setup the ip address to Accelerator in a project using the Asset pipeline v2 ?

    Thanks!
     
  13. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    No, @dzamani, the new command line argument for using the Accelerator is -cacheServerEndpoint. So, for example, if you were to start a brand new project that does not have the new Asset Import Pipeline enabled nor any pre-configured settings in the preferences for a Cache Server, you can do something like the following:

    -adb2 -enableCacheServer -cacheServerEndpoint 127.0.0.1:10080
     
    dzamani likes this.
  14. dzamani

    dzamani

    Joined:
    Feb 25, 2014
    Posts:
    122
    I see, I will need to update my jenkinsfile to handle that, I wasn't aware that enableCacheServer existed either.
    For backward compatibility, does something like -adb1 exist ?
     
  15. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    Yes, there is an -adb1 option, as well. We'll be updating our documentation to help make this more clear, too (should be updated here before the 2019.3 is released from the beta process).
     
    dzamani likes this.
  16. RG_Keith

    RG_Keith

    Joined:
    Mar 14, 2017
    Posts:
    35
  17. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    RG_Keith likes this.
  18. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    Regarding @fherbst's question above...
    This week's release has a new tool to help you understand all the configuration options available to the Accelerator by running unity-accelerator tool config-help.
     
    RG_Keith and fherbst like this.
  19. svorobyov

    svorobyov

    Joined:
    Feb 6, 2020
    Posts:
    3
    Hi!

    Previously, with CacheServer and Pipeline V1 it was possible to give parameters, like
    LocalCacheServerPath
    LocalCacheServerSize
    to use LOCAL caching in addition to the network CacheServer.
    Do these settings exist for Accelerator and pipeline V2?
    How can they be given using CLI with Unity.2019.3+?
    Without these V2 will never beat V1 in speed.
     
  20. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    @svorobyov, the new v2 pipeline uses its own database (stored in the Library folder) so there is no need for running a separate local cache server. Currently, there are no options to choose the location of this local database.

    In all of our testing v2 outperforms v1. If you find a situation where that is not the case and are willing to share the details of your project's setup, we'd love to learn more about it.
     
  21. DinoDiniBossa

    DinoDiniBossa

    Joined:
    Jun 8, 2020
    Posts:
    5
    I am seeing a situation where on starting up a project, importing occurs and it appears to be sending to the server assets that it should be able to pull because they would have been set up previously by another team member. Anyone else seen this or have any suggestions? Thanks.
     
  22. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,286
    I want to share the compressed textures cache between projects in v2, how do I do that?
    If they are stored locally in each project library folder?

    And I have to ask how does the accelerator work.
    My assumtion:

    1. Asset cache data base
    2. Project Library folder

    I have the accelerator storage set the an external 1 tb ssd
    And my unity project is also on the same external 1 tb sdd

    Does this mean that the cache is stored twice in the the external ssd?
     
  23. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    @Lars-Steenhoff , the Accelerator is a storage facility that the Asset Import Pipeline can optionally utilize to skip importing of new or changed assets if another member on your team has already processed them. It does not make local development any faster. The Asset Database already handles local caching for you so adding an Accelerator would be superfluous and will double the amount of storage as you've suggested.
     
  24. Lars-Steenhoff

    Lars-Steenhoff

    Joined:
    Aug 7, 2007
    Posts:
    3,286
    @bradunity
    Thanks for the details, one thing that is still not 100% clear is:

    How to share compressed textures between projects?

    I'm not working in a team, just different project locally all using the same asset store packages.

    The compression takes to much time that I don't want to have each project compress them again
     
  25. bradunity

    bradunity

    Unity Technologies

    Joined:
    Nov 12, 2013
    Posts:
    195
    Lars-Steenhoff likes this.
  26. krakentanz

    krakentanz

    Joined:
    Aug 13, 2020
    Posts:
    31
    We seem to have the exact same situation as described by @DinoDiniBossa above. Any pointers? Dino, did you get it to work in the mean time?

    Generating lightmap-UVs on the machine running the Accelerator Server. Saving, commiting and pushing to our git repo. Client pulls the commit, opens Unity and the dialog-box informs that it's importing the very asset for which UVs were recently generated on the server side. Takes ages, so i strongly assume it's doing the unwrapping again, which Accelerator is supposed to prevent it from doing, right? What are we missing here?

    * Both machines on the same network.
    * Accelerator-Connection successfull (judging from the message in the settings dialog and the icon in the satus bar of unity editor)
    * uta_agent_protobuf_cache_hits 2 (we tested it twice, so this number seems legit, but the UVs still get re-calculated on the client-machine for the asset we used for testing)
    * Accelerator log shows communication between the two computers, some succeed, some fail to deliver cached assets:
    from the log:
    Code (CSharp):
    1. {"level":"debug","ts":"2020-12-11T13:57:41.425+0100","msg":"handled putRequest","agent_id":[SERVERNAME]_id","agent_name":"[SERVERNAME]","component":"pbservice","subprocess_id":6,"conn_id":17,"remote":"[client ip]:55057","segment_id":487,"namespace":"81e94844d19a16919208533e08183531","key":"d0df3b3bdfeb099441e1fdd435ed8f40","size":1114,"status":"ok","status_code":0}
    2. {"level":"debug","ts":"2020-12-11T13:57:42.330+0100","msg":"handled getRequest","agent_id":"[SERVERNAME]_id","agent_name":"[SERVERNAME]","component":"pbservice","subprocess_id":6,"conn_id":17,"remote":"[client ip]:55057","segment_id":488,"namespace":"shadercache","key":"57991bd20e962d2b5e56f4dc6f48678d","status":"key not found","status_code":3,"size":0}
    what are we missing?
     
    Last edited: Dec 14, 2020