Search Unity

Unity Accelerator - Editors upload to cache, but never download.

Discussion in 'Unity Accelerator' started by JJRivers, Feb 10, 2021.

  1. JJRivers

    JJRivers

    Joined:
    Oct 16, 2018
    Posts:
    73
    First time attempting for our project to use the Unity Accelerator for ADB2 caching with Unity Accelerator.
    I have gotten it to function in the sense that all remotes and locals do upload and logs show that the put requests go through and it is indeed creating artifacts in the cache when a user changes something.

    But we can't get editors to download said artifacts.

    Now obviously the tool is highly oriented towards Unity Collaborate but i've seen it posted in several places that we should be able to use the ADB2 caching functionality even if we do not use Unitys version of source control (we use Git).

    Most of the time we spend on asset imports (think texture crunches and the like) not with the transfer of source files which don't take much time.

    So have i misunderstood what the function of the ADB2 server is and if not, why aren't client editors recognizing the artifacts another user uploaded as the same file and fetching that instead of importing that.

    Tested with material / texture files.


    I have a simple wizard install of Accelerator with no encryption and have verified that the connections to / from the server work. Logs show clients successfully doing put requests to the database, but all get requests fail with:

    Code (CSharp):
    1.   {
    2.     "level": "debug",
    3.     "ts": "2021-02-10T11:48:30.174+0200",
    4.     "msg": "handled getRequest",
    5.     "agent_id": "DesktopWork_id",
    6.     "agent_name": "DesktopWork",
    7.     "component": "pbservice",
    8.     "subprocess_id": 6,
    9.     "conn_id": 2,
    10.     "remote": "edited out for obvious reasons",
    11.     "segment_id": 13592,
    12.     "namespace": "81e94844d19a16919208533e08183531",
    13.     "key": "2916b0e0f0dd01b28018ff1e88323dd5",
    14.     "status": "key not found",
    15.     "status_code": 3,
    16.     "size": 0
    17.   },
    This leads me to suspect there is something that must be done to ensure all editors use the same method of saving the artifacts but i haven't found any mention or documentation to this. If it exists then it would be great if you could point me to it.
     
  2. henriksc

    henriksc

    Unity Technologies

    Joined:
    Aug 27, 2020
    Posts:
    21
    On the surface, your setup sounds good. And yes, the Accelerator can be used even if you are not using Collaborate. The "CacheServer" feature is completely source control independent, so you can use what ever VCS flavour you want.

    By default the editor should try to download import results and upload after an import. There are project settings and command line overrides to change this behaviour. Look at the "Configuring Editors to use an Accelerator" section in the Accelerator documentation https://docs.unity3d.com/2021.1/Documentation/Manual/UnityAccelerator.html

    If you send your editor.log and accelerator.log, then I can have a look at it and hopefully point you in a more specific direction.
     
  3. JJRivers

    JJRivers

    Joined:
    Oct 16, 2018
    Posts:
    73
    Thank you for your answer i will be posting logs tomorrow or at the latest over the weekend once i save some activity in the logs so you can actually see what the issue is.

    edit: And to clarify, current setup has all editors with download/upload both enabled, TLS off for both the Accelerator and the Editors.
     
  4. JJRivers

    JJRivers

    Joined:
    Oct 16, 2018
    Posts:
    73
    Attached is a zip with the relevant logs and a very brief README on what were the actions taken to generate them.
     

    Attached Files:

  5. henriksc

    henriksc

    Unity Technologies

    Joined:
    Aug 27, 2020
    Posts:
    21
    From your logs there are two things that stand out to me.

    1. It seems to take a very long time to actually get connected to the accelerator. Normally this happens right in the beginning when starting up an editor. But here it seems to fail the first attempt and eventually gets connected later on. It's difficult to say why this is, it's not something we've seen before. One thing you could try is to use the command line option "-cacheServerWaitForConnection 5000", it will force the editor wait for a connection for 5 seconds before continuing. You can also just pass "-cacheServerWaitForConnection" which will force it wait forever until a connection is established.

    2. Once it's connected, it seems to be working properly. Tries to download metadata and if not available it does the import and uploads the result.
    However if I look at an asset like "Assets/+RedStage/RawAssets/EnvironmentRaw/BushesRaw/Textures/T_Willow_Bush_02_Cross_A_T.png", its static dependency hash is different between the local and remote editor logs. Therefore it doesn't find it and ends up doing an import and upload. One reason for this, that I've seen before, is when line endings differ between machines. For instance, depending on configuration Git might select LF on one machine and CRLF on another. This would result in different hashes of the meta and content files.

    Let me know if any of this helps you.
     
  6. JJRivers

    JJRivers

    Joined:
    Oct 16, 2018
    Posts:
    73
    Apologies this was put on ice for the moment and i haven't tested yet. These sound something reasonable to test and i will do so within the coming week and get back to you. Thank you very much for your effort at helping us. Get back to you soonest we can!
     
unityunity