Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Cache Server 6.3.0 exits silently when trying to run from previous "cache_ram" data

Discussion in 'Cache Server' started by lasercannon, Jun 5, 2019.

  1. lasercannon

    lasercannon

    Joined:
    Nov 29, 2012
    Posts:
    72
    Hello,

    We've had our cache server with the RAM-backed module running for a few weeks now, and today I had to restart the server. The problem is that the cache server will not start up again (it closes with no log messages other than a message like "Cache path is 'H:/unity-cache/cache_ram'", UNLESS I clear out the page files from the cache folder. It seems that it's failing to load our previous cache state. What's going on here?

    For reference, here is the text of our config file. The biggest changes we made to the previous default.yml is the module type, cache paths, and maxPageCount from 10 to 100.

    Also for reference, I'm starting the cache server up by running 'unity-cache-server -nolegacy'. I've also tried just 'unity-cache-server'.

    Code (CSharp):
    1. Cache:
    2.   ## the cachePath key, if set, will override 'cachePath' within the individual module options below
    3.   # cachePath: "H:\unity-cache"
    4.   defaultModule: "cache_ram"
    5.   options:
    6.     processor:
    7.       putWhitelist: []
    8.     cache_ram:
    9.       cachePath: "H:/unity-cache/cache_ram"
    10.       pageSize: 100000000
    11.       maxPageCount: 100
    12.       minFreeBlockSize: 1024
    13.       persistence: true
    14.       persistenceOptions:
    15.         autosave: true
    16.         autosaveInterval: 10000
    17.         throttledSaves: false
    18.       highReliability: false
    19.       highReliabilityOptions:
    20.         reliabilityThreshold: 2
    21.         saveUnreliableVersionArtifacts: true
    22.         multiClient: false
    23.     cache_fs:
    24.       cachePath: "H:/unity-cache/cache_fs"
    25.       cleanupOptions:
    26.         expireTimeSpan: "P30D"
    27.         maxCacheSize: 0
    28.       persistence: true
    29.       persistenceOptions:
    30.         autosave: true
    31.         autosaveInterval: 10000
    32.         throttledSaves: false
    33.       highReliability: false
    34.       highReliabilityOptions:
    35.         reliabilityThreshold: 2
    36.         saveUnreliableVersionArtifacts: true
    37.         multiClient: false
    38. Mirror:
    39.   addresses: []
    40.   options:
    41.     queueProcessDelay: 2000
    42.     connectionIdleTimeout: 10000
    43. Server:
    44.   port: 8126
    45.   options:
    46.     allowIpv6: false
    47. Global:
    48.   logLevel: 3
     
  2. lasercannon

    lasercannon

    Joined:
    Nov 29, 2012
    Posts:
    72
    New findings:

    • Still happens when "persistent" is false
    • Still happens when increasing maxPageCount
    • Just tried running the cleanup script, and I get the following error, which may have something to do with it:

    Code (CSharp):
    1. C:\Users\Administrator>unity-cache-server-cleanup
    2. Cache path is H:/unity-cache/cache_ram
    3. Error: Duplicate key for property fileId: 352db1b773d4b474b980de13593fc73d-aab3c2afb47c9381dc29b4de66a9a3c5-i
    4.     at UniqueIndex.set (C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:7246:17)
    5.     at C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:5439:15
    6.     at Array.forEach (<anonymous>)
    7.     at Collection.ensureUniqueIndex (C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:5438:17)
    8.     at Loki.loadJSONObject (C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:1728:22)
    9.     at Loki.loadJSON (C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:1626:12)
    10.     at loadDatabaseCallback (C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:2515:20)
    11.     at readFileCallback (C:\Users\Administrator\AppData\Roaming\npm\node_modules\unity-cache-server\node_modules\lokijs\src\lokijs.js:2327:15)
    12.     at FSReqWrap.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:53:3)
    So it seems to be a crash caused by duplicate keys in a dictionary.

    This is pretty frustrating, since at some point our cache server crashes, and the only way to start it back up is to destroy all of our cache data. We switch platforms frequently (including consoles) so this is slowing us down a lot.
     
  3. Reichert

    Reichert

    Unity Technologies

    Joined:
    Jun 20, 2014
    Posts:
    63
    Thanks for the report. We'll track the bug and continue the discussion in the GitHub repo.
     
  4. robert-durnin

    robert-durnin

    Joined:
    Nov 21, 2018
    Posts:
    5
    Was this issue solved and can the server be used as advertised successfully? The github notes do not mention it, and the forums do not appear to be monitored by anyone employed by Unity Technologies.
     
  5. MisterJenkins2

    MisterJenkins2

    Joined:
    Jan 10, 2020
    Posts:
    1
    Faced the same issue today.
    Had to switch back to cache_fs mode
     
  6. azxwonder

    azxwonder

    Joined:
    Jan 10, 2017
    Posts:
    1
    Same issue.
    In my case, I debugged with the source code:
    /Users/xxx/.nvm/versions/node/v12.16.3/lib/node_modules/unity-cache-server
    , and found that the initialization process broke in
    CacheBase._initDb
    :
    Code (CSharp):
    1.  
    2. // unity-cache-server/lib/cache/cache_base.js
    3. class CacheBase extends EventEmitter {
    4.     ...
    5.     async _initDb(options) {
    6.         const db = new loki(this._dbPath, options);
    7.         const loadDb = promisify(db.loadDatabase).bind(db);
    8.         // await loadDb({});
    9.         await loadDb({}, function(err){
    10.             if (err) {
    11.                 helpers.log(consts.LOG_INFO, "CacheBase._initDb err:");
    12.                 helpers.log(consts.LOG_INFO, err);
    13.             } else {
    14.                 helpers.log(consts.LOG_INFO, "CacheBase._initDb loadDatabase done");
    15.             }
    16.         });
    17.         return db;
    18.     }
    19.     ...
    20. }
    21.  
    Then a exception was thrown: "Duplicate key for property fileId: xxxxxxxxxxxxxxxxxxxxx."
    Code (JavaScript):
    1.  
    2. // node_modules/unity-cache-server/node_modules/lokijs/src/lokijs.js
    3.     UniqueIndex.prototype.set = function (obj) {
    4.       var fieldValue = obj[this.field];
    5.       if (fieldValue !== null && typeof (fieldValue) !== 'undefined') {
    6.         if (this.keyMap[fieldValue]) {
    7.           throw new Error('Duplicate key for property ' + this.field + ': ' + fieldValue);
    8.         } else {
    9.           this.keyMap[fieldValue] = obj;
    10.           this.lokiMap[obj.$loki] = fieldValue;
    11.         }
    12.       }
    13.     };
    14.  
    Check the cache.db, the content's format is json. Some fileIds were duplicated. Maybe it's a bug from Unity Cache Server.
    Finally I removed .cache_ram and re-imported all.
     
unityunity