Search Unity

Question Upload saves from old database

Discussion in 'Cloud Save' started by 8bitgoose, Sep 10, 2022.

  1. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    448
    Hi,

    A cloud save system I am already using is being shut down. I'd like to export the data from that service and upload it to Unity Cloud save. Is this possible? The key would be the person's email.
     
  2. Laurie-Unity

    Laurie-Unity

    Unity Technologies

    Joined:
    Mar 5, 2020
    Posts:
    220
    Hi there,

    Thanks for sharing your question.

    The simple answer is YES, you can use Cloud Save to store the data exported from another cloud storage solution, indeed this is someting we have been able to help some GameSparks customers with, in advance of their shutdown at the end of September.

    However, there are a few things you will need to take into consideration in your implementation.

    • When a player accesses Cloud Save their data will ba stored against the PlayerID they were assigned by the UGS Authentication package when they signed in. So if you updload their exported data to UGS Cloud Save and save it by their email address, it will be stored under a different ID that cannot be accessed directly by the player from the Cloud Save SDK. But, fear not, there is a workaround.

      Whilst the player can't reach the exported and uploaded data directly, it is possible to access it from Cloud Code using a service token to access it, then save it back to Cloud Save under the player's ID.
    1. Export the data from the 3rd party storage solution. Preferrably put it in a database somewhere so you can track status and rerun the following steps if you have any problems or subsequent data updates.

    2. Create a tool to iterate over the players in the export and use the Cloud Code and Cloud Save API to upload them to Cloud Save , keyed by some string based ID that is common to the player before and after the Cloud storage migration.

    3. At runtime, have the player run a Cloud Code script that pulls their data from step 2 out of Cloud Save and re-saves it back to their own Cloud Save profile. This script should only be run once, the first time the player reconnects after you have exported their data from the other storage solution.
    • Be aware that there are storage limits on Cloud Save, you can find out more here. You are limited to 200 data 'slots' per player and each slot has a size limit of 16KB of JSON serializable data.

    • You may hit rate limits or experience occasional timeouts if you try and run an uploader (step 2.) too aggressively. So I would recommend using an intermediate Database to track the state of each player data upload, so you can retry and update etc..

    • Related to the last point, when uploading exported data (step 2.) It will be more efficient to send a batch of player data to Cloud Code as a single request, then have the Cloud Code iterate over the individual players hititng Cloud Save for each one. But be aware there is an execution time limit on each Cloud Code script. I found the optimal batch size to be ~25 players.

    • After you have uploaded a player's data to Cloud Save (under their email or another arbritray ID), you won't be able to see it in the Find A Player UI tool, because this wasn't really an authenticated player, but you could write a little cloud code script to check it. Or you could run the Migration script below from within the Cloud Code dashboard editor to see if it can find and load your saved player data.

    • Finally, these additional Cloud Save entries that you are upploading will count towards your metered billing for Cloud Save and Cloud Code for the month, but there is a generous Free Tier. You can find pricing information here
    This is the Cloud Code script I have used to upload player data to Cloud Code. It should be run from a tool that iterates over each exported player and uploads them in batches.

    GsSavePlayerDataBulk

    Code (JavaScript):
    1. /*
    2. *  -------- Save GameSparks User to Cloud Save--------------------
    3. *  Accepts an Array containing multiple players
    4. * Use Service Token to save GameSparks player's data to Cloud Save
    5. * ----------------------------------------------------------------
    6. */
    7.  
    8. const _ = require("lodash-4.17");
    9. const { DataApi } = require("@unity-services/cloud-save-1.2");
    10.  
    11. module.exports = async ({ params, context, logger }) => {
    12.  
    13.   const {
    14.     projectId,
    15.     playerId,
    16.     serviceToken
    17.   } = context;
    18.  
    19.   logger.info("Script parameters: " + JSON.stringify(params));
    20.  
    21.   const players = params.GsPlayerDataArray;
    22.   logger.info ("Players Found = " + players.length)
    23.  
    24.   // Service Token used to Authenticate so we can hit another player's data
    25.   const api = new DataApi({ accessToken: serviceToken});
    26.   logger.info("Authenticated within the following context: " + JSON.stringify(context));
    27.  
    28.   for(const player of players) {
    29.     await Save(player);
    30.   }
    31.  
    32.   // Return the JSON result to the client
    33.   return {
    34.  
    35.   };
    36.  
    37.   async function Save(player)
    38.   {
    39.     logger.info("Saving " + player.GameSparksUserID);
    40.     logger.info("ProjectId " + projectId);
    41.      
    42.     // Attempt to Save GameSparks player data from Cloud Save
    43.     const GSSave = await api.setItemBatch(projectId, player.GameSparksUserID, {"data" :
    44.       [
    45.         { key:"GameSparksUserID", value:player.GameSparksUserID },
    46.         { key:"ttl", value:player.ttl },
    47.         { key:"version", value:player.version },
    48.         { key:"data", value:player.data }
    49.       ]}
    50.     );
    51.  
    52.     const GSSaveResponse = JSON.stringify(GSSave.data);
    53.     logger.info('GSSave result ' + GSSaveResponse);
    54.    
    55.   }
    56. };

    This is a Cloud Code script that I have used to migrate data that was loaded into Cloud Save under a different key than the PlayerID. It should be run by the player at runtime

    Code (CSharp):
    1. /*
    2. *  -------- Migrate GameSparks User from Cloud Save------------------
    3. *  Use Service Token to load another player's data from Cloud Save
    4. *  then save it back to Cloud Save as for Authenticated Player
    5. * -------------------------------------------------------------------
    6. */
    7.  
    8. const _ = require("lodash-4.17");
    9. const { DataApi } = require("@unity-services/cloud-save-1.2");
    10.  
    11. module.exports = async ({ params, context, logger }) => {
    12.  
    13.   const {
    14.     projectId,
    15.     playerId,
    16.     serviceToken
    17.   } = context;
    18.  
    19.   // GameSparksUserID input parmameter present?
    20.   logger.info("Script parameters: " + JSON.stringify(params));
    21.  
    22.   // Service Token used to Authenticate so we can hit another player's data
    23.   const api = new DataApi({ accessToken: serviceToken});
    24.   logger.info("Authenticated within the following context: " + JSON.stringify(context));
    25.  
    26.   logger.info('Search Key' + params.playerSearchKey);
    27.  
    28.   // Attempt to get other player's data from Cloud Save
    29.   const GSLoad = await api.getItems(projectId, params.playerSearchKey, ["GameSparksUserID", "ttl", "version", "data"]);
    30.   const GSLoadResponse = JSON.stringify(GSLoad.data)
    31.   logger.info('GameSparks PlayerData Load result ' + GSLoadResponse);
    32.  
    33.   // Access Token used to Authenticate so we can hit authenticated player's data
    34.   const api2 = new DataApi(context);
    35.  
    36.   // Attempt to Save player data to Cloud Save
    37.   const UGSSave = await api2.setItemBatch(projectId, playerId, {"data" :
    38.     [
    39.       { key: GSLoad.data.results[0].key, value:GSLoad.data.results[0].value},
    40.       { key: GSLoad.data.results[1].key, value:GSLoad.data.results[1].value},
    41.       { key: GSLoad.data.results[2].key, value:GSLoad.data.results[2].value},
    42.       { key: GSLoad.data.results[3].key, value:GSLoad.data.results[3].value}
    43.     ]}
    44.   );
    45.   const UGSSaveResponse = JSON.stringify(UGSSave.data)
    46.   logger.info('UGS PlayerData Save result ' + UGSSaveResponse);
    47.    
    48.  
    49.   // Return the JSON result to the client
    50.   return {
    51.     GameSparksUserID: params.GameSparksUserID,
    52.     GameSparksLoadResponse: GSLoadResponse,
    53.     UGSSaveResponse: UGSSaveResponse
    54.   };
    55. };

    I hope that gives you a few pointers. Let me know if you need any more details.
     
  3. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    448
    Thank you for the detailed answer! I really appreciate it. I am going to look into this and I'll let you know if I run into some issues.
     
  4. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    448
    Hi @Laurie-Unity,

    I think I've got most of the script set up, however I am running into one issue. I am calling
    api.setItemBatch(
    and getting the error response from a Unity program that is attempting to upload the data

    Code (JavaScript):
    1. ScriptError
    2. (422) HTTP/1.1 422 Unprocessable Entity
    3. Unprocessable Entity
    4. Invocation Error
    5. RequiredError: Required parameter playerId was null or undefined when calling setItemBatch.
    6.  
    Does
    api.setItemBatch
    require a playerId now? Or am I missing something?

    https://cloud-code-sdk-documentation.cloud.unity3d.com/cloud-save/v1.0

    It looks like the playerID is required. Is this going to cause issues uploading GameSparks since we don't want a player ID associated with these entries especially if we are going to send up hundreds of separate players. Or do I need to generate a bunch of anonymous player IDs?
     
  5. Laurie-Unity

    Laurie-Unity

    Unity Technologies

    Joined:
    Mar 5, 2020
    Posts:
    220
    You linked to the documentation for Cloud Save v1.0

    My example works and is using Cloud Save v1.2
    Code (CSharp):
    1. const { DataApi } = require("@unity-services/cloud-save-1.2");
    and calls setItemBatch with the playerID substituted with the gameSparks UserID
    Code (CSharp):
    1.     // Attempt to Save GameSparks player data from Cloud Save
    2.     const GSSave = await api.setItemBatch(projectId, player.GameSparksUserID, {"data" :
    3.       [
    4.         { key:"GameSparksUserID", value:player.GameSparksUserID },
    5.         { key:"ttl", value:player.ttl },
    6.         { key:"version", value:player.version },
    7.         { key:"data", value:player.data }
    8.       ]}
    9.     );
    My second script, initiated by the player, reloads the data based on the gameSparks UserID and saves it back under the player's own playerID. e.g. I create an additional, intermediate Cloud Save record based on the player's gameSparks ID
     
  6. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    448
    Thanks for the quick response.

    I have copy pasted your code into the Cloud Code and still getting an issue. Here is the code on my Unity side. Maybe there is something I am missing.

    Code (CSharp):
    1.     public async void Awake()
    2.     {
    3.         await UnityServices.InitializeAsync();
    4.         await AuthenticationService.Instance.SignInAnonymouslyAsync();
    5.  
    6.         var arguments = new Dictionary<string, object>
    7.         {
    8.             { "GsPlayerDataArray", jsonData },
    9.         };
    10.         CloudCodeResponse response = await CloudCodeService.Instance.CallEndpointAsync<CloudCodeResponse>("UploadSaveDataFromGS", arguments);
    11.  
    12.         Debug.Log(response);
    13.     }
    And here is the JSON data I am sending up

    "[ { "GameSparksUserID": "newplayer@website.com", "ttl": "5555", "version": "1.3", "data": "this is data" }]"


    And still getting that same error response that the PlayerID is not provided. Do I need to send something else up from Unity's side?

    I am not too familiar with Javascript so maybe I am creating the data to be sent up wrong.
     
  7. Laurie-Unity

    Laurie-Unity

    Unity Technologies

    Joined:
    Mar 5, 2020
    Posts:
    220
    Can you DM me a link to your Cloud Code Script and I'll take a look
     
  8. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    448
    Okay great, I DM'd you.
     
  9. 8bitgoose

    8bitgoose

    Joined:
    Dec 28, 2014
    Posts:
    448
    After chatting with the very helpful @Laurie-Unity about this, turns out you need to call on JArray.Parse before submitting to the Cloud Code.

    Should be
    Code (CSharp):
    1.         var arguments = new Dictionary<string, object>
    2.         {
    3.             { "GsPlayerDataArray", JArray.Parse(jsonData) },
    4.         };
    The C# wrapper won't take a string parsed into JSON, has to be parsed into NewtonSoft's JArray to work correctly.
     
    GilbertoBitt and Laurie-Unity like this.