Search Unity

Uploading to Steam via Post Build Script

Discussion in 'Unity Build Automation' started by alan-lawrance, Jan 3, 2021.

  1. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Would it be possible to run steamcmd to upload a build to Steam by using a Post Build Script?

    I'm curious if anyone has tried it. The roadblocks may be:

    1. Access to the Steam upload tools. Could be solved by adding to own repo and accessing in build script using relative path?

    2. Making sure the Steam upload tools work. I believe this means using the mac version, as all cloud builds run on macs.

    3. Get access to the uncompressed build for upload. I am not sure if we can do that, or if there would need to be a manual unzip. Is there some scratch directory we could use?

    So the idea is to have the Post Build script automatically upload the build to Steam. It could use environment variables for things like the steam branch name.
     
  2. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Getting closer with this. I'm able to access the uncompressed build in $WORKSPACE/.build/last/<config> from a post build script.

    I have been able to successfully launch steamcmd.sh, but it fails due to needing a steam code. steamcmd supports a "set_steam_guard_code" parameter, but I don't think is going to work with Unity Cloud Build spinning up a new virtual machine for each build.
     
  3. tonemcbride

    tonemcbride

    Joined:
    Sep 7, 2010
    Posts:
    1,089
    I think @victorw will know more about whether it's possible but I'm not sure it would be easy, especially if you need to run custom upload tools as I doubt Unity would allow that. The only reason you can upload iOS builds is because the tools are part of xCode and are already installed.

    If it helps I used to upload to TestFlight using an external server. You can setup cloud build to notify you when a build is complete. I used that to call a script on my server that then used the cloud build API to get a list of the latest cloud builds, download the uncompress build using the data provided and upload it to TestFlight. Maybe you could do something similar for Steam?

    p.s. Here's a guide I wrote for creating a 'build bot' for Microsoft Teams. The PHP script might be handy if you take the server approach, it allows you to download all the latest cloud build info including download URLs etc... https://forum.unity.com/threads/guide-creating-a-cloud-build-bot-in-microsoft-teams.663301/
     
    Last edited: Jan 4, 2021
  4. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    I have a system that uses a node app on EC2 to listen for build notifications, which then downloads the build, unzips it, and then uploads to steam. However I am looking for a zero cost solution, and this feels like it is close (I've already verified that steamcmd launches fine). The only known roadblock left is the steam guard code.

    You can upload directly to TestFlight using a Post Build Script in UCB, which was the inspiration for doing the same for Steam.
     
  5. jaydeebee

    jaydeebee

    Joined:
    Dec 9, 2020
    Posts:
    4
    Heh, I'm in the forums today to see if anyone's solved this. I think I've actually gotten further than most, though, so hopefully I can point a direction. Ultimately, at least for me, I don't think post-build scripts will work for Steam without some changes from Unity, though.

    My environment: Mac build of a Mac target.

    The Steam docs mention that, if you're using a machine that gets reimaged regularly, like a VM, then you just need to make sure to repopulate the <steam>/config/config.vdf and <steam>/config/ssfn* files. The latter is a "Sentry File" (its exact filename is given on a "SentryFile" line in config.vdf), and contains the "proof" that the machine has previously passed Steamguard.

    But there's more.

    The additional difficulties and my workarounds:

    • [edit] All of this is being orchestrated by a post-build.sh script that I put into my git tree.[/edit]
    • Steam isn't installed on the build servers (...is it?). I added the ContentBuilder directory to my git tree, at mytree.git/tools/steam/, so for instance, mytree.git/tools/steam/builder_osx/steamcmd.sh is the path to steamcmd.
    • $HOME on the build server is /BUILD_PATH, so Steam's directory would normally be at /BUILD_PATH/Library/Application Support/Steam. Unfortunately, something in that path isn't writable, so you can't put Steam's files there.
    • I worked around this by adding a stub Steam directory within a stub home directory to my git repo. In other words, drop your config.vdf at mytree.git/tools/steam/home/Library/Application Support/Steam/config/config.vdf, and when you call steamcmd, make sure to export HOME=`pwd`/tools/steam/home first. The `pwd` will substitute the current directory, which ought to be your build root.
    • My config.vdf lists its SentryFile as home/Library/Application Support/Steam/ssfn<somenumber> (that is, uses a relative pathname), and when I run steamcmd, I do so from the mytree.git/tools/steam directory. My goal was to not commit absolute pathnames, in case things moved.
    • Similar for my mytree.git/tools/steam/config/*.vdf files: relative pathnames. Overall, pathnames will be tricky, since there're lots of opportunities for things to be different here.
    • In my case, I'm trying not to commit the Sentry file (or team username/password) to git, since they're sensitive. My post-build script looks for environmental variables set in the build environment and relies on those.

    Doing all that got it to upload the build to steam! Hooray!

    ...except that post-build runs before notarization happens, so the app that it uploads to Steam isn't notarized. This unfortunately seems to be a deal-breaker. I think we need a post-notarization script to do the upload for mac targets.

    I haven't tried Windows targets at all. That was going to be Next for me.
     
  6. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    Thanks for the detailed steps -- I am have been trying the same approach. I haven't got it to work quite yet for Windows (getting stopped on Steamguard), which may be due to me generating the ssfn file on Windows.

    The notarization step issue for Mac could be a show-stopper though, as if it can't work for Macs then this path is a dead-end for developers who support Mac (like me).

    Could Unity please offer some insight on whether we can get the post-build script to run after notarization?
     
  7. mickanio

    mickanio

    Joined:
    Jul 13, 2015
    Posts:
    1
    I'd really like to know about any updates to this as well
     
  8. alan-lawrance

    alan-lawrance

    Joined:
    Feb 1, 2013
    Posts:
    360
    gmgannon and mgear like this.
  9. Brogan89

    Brogan89

    Joined:
    Jul 10, 2014
    Posts:
    244
    Hey, I managed to get this working without webhooks, only using UCB.
    I put Steamwork's "ContendBuilder" at the root of my projects so it gets added to my source control.

    upload_2022-4-6_22-5-3.png

    If you're running your builds locally as well add these to your ignores

    I then used this post build method


    Code (CSharp):
    1.         [PostProcessBuild]
    2.         private static void PostBuild(BuildTarget target, string pathToBuiltProject)
    3.         {
    4.             if (target != BuildTarget.StandaloneWindows64)
    5.             {
    6.                 Debug.LogError($"Target not supported: {target}");
    7.                 return;
    8.             }
    9.  
    10.             var dir = pathToBuiltProject.Replace($"{Application.productName}.exe", "");
    11.  
    12.             Debug.Log($"Root path: {Environment.CurrentDirectory}");
    13.             Debug.Log($"PathToBuiltProject: {pathToBuiltProject}");
    14.             Debug.Log($"Post build path: {dir}");
    15.  
    16.             // remove things
    17.             Directory.Delete($"{dir}\\{Application.productName}_BurstDebugInformation_DoNotShip", true);
    18.  
    19.             // copy files into steam builder content
    20.             var moveToDir = $"{Environment.CurrentDirectory}\\ContentBuilder\\content\\";
    21.             Directory.Delete(moveToDir, true);
    22.             // need copy all files and not use Directory.Move() as it will empty the build directory resulting in a failed build on UCB
    23.             CopyFilesRecursively(dir, moveToDir);
    24.             Debug.Log($"Moved: {dir} to {moveToDir}");
    25.  
    26.             // start the steam upload
    27.             var StartInfo = new ProcessStartInfo
    28.             {
    29.                 FileName = $"{Environment.CurrentDirectory}\\ContentBuilder\\builder\\steamcmd.exe",
    30.                 CreateNoWindow = true,
    31.                 RedirectStandardOutput = true,
    32.                 RedirectStandardError = true,
    33.                 UseShellExecute = false,
    34.                 Arguments = $"+login {STEAM_USERNAME} {STEAM_PASSWORD} " +
    35.                             $"+run_app_build \"{Environment.CurrentDirectory}\\ContentBuilder\\scripts\\app_build.vdf\" " +
    36.                             "+quit"
    37.             };
    38.  
    39.             using var steamBuildScript = Process.Start(StartInfo);
    40.  
    41.             if (steamBuildScript == null)
    42.             {
    43.                 Debug.LogError($"Process failed to start. {StartInfo.FileName}");
    44.                 return;
    45.             }
    46.  
    47.             Debug.Log($"Steam builder started: {steamBuildScript.StartInfo.FileName}");
    48.  
    49.             steamBuildScript.WaitForExit();
    50.             var output = steamBuildScript.StandardOutput.ReadToEnd();
    51.             var error = steamBuildScript.StandardError.ReadToEnd();
    52.  
    53.             if (!string.IsNullOrEmpty(error))
    54.                 Debug.LogError($"Steam builder error: {error}");
    55.             else
    56.                 Debug.Log($"Steam builder completed. {output}");
    57.         }
    58.      
    59.         /// <summary>
    60.         /// Src: https://stackoverflow.com/questions/58744/copy-the-entire-contents-of-a-directory-in-c-sharp
    61.         /// </summary>
    62.         /// <param name="sourcePath"></param>
    63.         /// <param name="targetPath"></param>
    64.         private static void CopyFilesRecursively(string sourcePath, string targetPath)
    65.         {
    66.             // Now Create all of the directories
    67.             foreach (string dirPath in Directory.GetDirectories(sourcePath, "*", SearchOption.AllDirectories))
    68.                 Directory.CreateDirectory(dirPath.Replace(sourcePath, targetPath));
    69.  
    70.             // Copy all the files & Replaces any files with the same name
    71.             foreach (string newPath in Directory.GetFiles(sourcePath, "*.*",SearchOption.AllDirectories))
    72.                 File.Copy(newPath, newPath.Replace(sourcePath, targetPath), true);
    73.         }
    And it uploads to steam no worries.

    I only have it set up for windows builds and I haven't tested on UCB without running on a local machine first. There are probably some files that are cached to set up the environment.

    Hope this helps and saves you a bit of AWS money ;)
     

    Attached Files:

    Last edited: Apr 6, 2022
  10. JustJunuh

    JustJunuh

    Joined:
    Jun 3, 2015
    Posts:
    52
    I like this technique, but I feel like it's a bad idea to have your Steam login info so easily accessible from your repository. If it's a small or solo project, it might not be that big of a deal. If you could conceal that info somehow, that would be useful.

    This is super cool though because it sounds like you don't need a Jenkins server or AWS :)
     
    Brogan89 likes this.
  11. Brogan89

    Brogan89

    Joined:
    Jul 10, 2014
    Posts:
    244
    You would put that info the in the UCB environment variables.
    I've moved away from this approach now anyway because UCB is too slow and expensive :( I miss the old UCB
     
    JustJunuh likes this.
  12. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    How did you set the environment variables? And how did you enter the steam guard code?
     
  13. Benjamin-Gooding

    Benjamin-Gooding

    Unity Technologies

    Joined:
    Apr 12, 2022
    Posts:
    300
    Environment variables for a build target can be set in the "Advanced Settings" tab of the build target configuration.
     
  14. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Found it. Thank you.

    However, now I'm stuck with insufficient documentation. I am attempting to convert the above script into one usable here:

    https://support.unity.com/hc/en-us/...tifact-in-Cloud-Build-in-a-post-build-script-

    Sam Jones indicates that its possible to run a bash script. Okay, great. I added the example script into my GitHub repo, in a folder that makes sense.

    upload_2023-3-1_4-4-5.png

    I've confirmed this is the correct location in my windows path. However,


    Code (CSharp):
    1.  
    2. [2023-03-01T11:24:01.513Z] - 7.3.0.2.4.2 - INFO:        Compile ran in '478.6752099' seconds
    3. [warning] [2023-03-01T11:24:21.273Z] - 7.3.0.2.4.2 - WARN: ! Script configured, but not found at quantum_unity/ContentBuilder/bash-build.sh. Skipping.
    ... and I have no way of debugging what the state of the environment is in which the shell is executing... and the iteration times here are 30 minutes long. So I can't just guess and check till it works in a reasonable time frame. Can some additional insight/documentation be added to Sam Jones post... such as two example paths for a win / mac url?

    EDIT: This happened because the script path is relative to the unity project root, not the repository root.
     
    Last edited: Mar 4, 2023
  15. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    I'm getting frustrated trying to recreate your steps here. Can you provide a sanitized version of your final bash script?

    And did you solve the notarization problem?

    EDIT: I was not able to overcome the notarization problem.
     
    Last edited: Mar 4, 2023
  16. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    upload_2023-3-1_11-49-40.png

    Would this be correct or not? Would I need to chmod this and the executables for the mac build?

    EDIT: It is not correct, unless your Assets folder is in the root of your repository.
     
    Last edited: Mar 1, 2023
  17. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Okay, this is taking forever to test. An empty unity project on cloud build takes 15 minutes to run.

    I put the bash script in the root of the github project and I changed the config to:
    upload_2023-3-1_13-49-37.png
    But the result is:
    [warning] [2023-03-01T21:39:25.396Z] - 7.3.0.2.4.2 - WARN: ! Script configured, but not found at bash-build.sh. Skipping.

    its right there. in the root of the git repo.

    EDIT: It cannot be in the root of the repo for this to work it must be in the folder above the Assets folder. This is not ideal, so you need to use relative paths to find the correct folder.
     
    Last edited: Mar 1, 2023
  18. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Okay, turns out the 'post-build-script' path is relative your unity project folder, NOT your respository.

    So when I have a repository structure like:

    /project_unity/Assets/
    /tools_steam/post-build.sh
    /some-script.sh

    You have to specify:

    ../tools_steam/post-build.sh

    ../some-script.sh

    for the shell script to be found.

    I also highly recommend people experimenting with build scripts to upload a completely empty unity project with no dependencies and put their post-build script in the pre-build script location to test your script until you know it roughly works.

    That said, the default script from Sam Jones (https://support.unity.com/hc/en-us/...tifact-in-Cloud-Build-in-a-post-build-script-) returns some funky paths:

    BUILD_PATH/p\.build\last\steamuploadertest\SteamUploaderTest.exe
     
    Last edited: Mar 1, 2023
  19. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Another six hours and uh, the results are ... odd:

    Code (Boo):
    1. "BUILD_PATH/p/quantum_unity/../tools_steam/bash-build.sh" "BUILD_PATH/p/quantum_unity/temp20230301-1367-13d1v22" "BUILD_PATH/p/.build/last/steamuploadertest" standalonewindows64
    2. START
    3. 1| BUILD_PATH/p/quantum_unity/temp20230301-1367-13d1v22
    4. 2| BUILD_PATH/p/.build/last/steamuploadertest
    5. 3| standalonewindows64
    6. Unity Exe: /cygdrive/c/Program Files/Unity/Editor/Unity.exe
    7. Output Directory: BUILD_PATH/p/.build/last/steamuploadertest
    8. Project Directory: BUILD_PATH/p/quantum_unity
    9. Workspace: BUILD_PATH/p
    10. BUILD_PATH/p\.build\last\steamuploadertest\SteamUploaderTest.exe
    11. BUILD_PATH/p/quantum_unity/../tools_steam/bash-build.sh: line 26: =cp -rf BUILD_PATH/p/.build/last/steamuploadertest BUILD_PATH/p/tools_steam/ContentBuilder/content/content_windows: No such file or directory
    12. BUILD_PATH/p/quantum_unity/../tools_steam/bash-build.sh: line 29: =BUILD_PATH/p/quantum_unity/../tools_steam/ContentBuilder/builder/steamcmd.exe +login USERNAME_ENV PASS_ENV +run_app_build BUILD_PATH/p/quantum_unity/../tools_steam/ContentBuilder/scripts/simple_app_build.vdf +quit: No such file or directory
    13.  
    Logs confirm the ContentBuilder is in the relative path:

    But the empty folder for content doesn't exist, even tho there was a .file in the repro to ensure it was created. Now modifying the bash script to create the appropriate folder... and hopefully try again.

    Edit:

    Now running the paths through cygpath to try to get it executing both the copy and the upload commands.

    Code (CSharp):
    1.  
    2. #!/bin/bash
    3.  
    4. #This is a sample that will dump the variables you need for steam builds
    5.  
    6. echo "START"
    7. echo "1| $1" # unknown
    8. echo "2| $2" # output folder
    9. echo "3| $3" # buildtype
    10.  
    11. echo "Unity Exe: $UNITY_EXE"
    12. echo "Output Directory: $OUTPUT_DIRECTORY"
    13. echo "Project Directory: $PROJECT_DIRECTORY"
    14. echo "Workspace: $WORKSPACE"
    15.  
    16. #PLAYER_PATH=$UNITY_PLAYER_PATH
    17. #IF we are using a Windows Builder and using the path to pass it to a native Windows application we need to properly convert the cygwin player path to windows format
    18. #if [[ "$BUILDER_OS" == "WINDOWS" ]]; then
    19. #PLAYER_PATH=$(cygpath -wa "$UNITY_PLAYER_PATH")
    20. #fi
    21. #echo "$PLAYER_PATH"
    22.  
    23. if [[ "$BUILDER_OS" == "WINDOWS" ]]; then
    24.  
    25. outputPath="$WORKSPACE/tools_steam/ContentBuilder/content/content_windows"
    26. outputPathCyg=$(cygpath -wa "$outputPath")
    27. mkdir -p $outputPathCyg
    28. echo "COPY Artifacts to $outputPathCyg"
    29. cmd="cp -rf $OUTPUT_DIRECTORY $outputPathCyg"
    30. echo "COPY Command: $cmd"
    31. eval $cmd
    32.  
    33. pathToConfig="$WORKSPACE/tools_steam/ContentBuilder/scripts/simple_app_build.vdf"
    34. pathToConfigCyg=$(cygpath -wa "$pathToConfig")
    35.  
    36. cmd2="$WORKSPACE/tools_steam/ContentBuilder/builder/steamcmd.exe +login $BUILD_USERNAME $BUILD_PASSWORD +run_app_build $pathToConfigCyg +quit"
    37. echo "CMD2: $cmd2"
    38. eval $cmd2
    39.  
    40. else
    41. echo "You're a mac, mate."
    42. fi
    43.  
     
    Last edited: Mar 2, 2023
  20. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Sometimes the paths returned from cygpath end up garbled in the logs:

    BUILD_PATH/p\cp -rf BUILD_PATH/p\.build\last\steamuploadertest C\workspace\workspace\p\tools_steam\ContentBuilder\content\content_windows


    Not sure if this is due to double running the cygpath command on some fragment of the command or not.
     
  21. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    More path mangling:


    CMD2: BUILD_PATH/p/tools_steam/ContentBuilder/builder/steamcmd.exe +login USERNAME PASSWORD +run_app_build BUILD_PATH/p\tools_steam\ContentBuilder\scripts\simple_app_build.vdf +quit
    Steam Console Client (c) Valve Corporation - version 1676336512
    -- type 'quit' to exit --
    Loading Steam API...OK
    Logging in user 'USERNAME' to Steam Public...OK
    Waiting for client config...OK
    Waiting for user info...OK
    [error] [2023-03-02 01:25:16]: ERROR! App build file does not exist: C:workspaceworkspaceptools_steamContentBuilderscriptssimple_app_build.vdf.

    Where on earth did the above get turned into that?

    EDIT: The cygwin paths should not be used except for shell commands that are actually windows commands, like cp.
     
    Last edited: Mar 4, 2023
  22. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Local paths also not working:

    Code (CSharp):
    1. COPY Command: cp -rf BUILD_PATH/p/.build/last/steamuploadertest BUILD_PATH/p\tools_steam\ContentBuilder\content\content_windows
    2. CMD2: BUILD_PATH/p/tools_steam/ContentBuilder/builder/steamcmd.exe +login USERNAME PASSWORD +run_app_build scripts/simple_app_build.vdf +quit
    3. Steam Console Client (c) Valve Corporation - version 1676336512
    4. -- type 'quit' to exit --
    5. Loading Steam API...OK
    6. Logging in user 'AtomechLLCBuilder' to Steam Public...OK
    7. Waiting for client config...OK
    8. Waiting for user info...OK
    9. [error] [2023-03-02 01:46:15]: ERROR! App build file does not exist: scripts/simple_app_build.vdf.
    Edit: The issue here was the windows build doesn't look for files in the working directory when you execute steamcmd - its relative to the steamcmd executable file's location.
     
    Last edited: Mar 4, 2023
  23. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Well, I'm now almost two days into a feature that should be integrated in Unity Cloud natively. There are 3 major marketplaces for PC games. Steam, Epic, GOG. How is it that you guys haven't done this feature yourselves?


    Code (CSharp):
    1. COPY Artifacts to BUILD_PATH/p\tools_steam\ContentBuilder\content\content_windows
    2. COPY Command: cp -rf BUILD_PATH/p/.build/last/steamuploadertest BUILD_PATH/p\tools_steam\ContentBuilder\content\content_windows
    3. CMD2: BUILD_PATH/p/tools_steam/ContentBuilder/builder/steamcmd.exe +login USERNAME PASSWORD +run_app_build ~/scripts/simple_app_build.vdf +quit
    4. Steam Console Client (c) Valve Corporation - version 1676336512
    5. -- type 'quit' to exit --
    6. Loading Steam API...OK
    7. Logging in user 'USERNAME' to Steam Public...OK
    8. Waiting for client config...OK
    9. Waiting for user info...OK
    10. [error] [2023-03-02 02:27:31]: ERROR! App build file does not exist: [B]BUILD_PATH/p/tools_steam/ContentBuilder/scripts/simple_app_build.vdf.[/B]
    This is using ~/ for home relative paths, but it still isn't working. It's in the child folder of the bloody executable that i'm invoking that IS running.
     
  24. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Something is very wrong with the environment.

    pwd outputs:
    BUILD_PATH/p/tools_steam/ContentBuilder

    So I know I'm in the correct working directory.

    I execute:
    CMD2: BUILD_PATH/p/tools_steam/ContentBuilder/builder/steamcmd.exe +login <USERNAME> <PASSWORD> +run_app_build ./simple_app_build.vdf +quit

    The output is:
    Code (CSharp):
    1.  
    2. Steam Console Client (c) Valve Corporation - version 1676336512
    3. -- type 'quit' to exit --
    4. Loading Steam API...OK
    5. Logging in user 'AtomechLLCBuilder' to Steam Public...OK
    6. Waiting for client config...OK
    7. Waiting for user info...OK
    8. [error] [2023-03-02 03:45:01]: ERROR! App build file does not exist: ./simple_app_build.vdf.
    I am attemping to do dir | echo and ls | cat, but nothing appears in the log files, so I can't get any information about the file system or the layout of the file am looking for.

    Edit:

    ls -l 1>&2

    Finally started showing some information.
     
    Last edited: Mar 2, 2023
  25. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    upload_2023-3-1_20-7-37.png
    Okay, I don't know where to go from here. the above shows that the file is in the working directory.

    The below says it doesn't exist. :shrug:
    upload_2023-3-1_20-8-0.png

    EDIT:
    So when steamcmd executes in cygwin, the path passed to it is relative to the binary... not the working directory.
     
    Last edited: Mar 4, 2023
  26. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    Okay, I finally got this uploading... but for Mac only.

    Like mentioned above, the steps are you need to locally install all of the steam files on a mac.

    You need to set your home folder to a local folder inside of your repo.
    You need to chmod+x steamcmd.sh in the ContentBuilder/builder_osx/ path
    You need to run it once on a mac to download everything and patch once.
    You need to commit everything to your repo.

    You need to use a relative path for the VDF file. Here's the script that finally worked for me for mac: (still working on windows, something is F***ed in the environment there).

    Code (CSharp):
    1.  
    2. #!/bin/bash
    3.  
    4. echo "START"
    5.  
    6. #Use this to confirm your paths are what you expect in the log files.
    7. abspath(){
    8.   local thePath
    9.   if [[ ! "$1" =~ ^/ ]];then
    10.     thePath="$PWD/$1"
    11.   else
    12.     thePath="$1"
    13.   fi
    14.   echo "$thePath"|(
    15.   IFS=/
    16.   read -a parr
    17.   declare -a outp
    18.   for i in "${parr[@]}";do
    19.     case "$i" in
    20.     ''|.) continue ;;
    21.     ..)
    22.       len=${#outp[@]}
    23.       if ((len==0));then
    24.         continue
    25.       else
    26.         unset outp[$((len-1))]
    27.       fi
    28.       ;;
    29.     *)
    30.       len=${#outp[@]}
    31.       outp[$len]="$i"
    32.       ;;
    33.     esac
    34.   done
    35.   echo /"${outp[*]}"
    36. )
    37. }
    38.  
    39. if [[ "$BUILDER_OS" == "WINDOWS" ]]; then
    40.  
    41.   echo "Not solved."
    42. else
    43.  
    44.  outputPath="$WORKSPACE/tools_steam/ContentBuilder/content/content_mac"
    45.     mkdir -p $outputPath
    46.  
    47.     echo "mkdir -p $outputPath"
    48.     mkdir -p $outputPath
    49.     echo "COPY Artifacts to $outputPath"
    50.  
    51.     echo rm -rf $OUTPUT_DIRECTORY/YourProject_BackUpThisFolder_ButDontShipItWithYourGame
    52.     rm -rf $OUTPUT_DIRECTORY/YourProject_BackUpThisFolder_ButDontShipItWithYourGame
    53.     echo rm -rf $OUTPUT_DIRECTORY/YourProject_BurstDebugInformation_DoNotShip
    54.     rm -rf $OUTPUT_DIRECTORY/YourProject_BurstDebugInformation_DoNotShip
    55.  
    56.     cmd="cp -rf $OUTPUT_DIRECTORY/. $outputPath"
    57.     echo "COPY Command: $cmd"
    58.     eval $cmd
    59.  
    60.     #export HOME=`pwd`/tools_steam/ContentBuilder
    61.     export HOME=`pwd`/tools_steam/home
    62.     pathToConfig="../scripts/simple_app_build_mac.vdf"
    63.     cd $WORKSPACE/tools_steam/ContentBuilder/builder_osx
    64.     pwd
    65.     abspath $pathToConfig
    66.  
    67.     executePath="$WORKSPACE/tools_steam/ContentBuilder/builder_osx/steamcmd.sh"
    68.     echo chmod u+x $executePath
    69.     chmod u+x $executePath
    70.     executePath2="$WORKSPACE/tools_steam/ContentBuilder/builder_osx/steamcmd"
    71.     echo chmod u+x $executePath2
    72.     chmod u+x $executePath2
    73.  
    74.     cmd2="$executePath +login $BUILD_USERNAME $BUILD_PASSWORD +run_app_build $pathToConfig +quit"
    75.     echo "CMD2: $cmd2"
    76.     eval $cmd2
    77. fi
    78.  
     
    Last edited: Mar 3, 2023
  27. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    ... and now I have it working for Windows as well.

    Here's the final script. If I find the energy, I will create the full set of steps. The two keys are Jaydeebee's post to understand WHAT to do. Here's an example of doing it:

    Code (CSharp):
    1. #!/bin/bash
    2.  
    3. echo "START"
    4. echo "1| $1" # unknown
    5. echo "2| $2" # output folder
    6. echo "3| $3" # buildtype
    7.  
    8. echo "Unity Exe: $UNITY_EXE"
    9. echo "Output Directory: $OUTPUT_DIRECTORY"
    10. echo "Project Directory: $PROJECT_DIRECTORY"
    11. echo "Workspace: $WORKSPACE"
    12.  
    13. # Useful if you need to debug
    14. abspath(){
    15.   local thePath
    16.   if [[ ! "$1" =~ ^/ ]];then
    17.     thePath="$PWD/$1"
    18.   else
    19.     thePath="$1"
    20.   fi
    21.   echo "$thePath"|(
    22.   IFS=/
    23.   read -a parr
    24.   declare -a outp
    25.   for i in "${parr[@]}";do
    26.     case "$i" in
    27.     ''|.) continue ;;
    28.     ..)
    29.       len=${#outp[@]}
    30.       if ((len==0));then
    31.         continue
    32.       else
    33.         unset outp[$((len-1))]
    34.       fi
    35.       ;;
    36.     *)
    37.       len=${#outp[@]}
    38.       outp[$len]="$i"
    39.       ;;
    40.     esac
    41.   done
    42.   echo /"${outp[*]}"
    43. )
    44. }
    45.  
    46.  
    47. if [[ "$BUILDER_OS" == "WINDOWS" ]]; then
    48.     outputPath="$WORKSPACE/tools/steam/ContentBuilder/content/content_windows"
    49.     outputPathCyg=$(cygpath -wa "$outputPath")
    50.  
    51.     # this is the only one that needs the cygwin path
    52.     mkdir -p $outputPathCyg
    53.     echo "COPY Artifacts from $OUTPUT_DIRECTORY to $outputPath"
    54.     cmd="cp -rf $OUTPUT_DIRECTORY/. $outputPath"
    55.     echo "COPY Command: $cmd"
    56.     eval $cmd
    57.  
    58.     # make sure you know where your content builder is
    59.     export HOME=`pwd`/tools/steam/ContentBuilder
    60.  
    61.     #needs to be a relative path to the builder or builder_osx folder.
    62.     pathToConfig="../simple_app_build.vdf"
    63.     echo $pathToConfig
    64.  
    65.     cd ~/
    66.  
    67.     #useful confirmation that you are in the right folder.
    68.     pwd
    69.     ls -l 1>&2
    70.  
    71.     # use the environment variables to set username and password or replace here directly.
    72.     cmd2="$WORKSPACE/tools/steam/ContentBuilder/builder/steamcmd.exe +login $BUILD_USERNAME $BUILD_PASSWORD +run_app_build $pathToConfig +quit"
    73.     echo "CMD2: $cmd2"
    74.     eval $cmd2
    75. else
    76.  
    77.     outputPath="$WORKSPACE/tools/steam/ContentBuilder/content/content_mac"
    78.     mkdir -p $outputPath
    79.  
    80.     # mac does NOT want a cygwin path obviously
    81.     echo "mkdir -p $outputPath"
    82.     mkdir -p $outputPath
    83.     echo "COPY Artifacts to $outputPath"
    84.  
    85.     # replace SpiderSkirmish with your _builder configuration_ name, NOT your project name
    86.     echo rm -rf $OUTPUT_DIRECTORY/SpiderSkirmishMac_BackUpThisFolder_ButDontShipItWithYourGame
    87.     rm -rf $OUTPUT_DIRECTORY/SpiderSkirmishMac_BackUpThisFolder_ButDontShipItWithYourGame
    88.     echo rm -rf $OUTPUT_DIRECTORY/SpiderSkirmishMac_BurstDebugInformation_DoNotShip
    89.     rm -rf $OUTPUT_DIRECTORY/SpiderSkirmishMac_BurstDebugInformation_DoNotShip
    90.  
    91.     cmd="cp -rf $OUTPUT_DIRECTORY/. $outputPath"
    92.     echo "COPY Command: $cmd"
    93.     eval $cmd
    94.  
    95.     # this is where you will pre-install your steam files. You need to run it once on a mac and set this path to home before executing steamcmd.sh
    96.     export HOME=`pwd`/tools/steam/home
    97.     pathToConfig="../scripts/simple_app_build_mac.vdf"
    98.     cd $WORKSPACE/tools/steam/ContentBuilder/builder_osx
    99.     pwd
    100.  
    101.     # useful for confirming you have the right folder
    102.     ls -l 1>&2
    103.  
    104.     # both needs to be given execution permission.
    105.     binPath="$WORKSPACE/tools/steam/ContentBuilder/builder_osx/steamcmd"
    106.     executePath="$WORKSPACE/tools/steam/ContentBuilder/builder_osx/steamcmd.sh"
    107.     echo chmod u+x $binPath
    108.     chmod u+x $binPath
    109.     echo chmod u+x $executePath
    110.     chmod u+x $executePath
    111.  
    112.     #finally execute.
    113.     cmd2="$executePath +login $BUILD_USERNAME $BUILD_PASSWORD +run_app_build $pathToConfig +quit"
    114.     echo "CMD2: $cmd2"
    115.     eval $cmd2
    116. fi
    Major notes - run both of your bash scripts in a windows cygwin terminal locally, and a mac terminal locally first. You'll save yourself major headaches.

    Your
    cp -rf
    command needs to have a /. after the folder path or it will copy the folder name into the destination.
    Your bash file as set in the configuration for the build target needs to be relative to your unity directory, not your repository.
    You need to set
    chmod +x
    on both the bash scripts and the steam executables.
    Log messages will NOT appear in the log by default. You can direct output to stderr if you want them to show up.

    Start with the mac auto-build first, its cleaner with fewer error prone steps.
     
    Last edited: Mar 6, 2023
  28. LFG-Sky

    LFG-Sky

    Joined:
    May 10, 2022
    Posts:
    5
    I've been looking for a solution like this. May I ask, since you are uploading both OS builds separately. How exactly do are you able to set two different Build IDs to the default Steam branch? Currently, my manual solution is to upload both macOS & Windows build together but I don't see a different way if I want both to be active to Steam users at the same time if only one BuildID can be set active.
     
  29. LFG-Sky

    LFG-Sky

    Joined:
    May 10, 2022
    Posts:
    5
    Nevermind, I've figured out that the branches are merged depending on which depot was updated.

    Does anyone have a solution for the Steam Guard Code? I'm unable to disable the Steam Guard even though there is an option inside account settings to do so.
     
    Xelnath likes this.
  30. Xelnath

    Xelnath

    Joined:
    Jan 31, 2015
    Posts:
    402
    When you run the process locally on your own machine, it will generate a trusted key. If you upload this to steam's cloud build, it will treat it as if you had done so from the unity machine.,
     
  31. EnduvoJD

    EnduvoJD

    Joined:
    Aug 12, 2016
    Posts:
    51
    Where is Unity on this thread?

    We now have the "Unity DevOps" service, we've been moved to paying for consumption-based usage, but Unity Build Automation is severely under-documented and under-supported compared to other areas of Unity.
     
    Last edited: Sep 2, 2023