Upload symbols to Cloud Diagnostics when building from command line

On our CI server we build iOS build by running BuildPipeline.BuildPlayer() in Editor batch mode. Turns out that Xcode project that is exported that way has empty USYM_UPLOAD_AUTH_TOKEN and because of that process_symbols.sh step fails to upload symbols to Unity Cloud Diagnostics. Looks like editor do not log-in into unity cloud services when running in batch mode. I've checked that by calling UnityEditor.Connect.UnityConnect.instance.GetAccessToken() in post build step and it returned empty string. Is there any way to force Unity Editor to log-in to cloud services when running in batch mode? Or maybe I can obtain upload auth token by some REST api calls in by post build steps?

 

Are there any plans to make it work in the future?

 

Any updates on this?

 

I really hope this will be backported to upcoming 2018.4 LTS

 

Please inform to this thread which version applied.

 

Sorry for the delay on this. This fix will be backported up to 2018.4. You should expect it in the latest patch, 2018.4.1, as well as in 2019.1.3f1.

 

In Unity 2019.1.3 patch notes there is a line:
Services: Fixed symbol upload for Cloud Diagnostics not working in batch mode. (851655, 1140370)
Can't find the same for 2018.4.1

 

After upgrading from 2018.4.0f1 to 2018.4.1f1, I can't build ios projects with jenkins, it gives "Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable" error. With 2018.4.0f1, everything is fine.

 

I found that to be the case when upgrading from 2018.4.0f1 to 2018.4.1f1 as well. Adding the "-username" and "-password" options to the Unity command line arguments seemed to resolve it, however.

 

I have the same issue with USYM_UPLOAD_AUTH_TOKEN in Unity 2019.1.3f1. I tried to follow fhickman's advice and now i am getting this error:

build 14-Jun-2019 17:00:23 [UnityConnectServicesConfig] config is NOT valid, switching to default
error 14-Jun-2019 17:00:23 [warn] kq_init: detected broken kqueue; not using.: Undefined error: 0
error 14-Jun-2019 17:00:23 [warn] kq_init: detected broken kqueue; not using.: Undefined error: 0
error 14-Jun-2019 17:00:23 [warn] kq_init: detected broken kqueue; not using.: Undefined error: 0

Anybody encountered that ?

 

I have the same problem in 2018.4.3f1

First missing file usymtool.

Code (CSharp):

1. Users/mmalek/tests/Repro/YlandsClassID91/Builds/iOS/DEV_mobile/Dist/Ylands/process_symbols.sh: line 8: /Users/mmalek/tests/Repro/YlandsClassID91/Builds/iOS/DEV_mobile/Dist/Ylands/usymtool: No such file or directory

After copy missing script from Unity app (Unity.app/Contents/Tools/macosx/) and rerun xCode build I got error.

Code (CSharp):

1. Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable

 

Also just got bitten by this USYM_UPLOAD_AUTH_TOKEN bug in 2018.4.2.

@ryanc-unity any idea when the fix will be backported to the LTS branch? Is there any proper workaround while we're waiting? Thanks!

 

Hi @DarekRusin, I'm sorry for the late response. When using Cloud Diagnostics with -batchmode it's now required that you specify both the -username and -password arguments to avoid the USYM_UPLOAD_AUTH_TOKEN error message.

We are currently looking into a known issue where using the -runTests argument throws this same error on iOS even after including both -username and -password. You can track our progress on that fix here.

 

@ryanc-unity thanks so much! I did not now that specifying username and password is now required. Once I've added these, everything started working properly

 

In teamcity we fix it, by patching process_symbols.sh in xcode project.

Just change strings with usymtool on something like there:

#!/bin/sh

# process_symbols
#
# Copyright (c) 2015 Unity Technologies. All rights reserved.

if [ "${SYNCHRONOUS_SYMBOL_PROCESSING}" = "TRUE" ]; then

echo test

else
nohup "$PROJECT_DIR/usymtool" -symbolPath "$DWARF_DSYM_FOLDER_PATH/$DWARF_DSYM_FILE_NAME" > /dev/null 2>&1 &
disown
fi

 

@ryanc-unity Can we get info on what usymtool is, and what's the downside to not using it?
If it's just for desymbolicating via Unity Could then it sounds like it should be an opt-in feature because most teams already doing desymbolication are already using something else.
The solution of not using it also came up here https://forum.unity.com/threads/usymtool-is-eating-up-all-my-ram.639025/

 

Hi @psydent, sure thing. usymtool is used to process symbol files from the build and upload them to the Unity Cloud Diagnostics servers. It is required if Cloud Diagnostics is enabled. If you are having an issue with it, please submit a formal bug report with reproduction steps.

 

I'm not sure if'll help anyone else, but when I ran the build command from command line without using batchmode I had the same error appearing in my logs even though I was logged into my account via Unity services.

Code (CSharp):

1. Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable

Switching my command line to use batchmode and adding the username and password resolved the issues however (but adding just the username and password values without adding the batchmode flag still threw the error)

As an aside, is there any way for us to generate the auth token manually so we don't have to store log in details on our build server in plain text?

 

Hi @tonic-joel, the batchmode command line argument is required in order to be able to run Unity silently from the command line. You need to include batchmode, along with the username and password arguments, in order to build while using Cloud Services like Diagnostics, IAP, and Analytics.

There currently isn't a publicly available API for generating auth tokens. I understand being reluctant to use plain text log in details so I will pass this along to the team to see what alternatives there might be.

 

Im facing the issue in the version 2019.3.0f6.I've added username and password in the batch command .Im using jenkins to make ios builds.
This is my batch command

Code (CSharp):

1. -quit -batchmode -serial xxxxxxxxxxxxxx -username 'yyyyy' -password 'zzzzzz' -executeMethod JenkinsBuildHelper.PerformAndroidIOSBuild

.
This is error message

Code (CSharp):

1. bin/sh -c /Users/jenkins/Library/Developer/Xcode/DerivedData/myproj-gecywtqqtdtvjabrrtiojeiyhwwrh/Build/Intermediates.noindex/ArchiveIntermediates/myproj/IntermediateBuildFilesPath/Unity-iPhone.build/Release-iphoneos/UnityFramework.build/Script-52774D479664D9325892DECA.sh
2. time="2020-07-03T13:50:48+05:30" level=warning msg="Failed to create log file at /Users/jenkins/Library/Logs/Unity/symbol_upload.log: open /Users/jenkins/Library/Logs/Unity/symbol_upload.log: no such file or directory"
3. time="2020-07-03T13:50:48+05:30" level=fatal msg="Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable"
4. Command PhaseScriptExecution failed with a nonzero exit code

 

Its working after adding CD access to my jenkins machine user.Thank you

 

I have to ask: does the user have to have a seat license? I'd rather not provide my actual Unity ID user/pass into our CI agent, but if I can get away with just putting in a dummy CI user just for symbol upload that'd be great. Our agents already have seats by way of running on our Windows workstations already.

If not, this is quite a frustrating oversight, and is another one of those instances where Unity's licensing policy hurts paying users more than anyone else.

 

Hi all,
Asper this doc we can upload libmain,libunity,libil2cpp symbols to unity cloud diagnostics. We use unity as a libraray in android. Our Build automation system(CI) was able to upload libil2cpp symbols(which gets generated from android studio gradle) automatically to unity CD. However, we could see that our libmain,libunity symbols are missing and which caused unreadle memory addresses in the crash stack trace.
By using this EditorUserBuildSettings.androidCreateSymbols we can get the symbols of libmain,libunity, can someone confirm/help me if we can upload these symbols programatically (via CI jenkins) without manual upload to CD dashboard?? or by using above API unity uploads all(libmain,libunity and libil2cpp) symbols automatically to CD??
Please point me in right direction.
Thanks in advance