Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Upload symbols to Cloud Diagnostics when building from command line

Discussion in 'Unity Cloud Diagnostics' started by stopiccot_tds, Dec 29, 2018.

  1. stopiccot_tds

    stopiccot_tds

    Joined:
    Oct 1, 2016
    Posts:
    111
    On our CI server we build iOS build by running BuildPipeline.BuildPlayer() in Editor batch mode. Turns out that Xcode project that is exported that way has empty USYM_UPLOAD_AUTH_TOKEN and because of that process_symbols.sh step fails to upload symbols to Unity Cloud Diagnostics. Looks like editor do not log-in into unity cloud services when running in batch mode. I've checked that by calling UnityEditor.Connect.UnityConnect.instance.GetAccessToken() in post build step and it returned empty string. Is there any way to force Unity Editor to log-in to cloud services when running in batch mode? Or maybe I can obtain upload auth token by some REST api calls in by post build steps?
     
  2. johng_unity

    johng_unity

    Unity Technologies

    Joined:
    Nov 5, 2015
    Posts:
    158
    Unfortunately, symbol upload does not work in batch mode.
     
  3. benzo-turbo

    benzo-turbo

    Joined:
    Jan 28, 2015
    Posts:
    1
    Are there any plans to make it work in the future?
     
  4. johng_unity

    johng_unity

    Unity Technologies

    Joined:
    Nov 5, 2015
    Posts:
    158
    We are currently working on a change to support batch mode for symbol upload. We'll be able to provide more details on what versions of Unity the change will be available in a couple of weeks.
     
  5. maxthecat

    maxthecat

    Joined:
    May 26, 2014
    Posts:
    29
    Any updates on this?
     
  6. johng_unity

    johng_unity

    Unity Technologies

    Joined:
    Nov 5, 2015
    Posts:
    158
    We have made a change to support this in our mainline branch and we are now working on identifying which versions of Unity the change can be brought to.
     
  7. stopiccot_tds

    stopiccot_tds

    Joined:
    Oct 1, 2016
    Posts:
    111
    I really hope this will be backported to upcoming 2018.4 LTS
     
  8. rumaniel

    rumaniel

    Joined:
    Jun 3, 2015
    Posts:
    5
    Please inform to this thread which version applied. o_O
     
  9. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Sorry for the delay on this. This fix will be backported up to 2018.4. You should expect it in the latest patch, 2018.4.1, as well as in 2019.1.3f1.
     
    rumaniel likes this.
  10. stopiccot_tds

    stopiccot_tds

    Joined:
    Oct 1, 2016
    Posts:
    111
    In Unity 2019.1.3 patch notes there is a line:
    Services: Fixed symbol upload for Cloud Diagnostics not working in batch mode. (851655, 1140370)
    Can't find the same for 2018.4.1
     
  11. ilterbilguven

    ilterbilguven

    Joined:
    Feb 1, 2018
    Posts:
    6
    After upgrading from 2018.4.0f1 to 2018.4.1f1, I can't build ios projects with jenkins, it gives "Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable" error. With 2018.4.0f1, everything is fine.
     
  12. fhickman

    fhickman

    Joined:
    Feb 10, 2016
    Posts:
    9
    I found that to be the case when upgrading from 2018.4.0f1 to 2018.4.1f1 as well. Adding the "-username" and "-password" options to the Unity command line arguments seemed to resolve it, however.
     
    Ryan-Unity likes this.
  13. Ziggy90

    Ziggy90

    Joined:
    Mar 15, 2017
    Posts:
    6
    I have the same issue with USYM_UPLOAD_AUTH_TOKEN in Unity 2019.1.3f1. I tried to follow fhickman's advice and now i am getting this error:

    build 14-Jun-2019 17:00:23 [UnityConnectServicesConfig] config is NOT valid, switching to default
    error 14-Jun-2019 17:00:23 [warn] kq_init: detected broken kqueue; not using.: Undefined error: 0
    error 14-Jun-2019 17:00:23 [warn] kq_init: detected broken kqueue; not using.: Undefined error: 0
    error 14-Jun-2019 17:00:23 [warn] kq_init: detected broken kqueue; not using.: Undefined error: 0

    Anybody encountered that ?
     
  14. MildaGenius2

    MildaGenius2

    Joined:
    May 6, 2013
    Posts:
    19
    I have the same problem in 2018.4.3f1 :(

    First missing file usymtool.

    Code (CSharp):
    1. Users/mmalek/tests/Repro/YlandsClassID91/Builds/iOS/DEV_mobile/Dist/Ylands/process_symbols.sh: line 8: /Users/mmalek/tests/Repro/YlandsClassID91/Builds/iOS/DEV_mobile/Dist/Ylands/usymtool: No such file or directory
    After copy missing script from Unity app (Unity.app/Contents/Tools/macosx/) and rerun xCode build I got error.

    Code (CSharp):
    1. Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable
     
    Last edited: Jul 8, 2019
  15. DarekRusin

    DarekRusin

    Joined:
    Nov 15, 2013
    Posts:
    47
    Also just got bitten by this USYM_UPLOAD_AUTH_TOKEN bug in 2018.4.2.

    @ryanc-unity any idea when the fix will be backported to the LTS branch? Is there any proper workaround while we're waiting? Thanks!
     
  16. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi @DarekRusin, I'm sorry for the late response. When using Cloud Diagnostics with -batchmode it's now required that you specify both the -username and -password arguments to avoid the USYM_UPLOAD_AUTH_TOKEN error message.

    We are currently looking into a known issue where using the -runTests argument throws this same error on iOS even after including both -username and -password. You can track our progress on that fix here.
     
    DarekRusin likes this.
  17. DarekRusin

    DarekRusin

    Joined:
    Nov 15, 2013
    Posts:
    47
    @ryanc-unity thanks so much! I did not now that specifying username and password is now required. Once I've added these, everything started working properly :)
     
    Ryan-Unity likes this.
  18. Bonich

    Bonich

    Joined:
    Oct 25, 2012
    Posts:
    2
    In teamcity we fix it, by patching process_symbols.sh in xcode project.

    Just change strings with usymtool on something like there:

    #!/bin/sh

    # process_symbols
    #
    # Copyright (c) 2015 Unity Technologies. All rights reserved.

    if [ "${SYNCHRONOUS_SYMBOL_PROCESSING}" = "TRUE" ]; then

    echo test

    else
    nohup "$PROJECT_DIR/usymtool" -symbolPath "$DWARF_DSYM_FOLDER_PATH/$DWARF_DSYM_FILE_NAME" > /dev/null 2>&1 &
    disown
    fi
     
  19. psydent

    psydent

    Joined:
    Feb 11, 2013
    Posts:
    12
  20. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi @psydent, sure thing. usymtool is used to process symbol files from the build and upload them to the Unity Cloud Diagnostics servers. It is required if Cloud Diagnostics is enabled. If you are having an issue with it, please submit a formal bug report with reproduction steps.
     
  21. tonic-joel

    tonic-joel

    Joined:
    Nov 9, 2018
    Posts:
    2
    I'm not sure if'll help anyone else, but when I ran the build command from command line without using batchmode I had the same error appearing in my logs even though I was logged into my account via Unity services.

    Code (CSharp):
    1. Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable
    Switching my command line to use batchmode and adding the username and password resolved the issues however (but adding just the username and password values without adding the batchmode flag still threw the error)

    As an aside, is there any way for us to generate the auth token manually so we don't have to store log in details on our build server in plain text?
     
    DarekRusin likes this.
  22. Ryan-Unity

    Ryan-Unity

    Joined:
    Mar 23, 2016
    Posts:
    1,993
    Hi @tonic-joel, the batchmode command line argument is required in order to be able to run Unity silently from the command line. You need to include batchmode, along with the username and password arguments, in order to build while using Cloud Services like Diagnostics, IAP, and Analytics.

    There currently isn't a publicly available API for generating auth tokens. I understand being reluctant to use plain text log in details so I will pass this along to the team to see what alternatives there might be.
     
  23. sandeepsmartest

    sandeepsmartest

    Joined:
    Nov 7, 2012
    Posts:
    139
    Im facing the issue in the version 2019.3.0f6.I've added username and password in the batch command .Im using jenkins to make ios builds.
    This is my batch command
    Code (CSharp):
    1. -quit -batchmode -serial xxxxxxxxxxxxxx -username 'yyyyy' -password 'zzzzzz' -executeMethod JenkinsBuildHelper.PerformAndroidIOSBuild
    .
    This is error message
    Code (CSharp):
    1. bin/sh -c /Users/jenkins/Library/Developer/Xcode/DerivedData/myproj-gecywtqqtdtvjabrrtiojeiyhwwrh/Build/Intermediates.noindex/ArchiveIntermediates/myproj/IntermediateBuildFilesPath/Unity-iPhone.build/Release-iphoneos/UnityFramework.build/Script-52774D479664D9325892DECA.sh
    2. time="2020-07-03T13:50:48+05:30" level=warning msg="Failed to create log file at /Users/jenkins/Library/Logs/Unity/symbol_upload.log: open /Users/jenkins/Library/Logs/Unity/symbol_upload.log: no such file or directory"
    3. time="2020-07-03T13:50:48+05:30" level=fatal msg="Please provide an auth token with USYM_UPLOAD_AUTH_TOKEN environment variable"
    4. Command PhaseScriptExecution failed with a nonzero exit code
     
  24. sandeepsmartest

    sandeepsmartest

    Joined:
    Nov 7, 2012
    Posts:
    139
    Its working after adding CD access to my jenkins machine user.Thank you
     
    Ryan-Unity likes this.
  25. BenWoodford

    BenWoodford

    Joined:
    Sep 29, 2013
    Posts:
    116
    I have to ask: does the user have to have a seat license? I'd rather not provide my actual Unity ID user/pass into our CI agent, but if I can get away with just putting in a dummy CI user just for symbol upload that'd be great. Our agents already have seats by way of running on our Windows workstations already.

    If not, this is quite a frustrating oversight, and is another one of those instances where Unity's licensing policy hurts paying users more than anyone else.
     
  26. sandeepsmartest

    sandeepsmartest

    Joined:
    Nov 7, 2012
    Posts:
    139
    Hi all,
    Asper this doc we can upload libmain,libunity,libil2cpp symbols to unity cloud diagnostics. We use unity as a libraray in android. Our Build automation system(CI) was able to upload libil2cpp symbols(which gets generated from android studio gradle) automatically to unity CD. However, we could see that our libmain,libunity symbols are missing and which caused unreadle memory addresses in the crash stack trace.
    By using this EditorUserBuildSettings.androidCreateSymbols we can get the symbols of libmain,libunity, can someone confirm/help me if we can upload these symbols programatically (via CI jenkins) without manual upload to CD dashboard?? or by using above API unity uploads all(libmain,libunity and libil2cpp) symbols automatically to CD??
    Please point me in right direction.
    Thanks in advance
     
    PatrickMZ likes this.