CI unity to .ipa

Hi,
I'm trying to make an CI to automate the release of the unity project to AdHoc and deploy to AppCenter. I already have the certificate with the private key along the AdHoc provisioning and development provisioning profile. I don't want any human intervention nor GUI operation and the provisioning with the UnityFramework project is bugging me for days now. He doesn't like manual provisioning and doesn't seem to be affect by the player settings netter. I'm using unity 2019.3.9f1 and Xcode 11.4.1. Here is my flow:

Code (JavaScript):

1. // Unity
2. export PATH=/Applications/Unity/Hub/Editor/2019.3.9f1/Unity.app/Contents/MacOS:$PATH
3. cd /var/jenkins/workspace/MyProj/
4. export AMOTUS_UNITY_BUILD_DIR=`pwd`/AppBuild  // This set our output path where a subfolder with the platform either iOS or Android to put the result
5. Unity -quit -batchmode -nographics -logFile - -buildTarget Android -projectPath . -executeMethod BuildCommand.BuildAndroid
6. Unity -quit -batchmode -nographics -logFile - -buildTarget iOS -projectPath . -executeMethod BuildCommand.BuildIOS
7.  
8. // Keychain and provisioning
9. security list-keychains -s /Library/Keychains/System.keychain /Users/jenkins/Library/Keychains/login.keychain-db /var/jenkins/workspace/MyProj/Mobile.keychain
10. open /var/jenkins/workspace/MyProj/Mobile_Development.mobileprovision
11. open /var/jenkins/workspace/MyProj/Mobile_AdHoc.mobileprovision
12. security unlock-keychain /var/jenkins/workspace/MyProj/Mobile.keychain
13. security set-keychain-settings /var/jenkins/workspace/MyProj/Mobile.keychain
14. security set-key-partition-list -S apple-tool:,apple: -s -k $PASSWORD_KEYCHAIN /var/jenkins/workspace/MyProj/Mobile.keychain
15. security default-keychain -s /var/jenkins/workspace/MyProj/Mobile.keychain
16.  
17. // Xcode iOS
18. cd AppBuild/iOS
19. xcodebuild -project Unity-iPhone.xcodeproj -allowProvisioningUpdates -configuration Release -scheme Unity-iPhone build
20. xcodebuild -project Unity-iPhone.xcodeproj -allowProvisioningUpdates -configuration Release -scheme Unity-iPhone CODE_SIGN_STYLE="Manual" PROVISIONING_PROFILE_SPECIFIER="Mobile Development" build
21. xcodebuild -project Unity-iPhone.xcodeproj -allowProvisioningUpdates -configuration Release -scheme Unity-iPhone -archivePath /var/jenkins/workspace/MyProj/AppBuild/iOS/tmp.xcarchive archive
22. xcodebuild -exportArchive -allowProvisioningUpdates -archivePath /var/jenkins/workspace/MyProj/AppBuild/iOS/tmp.xcarchive archive -exportPath . -exportOptionsPlist /var/jenkins/workspace/MyProj/exportOptionsAdHoc.plist

If I try with automatic signing from the player setting and the development to build and archive it work but the .ipa with the adhoc provisioning fail.

Code (CSharp):

1. Check dependencies
2. 17:43:45  Code Signing Error: Revoke certificate:  Your account already has an Apple Development signing certificate for this machine, but its private key is not installed in your keychain. Xcode can create a new one after revoking your existing certificate.
3. 17:43:45  Code Signing Error: No profiles for 'com.mycie.myapp' were found:  Xcode couldn't find any iOS App Development provisioning profiles matching 'com.mycie.myapp'.
4. 17:43:45  Code Signing Error: Code signing is required for product type 'Application' in SDK 'iOS 13.4'
5. 17:43:45  Code Signing Error: Code signing is required for product type 'Application' in SDK 'iOS 13.4'

But building from Xcode gui and selecting the manual and selecting the provisioning from the dropdown seem to work just fine and I can build and generate the archive and the .ipa?! why is automatic cannot find it?!? The certificate and the private key is provided into the added keychain. The provisioning is working when I go through the GUI so the file and certificate are good. Is there a way to give indication without failing automatic?

If I try manual signing (into player settings) and change the build command line above with

Code (JavaScript):

1. xcodebuild -project Unity-iPhone.xcodeproj -allowProvisioningUpdates -configuration Release -scheme Unity-iPhone CODE_SIGN_STYLE="Manual" PROVISIONING_PROFILE_SPECIFIER="Mobile AdHoc" build

I get the following error (the Apple distribution is correct and team is correct) but the unity framework doesn't like any signing, if I go inspect the project the iPhone-Unity project is manual signing, but the UnityFramework is still in automatic and should not require signing?!?

Code (JavaScript):

1. Signing Identity:     "Apple Distribution: ***** (****)"
2. 12:54:30
3. 12:54:30      /usr/bin/codesign --force --sign **** --preserve-metadata=identifier,entitlements,flags --timestamp=none /Users/jenkins/Library/Developer/Xcode/DerivedData/Unity-iPhone-cnjsujuqpqggxmemraozskoostni/Build/Products/Release-iphoneos/MyProj.app/Frameworks/UnityFramework.framework
4. 12:54:31  /Users/jenkins/Library/Developer/Xcode/DerivedData/Unity-iPhone-cnjsujuqpqggxmemraozskoostni/Build/Products/Release-iphoneos/MyProj.app/Frameworks/UnityFramework.framework: errSecInternalComponent
5. 12:54:31  Command /usr/bin/codesign failed with exit code 1

What is the proper way to build those unity project from the command line without human intervention that can deploy with either AdHoc and AppStore?
Apple make it really hard to perform such a simple task, is nobody at Apple that use CI and command line and where is gone the it just work Moto, I guess it died with Jobs... The unityframework is not helping here either. some clear instruction would be welcome.

Thanks

 

Nobody have any clue on how to achieve this properly and what is the proper way to build without using the GUI? what is the procedure to do this!?

 

I did try to remove the provisioning files from the system and let Xcode do his own jobs on his own and not add them. This seem to work from the Terminal. Probably something to do with the Jenkins slave accessed into ssh and the provisioning! The exact same command on the build folder from the Terminal is working but not when executed from Jenkins (Slave, through SSH). So the SSH session is interfering with the build process somehow.

The only thing I did found is the keychain unlock and allow access to apple tooling which I already did.

 

Ok found one pit fall, the keychain need to be lock and unlock by the ssh user to ensure the proper access is granter to the ssh user! Now I can generate the build and the archive. But I end up with the following error:

Code (JavaScript):

1. xcodebuild -exportArchive -allowProvisioningUpdates -archivePath /var/jenkins/workspace/JenkinsDevel/TestJenkinsfile/AppBuild/iOS/tmp.xcarchive -exportPath . -exportOptionsPlist /var/jenkins/workspace/JenkinsDevel/TestJenkinsfile/exportOptionsAdHoc.plist
2. 16:30:45  2020-05-12 16:30:44.837 xcodebuild[65248:7128642] [MT] IDEDistribution: -[IDEDistributionLogging _createLoggingBundleAtPath:]: Created bundle at path '/var/folders/91/159j7k_d2754lllqnn34w5lc0000gr/T/Unity-iPhone_2020-05-12_16-30-44.837.xcdistributionlogs'.
3. 16:38:12  error: exportArchive: Code signing "UnityFramework.framework" failed.
4. 16:38:12
5. 16:38:12  Error Domain=IDEDistributionPipelineErrorDomain Code=0 "Code signing "UnityFramework.framework" failed." UserInfo={NSLocalizedDescription=Code signing "UnityFramework.framework" failed., NSLocalizedRecoverySuggestion=View distribution logs for more information.}

I need to find out what this error means. This is horribly painful for something that should be super easy.

 

Digging further, the archive is made with the automatic signing properly. Now I need to generate the .ipa with the AdHoc provisioining for example but I get the following error as:

Code (JavaScript):

1. 16:56:15  + xcodebuild -exportArchive -allowProvisioningUpdates -archivePath /var/jenkins/workspace/JenkinsDevel/TestJenkinsfile/AppBuild/iOS@tmp/xcode/tmp.xcarchive -exportPath . -exportOptionsPlist /var/jenkins/workspace/JenkinsDevel/TestJenkinsfile/exportOptionsAdHoc.plist
2. 16:56:17  2020-05-12 16:56:17.042 xcodebuild[69851:7150745] [MT] IDEDistribution: -[IDEDistributionLogging _createLoggingBundleAtPath:]: Created bundle at path '/var/folders/91/159j7k_d2754lllqnn34w5lc0000gr/T/Unity-iPhone_2020-05-12_16-56-17.041.xcdistributionlogs'.
3. 17:03:44  error: exportArchive: Code signing "UnityFramework.framework" failed.
4. 17:03:44
5. 17:03:44  Error Domain=IDEDistributionPipelineErrorDomain Code=0 "Code signing "UnityFramework.framework" failed." UserInfo={NSLocalizedDescription=Code signing "UnityFramework.framework" failed., NSLocalizedRecoverySuggestion=View distribution logs for more information.}

If I dig the log:

Code (JavaScript):

1. 2020-05-12 21:03:33 +0000 [MT] Running /usr/bin/codesign '-vvv' '--force' '--sign' 'BUNCH OF NUMBER HERE' '--entitlements' '/var/folders/91/159j7k_d2754lllqnn34w5lc0000gr/T/XcodeDistPipeline.~~~MTiqKC/entitlements~~~Bxk5RY' '--preserve-metadata=identifier,flags,runtime' '/var/folders/91/159j7k_d2754lllqnn34w5lc0000gr/T/XcodeDistPipeline.~~~MTiqKC/Root/Payload/as-built-drawing-app.app/Frameworks/UnityFramework.framework'
2. 2020-05-12 21:03:33 +0000  /var/folders/91/159j7k_d2754lllqnn34w5lc0000gr/T/XcodeDistPipeline.~~~MTiqKC/Root/Payload/as-built-drawing-app.app/Frameworks/UnityFramework.framework: errSecInternalComponent
3. 2020-05-12 21:03:33 +0000 [MT] /usr/bin/codesign exited with 1

Anybody have an idea of what this mean? How to solve the problems?!? Why is it signing the Unity.Framework anyway, doesn't this framework doesn't need signing??? anything special needed to be done before?! or any config into the export.plist necessary?!

 

Probably something to do with the ssh user, the exact same command on the same archive from same user login into a terminal complete successfully:

Code (JavaScript):

1. Exported Unity-iPhone to: /var/jenkins/workspace/JenkinsDevel/TestJenkinsfile/AppBuild/
2. ** EXPORT SUCCEEDED **

The unity framework signing is looking for something that my ssh user cannot access, I dunno what is not already cover with the above keychain unlock, default, apple permission!?!? The AdHocProvisioning is also installed. The exportOptionsAdHoc.plist look like this:

Code (JavaScript):

1. <?xml version="1.0" encoding="UTF-8"?>
2. <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3. <plist version="1.0">
4. <dict>
5.     <key>signingStyle</key>
6.     <string>manual</string>
7.     <key>method</key>
8.     <string>ad-hoc</string>
9.     <key>teamID</key>
10.     <string>MYTEAMIDHERE</string>
11.     <key>provisioningProfiles</key>
12.     <dict>
13.         <key>com.mycie.myapp</key>
14.         <string>MyProvisioning AdHoc</string>
15.     </dict>
16. </dict>
17. </plist>

 

I'm struggling with something similar. If I find anything useful, I'll contribute it, here.

 

I managed to have a working recipe, but removing all provisioning file, only adding the AdHoc or AppStore one (do not leave any developer provisioning on the machine and the apple dev account (seem like xcode doesn't like this even if not installed on the machine).

I do the whole build/archive with automatic signing. Only when moving from the archive to .ipa, I do sign the application with the AdHoc, AppStore signing.

The login keychain must also be unlock (I keep the certificate of the applicaiton into his own keychain file into the repos and unlocking it not enough, you also have to unlock the login one (I have no idea what it fetch into it, but xcodebuild need it).

Also lock then unlock the keychain if you are an ssh user (the unlock form the same GUI user is not usable by ssh user, go figure why, but locking the keychain and unlocking it solve problems for CI ssh users).

You also need to allow apple and apple-tool kaychain partition like stated above.

All those make it work for me, result might vary with version and setup. The Apple signing is such a pain, the black box requirement to sign is painful at best, I'm sure they added a hell layer just for the people who create this dev experience.

 

SOLUTION / Solved:

You probably have the keychain locked in the shell command line you're using.

You can unlock it in the command line with the following command line:

Code (Shell):

1. # Unlock keychain
2. security unlock-keychain -p mypassword
3.  
4. # Pack
5. xcodebuild -verbose -exportArchive -archivePath Unity-iPhone.xcarchive -exportPath myexportpath/ -exportOptionsPlist /my_export_options.plist

 

Yeah this is what I do, since I carry my .keychain file for my project along with it. There is some extra step to make this work:

Code (JavaScript):

1.  
2. // Add my keychain file to list of keychain
3. sh("security list-keychains -s  \$(echo \$(echo `security list-keychains | xargs` \"${keychain_file}\") | xargs -n1 | sort -u | xargs)");
4.  
5. // Lock keychain, just in case it was already unlock, gui user and ssh user from Jenkins unlock request are not playing well together, go figure why, one hell of Apple way
6. sh("security lock-keychain ${keychain}");
7.  
8. // Do the same for the login keychain
9. sh("security lock-keychain login");
10.  
11. // Unlock my keychain
12. sh("security unlock-keychain -p \"${password}\" ${keychain}");
13.  
14. // Unlock login.keychain too, I have no idea why thi is required but you have to do it, another of Apple secrets
15. sh("security unlock-keychain -p \"${password}\" login");
16.  
17. // Keep keychain unlock
18. sh("security set-keychain-settings ${keychain}");
19.  
20. // Allow apple signing tool to access keychain
21. sh("security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k \"${password}\" ${keychain}");
22.  
23. // Set my keychain as the default one
24. sh("security default-keychain -s ${keychain}");
25.  
26. // Build here
27.  
28. // Remove keychain
29. sh("security delete-keychain \"${keychain}\"");
30.  

Then I can build and sign properly. This is one hell of a flow to discover to make this work.

 

Had you solved the problem?

my error is that autosign will cover unity-iphone and unityframework.
so that can't archive if I use CI. it's not about the keychain .

 

I do not sign the application until the archive is made, then I sign it. Else the Unity framework always complain unmatching signing and it doesn't seem to allow strong signing. I just gave up on this, leave it into dev auto sign, until archive is made, then I use xcode tools to sign the archive and generate the .ipa.

This whole process is so painful. Both Unity and Apple bad tooling doesn't help to make this work properly. They both lack any information and flow example for this.

 

Ok, I set archive args: CODE_SIGNING_ALLOWED="NO".
it can be archived success, then I use xcodebuild -exportArchive. Did it need sign on exportIPA by Jenkins CI?

 

What I do is generate the archive then when I do the export to .ipa I do the signing with the export plist:

Code (CSharp):

1. xcodebuild -exportArchive -allowProvisioningUpdates -archivePath \"${archive_path}\" -exportPath \"${ipa_folder}\" -exportOptionsPlist \"${export_options_path}\""

Use something like that for the export options plist file:

Code (CSharp):

1. <?xml version="1.0" encoding="UTF-8"?>
2. <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3. <plist version="1.0">
4. <dict>
5.     <key>signingStyle</key>
6.     <string>manual</string>
7.     <key>method</key>
8.     <string>app-store</string>
9.     <key>teamID</key>
10.     <string>INSERT_TEAM_ID_HERE</string>
11.     <key>provisioningProfiles</key>
12.     <dict>
13.         <key>ca.amotus.NubitusApp</key>
14.         <string>PROVIONING_FILENAME_HERE_NO_EXTENSION</string>
15.     </dict>
16. </dict>
17. </plist>

 

Make sure to open the provisioning file with xcode first or install them on the node machine.
This is why I keep all my provisioning, keychain and Options plist into a repository. You probably want to secure that repos and limit access to it. So the CI script install on the node the needed stuff and use them, and clean after. This limit me to a single build per node to avoid problems but seem to be working just fine as long as you figure all the keychain mess above. I also clean any previous provisioning and leave only the one I need for the current build to avoid xcode easily getting all mix up with which provisioning to take. The only thing I wasn't able to automate was the appleid signing into xcode by the CI. This seem nearly impossible, you need to do it by the GUI, maybe using the build bot into xcode could solve this but I did not had time to dig into it.

 

Our apps are so many, and each one can't be set provisioningProfiles , so I must signing automatic, this is my plist:

Code (CSharp):

1. <?xml version="1.0" encoding="UTF-8"?>
2. <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3. <plist version="1.0">
4. <dict>
5.     <key>compileBitcode</key>
6.     <false/>
7.     <key>destination</key>
8.     <string>export</string>
9.     <key>method</key>
10.     <string>ad-hoc</string>
11.     <key>signingStyle</key>
12.     <string>automatic</string>
13.     <key>stripSwiftSymbols</key>
14.     <true/>
15.     <key>teamID</key>
16.     <string>TEAEID..</string>
17.     <key>thinning</key>
18.     <string>&lt;none&gt;</string>
19. </dict>
20. </plist>
21.  

I found you use provisioningProfiles to manul it, this is only different between us. It's so weired!

 

this plist is copy from that I manully archived by Xcode GUI.

 

After CI build, this is error output

Code (CSharp):

1. 09:36:54 ** ARCHIVE SUCCEEDED **
2. 09:36:54
3. 09:36:54 2020-12-02 09:36:54.799 xcodebuild[41694:1936435] [MT] IDEDistribution: -[IDEDistributionLogging _createLoggingBundleAtPath:]: Created bundle at path '/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/Unity-iPhone_2020-12-02_09-36-54.799.xcdistributionlogs'.
4. 09:36:55 error: exportArchive: There are no accounts registered with Xcode.
5. 09:36:55
6. 09:36:55 Error Domain=IDEProvisioningErrorDomain Code=24 "There are no accounts registered with Xcode." UserInfo={IDEDistributionIssueSeverity=3, NSLocalizedDescription=There are no accounts registered with Xcode., NSLocalizedRecoverySuggestion=Add your developer account to Xcode}
7. 09:36:55
8. 09:36:55 error: exportArchive: No signing certificate "iOS Distribution" found
9. 09:36:55
10. 09:36:55 Error Domain=IDECodesignResolverErrorDomain Code=1 "No signing certificate "iOS Distribution" found" UserInfo={IDEDistributionIssueSeverity=3, NSLocalizedRecoverySuggestion=No "iOS Distribution" signing certificate matching team ID "7235MNZLSP" with a private key was found., IDEProvisioningError_UserInfoKey_IDEProvisioningErrorSpecifier=iOS Distribution, NSLocalizedDescription=No signing certificate "iOS Distribution" found, IDEProvisioningError_UserInfoKey_IDEProvisioningErrorPlatform=com.apple.platform.iphoneos, IDEProvisioningError_UserInfoKey_IDEProvisioningErrorAction=5, IDEProvisioningError_UserInfoKey_IDEProvisioningErrorTeam=<IDEProvisioningBasicTeam: 0x7feb29890ce0; teamID='7235MNZLSP', teamName='(null)'>}

 

I never found a solution where the automatic actually worked, this is why I used manual, I only have a single export .plist and add all my application into it. I do generate an adhoc and a store provisioning for each of my application. i think you could script that to do it for youif you have that many application. Sorry, my knowledge of that Apple signing is limited and the behind the scene is so damn opact I have no idea how automatic select his signing and even which one it did try or why.

Having at least a debug trace about which provisioning was used would be good, and why it cannot be used even better, but sadly the console output is as verbose as seaweed in the Sahara.

 

We also faced this issue.
What I found, that in DistributionSummary.plist which we're getting after signing, a section of the team doesn't have "Name" of the team. other sections are equals by CI. Manually all fields are filled

 

Hi
I am also trying to automate iOS app build through command line. While exporting using
"xcodebuild -exportArchive", I am getting the following error:


2021-02-25 14:04:18.894 xcodebuild[9142:73315] [MT] IDEDistributionMethodManager: -[IDEDistributionMethodManager orderedDistributionMethodsForTask:archive:]: Error = Error Domain=IDEDistributionMethodManagerErrorDomain Code=2 "Unknown Distribution Error" UserInfo={NSLocalizedDescription=Unknown Distribution Error}

error: exportArchive: exportOptionsPlist error for key 'method': expected one of {}, but found ad-hoc

Error Domain=IDEFoundationErrorDomain Code=1 "exportOptionsPlist error for key 'method': expected one of {}, but found ad-hoc" UserInfo={NSLocalizedDescription=exportOptionsPlist error for key 'method': expected one of {}, but found ad-hoc}

** EXPORT FAILED **


Does anyone has working solution to automate the process? @JeromeGodboutAmotus, Have you seen this error?
I would really appreciate your help. Thanks

 

From the error message it seem like you have a typo or wrongly formated exportInfo.plist file, check it again and run an xml validator on it to ensure nothing is wrong. Yiou can also open it with xcode to see if he can see all the field properly.

 

Damn, it seem to be broken into recent Mac OS release, seem like the keychain has changed and the tricks above doesn't work anymore. The keychain unlock and operation seem to failed into Big Sur (still work into previous mac os version). Seriously wish Apple would make it easy to do this, why can't I have a tool that take a .keychain file with the password and the provisioning file and be done with it!? Come on Apple this is not into the "it just work" mindset at all. I understand why people can create a business just for this nightmare (fastlane anyone)!

 

I dig a little further, seem like path with space ain't working anymore, the command path is escaped with "", so it seem like a regression into security command line... damn

 

"It just works" is what Apple says not what they do.