Build not always triggering off of a commit

Description of issue:
Commits pushed after a build was queued but before the build actually starts don't get picked up in that build and don't queue another one. So I have to manually trigger or push more changes.

Unity version:
2019.1

XCode version:
N/A

Source control:
git

Local build platform:
Visual Studio / Windows / Unity 2019.1

Build target:
Android

Relevant plugins being used (i.e. Google Play Game Services, other plugins that may perform automatic post-build behaviors)
I doubt it's relevant but I have Unity Ads installed.

Project name:
Last Man Standing

Build log (relevant chunk):
Build #67
STARTED
APR 26, 2019 6:11:36 PM
FINISHED
APR 26, 2019 6:56:36 PM

Files
rev # Timestamp User Files
74428f67aa56e6605dad80d9161c9f359d159e9d 4/26/19 6:11 PM MaxGuernseyIII 8 files changed
Notes: Staging for Unity Cloud Build

389f1644272f7949c068305c21458b5e7e4d1494 4/26/19 9:40 AM MaxGuernseyIII 9 files changed
Notes: Staging for Unity Cloud Build

Build #68
STARTED
APR 27, 2019 10:54:00 AM
FINISHED
APR 27, 2019 11:00:59 AM

Files
rev # Timestamp User Files
69eead83e3282a0ab99c53f7b07efe33f705196d 4/27/19 10:53 AM MaxGuernseyIII 3 files changed
Notes: Staging for Unity Cloud Build

2c961f94b473d84c19184524cbee0c795f2d4046 4/26/19 6:16 PM MaxGuernseyIII 5 files changed
Notes: Staging for Unity Cloud Build



Does it build locally from a fresh checkout? (Instructions below):
This question is not relevant. The project builds but sometimes I have to manually kick it off or wait for an unacceptable delay.

 

A note that is not relevant but will prevent people from pursuing a red herring is that UCB is at the end of my pipeline, not its entirety. That's why all the commit messages are the same. There's an Azure Pipelines script that does the push for me so that I don't have to interact with the staging git repo at all. It doesn't matter who does the push. This is about what happens once assets and binaries are pushed to git.

 

So I'll take the sound of those crickets chirping to mean nobody cares about this and I should just remember to manually run a build when wait times are really long.

 

This seems to be happening more frequently, not less.

 

Hey @MaxGuernseyIII did you hear back on this or resolve firing builds after a commit? I am also seeing UCB not kicking off after I commit to github - note I don't do this from the Unity editor, thanks for your help, Sergio

 

They did make a change but it doesn't solve the problem completely.

When I mentioned that, the response was (paraphrased) "The propose of auto triggering is to reduce the need to manually trigger, not eliminate it all together. So this isn't a bug."

 

thanks @MaxGuernseyIII , Works now except for an outage I've experienced for over 35min but support looked into it and resolved it.

 

I'm glad it works for you. I still plan to switch the Unity part of my pipeline to Azure DevOps because UCB misses a commit on a regular basis.

 

Yeah @MaxGuernseyIII I'd also like to have a more reliable cloud build, but can azure devops build for ios if i'm working from a pc? i haven't yet looked into it, maybe i can setup a virtual osx server within azure to run it, cheers!

 

You can always setup a webhook to achieve the same effect.
We used to have a hook to create a build configuration and start the build for every pull request made within Azure DevOps to UCB.

 

@Tarrag I looked into it. I think you can get away with using a PC for most things but Apple seems to make you use a device to enable two-factor authentication. Once you've done that, I think you can get your provisioning files and whatnot. There was also a service that let you rent a Mac by the hour. I was thinking about using that.

 

@ttermeer-reboundcg Yeah. That's not too far from what I have. Every time a build completes, the build checks stuff in to git and moves through. I could also turn off the auto-build feature and have Azure DevOps "push the button" for me, too. I question whether that's more work than another option. Plus, I haven't figured out a good way to re-insert the stuff back into the Azure DevOps pipeline. Right now, I use a webhook to start the build via an Azure Function. The build uses the API to get the build. What I can't see is a good way to connect the initiating build in Azure DevOps to the output generated by UCB. I have flow but no traceability. If I switch over to an all-Azure-DevOps solution, I will get triggers that actually work and traceability all the way from check-in to release.

 

@Tarrag I misread your comment. They have hosted Mac agents, so I don't know that it matters. Whatever you did to get UCB building can be done to get it building on Azure DevOps. The tricky bit would be making sure that the right version of Unity is installed. That actually seems like it would be a pain to maintain. Maybe waiting for Microsoft to buy Unity is a better plan.

 

Thanks for the update @MaxGuernseyIII

 

We are considering this approach so that we can leverage the release pipeline (for approvals and consistency). Can you provide more information on how you did this?

 

I don't know how @ttermeer-reboundcg did it but I set up an Azure Function that could receive the webhooks and then using the Azure DevOps API to trigger a build with some parameters. If there's a way to set up a receiving webhook directly through Azure DevOps, it's either brand new or very well hidden.

 

@MaxGuernseyIII Does that mean your commit triggers the UCB build and then sends the webhook to the Azure Function and then the function sends a API post to trigger a build?

 

@mayeager22 I suggest you read the following articles :
https://docs.microsoft.com/en-us/azure/devops/service-hooks/services/webhooks?view=azure-devops
https://build-api.cloud.unity3d.com/docs/1.0.0/index.html

I personally was using hook.io that used to provide free webhook server but you can use any php or js server (anything that can send a web request)
Here the script I used to create a build config and start the build from a pull request in Azure DevOps

Code (JavaScript):

1. /**
2. * Hook on PR request to create build on Unity Cloud
3. */
4. module['exports'] = function myService (hook) {
5.  
6.     var name = "PR Android " + hook.params.resource.pullRequestId;
7.  
8.     var request = require('request');
9.  
10.     var buildInfo = {
11.       "name": name,
12.       "platform": "android",
13.       "enabled": true,
14.       "settings": {
15.         "autoBuild": true,
16.         "unityVersion": "latest2017_1",
17.         "executablename": "",
18.         "scm": {
19.           "type": "git",
20.           "branch": hook.params.resource.sourceRefName.replace("refs/heads/",""),
21.           "subdirectory": "",
22.           "client": ""
23.         },
24.         "platform": {
25.           "bundleId": "com.company.project",
26.           "xcodeVersion": ""
27.         },
28.         "advanced": {
29.           "xcode": {
30.             "useArchiveAndExport": false
31.           },
32.           "unity": {
33.             "preExportMethod": "",
34.             "postExportMethod": "",
35.             "preBuildScript": "",
36.             "postBuildScript": "",
37.             "scriptingDefineSymbols": "",
38.             "playerExporter": {
39.               "sceneList": [],
40.               "buildOptions": []
41.             },
42.             "playerSettings": {
43.               "Android": {
44.                 "useAPKExpansionFiles": false
45.               }
46.             },        
47.             "editorUserBuildSettings": {
48.               "androidBuildSystem": "gradle"
49.             },
50.             "runUnitTests": true,
51.             "assetBundles": {
52.               "buildBundles": false,
53.               "basePath": "",
54.               "buildAssetBundleOptions": "",
55.               "copyToStreamingAssets": false,
56.               "copyBundlePatterns": []
57.             },
58.             "failedUnitTestFailsBuild": true,
59.             "unitTestMethod": "UnityTest.Batch.RunUnitTests",
60.             "integrationTestMethod": "",
61.             "failedIntegrationTestFailsBuild": false,
62.             "integrationTestSceneList": []
63.           }
64.         }
65.       },
66.       "credentials": {
67.         "signing": {
68.           "credentialid": "SIGNING_CREDENTIAL"
69.         }
70.       }
71.     };
72.  
73.     var options = {
74.       method: 'POST',
75.       url: 'https://build-api.cloud.unity3d.com/api/v1/orgs/YOURORG/projects/YOURPROJECT/buildtargets',
76.       headers: {
77.         'Authorization': 'Basic AUTHID_FOR_UCB'
78.       },
79.       json: true,
80.       body: buildInfo,
81.     };
82.  
83.  
84.  
85.     //callback when the build config is created
86.     //we then start the build
87.     function callbackCreate(error, response, body) {
88.       if (error) {
89.         return hook.res.end("error: " + error);
90.       }else{
91.         //start the build
92.         options.url = 'https://build-api.cloud.unity3d.com/api/v1/orgs/YOURORG/projects/YOURPROJECT/buildtargets/' + body.buildtargetid + '/builds';
93.         options.body = {clean: true};
94.         request(options,callbackBuild);
95.       }
96.     }
97.  
98.     //callback when the build has been started
99.     function callbackBuild(error, response, body){
100.       if (error) {
101.         return hook.res.end("error: " + error);
102.       }
103.       hook.res.json(body);  
104.     }
105.  
106.     //create the build
107.     if(hook.params.resource.targetRefName == "refs/heads/master")
108.         request(options, callbackCreate);
109.     else
110.       hook.res.end();
111.   };

 

My commit triggers UCB. A UCB webhook trigger calls my Azure Function. The Azure Function captures critical details about a build (the link, id, et cetera), packs them up as parameters, and invokes the appropriate Azure DevOps build. Whichever build is invoked downloads the UCB output and does whatever needs to be done (e.g., applying a custom WebGL template) then publishes everything necessary for further processing as artifacts that can be consumed by downstream build or release pipelines.

Downstream builds are things like on-device or on-emulator test passes.

 

I think UCB calls it a webhook "integration".

 

@MaxGuernseyIII What task are you using in your Azure DevOps pipeline to download the UCB output?

 

@mayeager22 I mixture of sadness and Powershell.

The sadness is in the form of it only working on Linux agents (at least when I wrote it. Here's the Powershell part:

Code (Powershell):

1. $Descriptor = Invoke-WebRequest -Uri '$(Download.BuildPath)' -Method 'GET' -Headers @{ Authorization = '$(Download.Authorization)' } | ConvertFrom-Json
2.  
3. $DownloadLink = $Descriptor.links.download_primary.href
4.  
5. Invoke-WebRequest -Uri $DownloadLink -Outfile '$(Download.ToFile)'  -Method 'GET'

 

@MaxGuernseyIII I am trying something very similar on a Windows agent, but get errors about the signature not matching (SignatureDoesNotMatch). Do you recall if that was the issue you had?

I get it downloading from Linux, but sadly I have a task that run a Powershell to start the builds on UCB that won't launch on Linux.

 

Yeah. It's a pain in the butt. I ran it on Linux for some similar reason.

I created a separate pipeline for that. It has a Linux job with a single focus: download the file and insert it into the pipeline as an artifact.

So the code you are looking at is pretty much all there is to the pipeline:

1. Make sure the download path is a real thing.
2. Execute that code.
3. Upload the downloaded file as a pipeline artifact.

 

@MaxGuernseyIII I was able to get everything working on a Linux hosted agent (after I overcame some issues with PowerShell Core breaking changes in Invoke-RestMethod) Thank you for pointing me in that direction to fix the download issue!

 

@mayeager22 I'm glad you got it fixed.