Feedback My disappointing experience using Unity in a CI pipeline

This past week I've been trying to set up a CI pipeline for my game. I'm currently using the Personal edition of Unity 2019.2.2, but will upgrade to Unity Plus if I need to. I want my pipeline to automatically do these three things, so I don't have to do them by hand:

• To ensure that my tests pass when I make changes to my game
  
• To ensure that my game builds on all the platforms I support
• To deploy to all platforms and storefronts I support

I looked into Unity Cloud Build, but unfortunately that's not an option for me, for these reasons:

• My game uses Burst, which cannot be cross-compiled, and therefore cannot be built on Unity Cloud Build
  
• Unity Cloud Build does not support building for consoles, which I intend to do
• I'm using Trimmer to build multiple variations of my game, and Unity Cloud Build doesn't support some of the features that Trimmer uses

Instead, I decided to try out the GitHub Actions beta for these reasons:

• I already have a paid GitHub plan, so I get 3000 minutes included per month
• It supports Windows, macOS, and Linux, so I can build for any supported Unity platform on the same pipeline
  
• If I want to add some build step external to Unity (like importing files or generating code), I can do that at my leisure

GitHub Actions has been a great experience for me; it's doing everything I want it to and doesn't get in my way with technical limits. Seriously, give it a try...with a project unrelated to Unity. Here's why.

Unity recently published some improvements to their test framework, and has expressed a renewed interest in CI. These new efforts led me to believe that I could set something up that would at least fit my requirements, even if there were some warts along the way. I was mistaken.

Unity's licensing mechanism is 100% getting in the way of me building the CI pipeline that I want. Currently, a Unity license must be obtained manually, either in the editor or through the process described here and here. For manual use (i.e. when actually designing a game), this is fine; I had no issue installing and licensing Unity for my development machine.

The problem is that you still have to do this when installing Unity on a CI runner. Most CI services don't make promises about a runner's identifying details (network address, hostname, serial numbers, other OS identifiers). As a result, a Unity license for a CI runner could become useless at any time without warning. In fact, that's exactly what's been happening with my own pipeline. This undermines every effort the Unity developers make to offer a smooth automation experience. What's the point of automated builds, tests, or deployments if I have to manually license Unity every time I run my CI pipeline?

I don't have a problem with downloading and installing Unity on the CI runners, nor do I with actually invoking it on the command line. It's rougher around the edges than I'd prefer, but at least it fits my needs. All I want is to be able to use Unity for automated builds, test, and deployments in a CI runner without any manual intervention; the best way I can see that is to remove or totally automate the licensing process for CI pipelines.

Or, if your legal team would get nervous, perhaps consider special CI-only builds of Unity that don't need to be licensed, but can only be run headless? Such a build couldn't be used for designing content, and thus would not be worth pirating.

I love Unity and it's generally a bliss to work with, but in this case I'm incredibly disappointed. I know the dev team can do better.

 

Hi, Richard. Thank you for your reply, especially on a Saturday night.

No. This is what I did for each operating system;

1. Spin up a runner
2. Install Unity (or, in the case of Linux runners, pull this Docker image) as part of the script
   
3. Request a license as described here (section "Create activation file and import license by command"), and save the .alf file
4. Add the generated .alf file as an artifact, then manually get a license as described here (The runner is finished by this point)
5. Manually add the contents of the .ulf license file as a secret to my GitHub configuration (not the repository, to a config item on the website)
   
6. Spin up a new runner of the same OS (I can't continue using the first runner because of the manual intervention needed)
7. Install Unity again
8. Activate the license as described here (section "Create activation file and import license by command")
9. Run the tests or build the game (depending on the workflow that's running)
10. If applicable, deploy the game

If I already have a license then steps 1-5 are ignored. They're still run, but I haven't figured out how to conditionally disable them yet.

The problem is:

• This works well enough on Github Actions' Linux runners, I guess because of the Docker image I'm using. But I have no idea when the license will be made useless under my feet (because of changes to the runner's environment)
• I believe that at a minimum I have to do this every time I upgrade Unity (which I do frequently, even for patch releases)
• This doesn't work at all on the macOS or Windows runners, because whatever information Unity uses to request a license doesn't seem to be constant as I expected. As a result, I have not managed to make a single successful macOS or Windows build on GitHub Actions. When I try to activate Unity on these runners I will get errors that look like this:

Code (csharp):

1. /Applications/Unity/Unity.app/Contents/MacOS/Unity -quit -batchmode -username <redacted> -password <redacted> -manualLicenseFile /Users/runner/.local/share/unity3d/Unity/Unity_lic.ulf -nographics -silent-crashes -disable-assembly-updater -logFile -
2.  
3. Initiating legacy licensing module
4. DisplayProgressbar: Unity license
5.  
6. LICENSE SYSTEM [2019831 16:49:13] Next license update check is after 2019-09-01T16:20:51
7.  
8.  
9. LICENSE SYSTEM [2019831 16:49:13] VMuD6VlWKP6I != VMPkeFQqLH/F
10.  
11. [UnityConnectServicesConfig] config is NOT valid, switching to default
12. User <redacted> logged in successfully
13. Cancelling DisplayDialog: Failed to activate/update license. Timeout occured while trying to update license. Please try again later or contact support@unity3d.com
14. This should not be called in batch mode.
15. (Filename: /Users/builduser/buildslave/unity/build/Editor/Platform/OSX/EditorUtility.mm Line: 288)
16.  
17.  

It looks about the same on the Windows runners, but I can post more detailed information or logs if you'd like.

And, of course, these extra steps eat up my runner minutes.

Color me intrigued. If it's at all possible, can you tell me more about how it'll work, and what kind of timeline you have in mind?

 

Huh. Wasn't aware of this. So Plus/Pro don't use the

.alf

/

.ulf

system?

Forgot to mention; another big plus of GitHub Actions is that I can run many jobs at the same time. As of right now, my attempt at a pipeline runs ten jobs concurrently; four to activate Unity (one for each build target), four to build and deploy the game (one for each build target), and two to run the tests (play mode and edit mode).

I could try this out, but the FAQ tells me that I can only use two machines per license. That's a problem, because I have Unity Personal installed on three computers.* Even if I uninstalled Unity on two of them (I can't because I use them for development), I wouldn't be able to run more than one job at a time.

*Two unique machines, one of which has two operating systems installed.

Fair enough. Thank you for the honesty; if this is already in the oven then I suppose all you can do is keep cooking it. Do you at least have an idea of the order of magnitude you're working with? (Weeks? Months? Years?)

 

+1 for this issue in general.
Also +1 for a completely headless unity executable and headless installer. This would make it possible to run on more efficient headless OSes and make it possible to automate the installation of new unity versions on the runners. The last one is important if you update often.

Edit: It would be a dream if it could be run on Windows Nano. Windows Server is so heavy.

 

Is this inherent to the licensing process? There's no way you could offer serials for Personal?

My budget is best described as "shoestring". I can't justify three (builders + test runners) Plus licenses right now.

Can you tell us more about these client-side pieces that made it to 2019.3?

 

omg I've been banging my head against the wall trying to get github action working. following this thread

 

As a quick fix, could you not just add something like a

-personal

command line switch? These licenses are handed out for free anyway.

 

@superpig i've tried the -serial option a long time ago, it did not work for us.
See my other (orphan) thread that received 0 replies here: https://forum.unity.com/threads/automated-unity-license-activation.770069/

 

I have a similar problem with the current license:

I have a github repository with a github action that downloads and install unity personal in a docker container with a manual activated license. https://github.com/MirrorNG/MirrorNG/runs/440714979?check_suite_focus=true

It was a royal pain to set up, but alas, it is done.

The challenge I have is that the license is set up as a secret, which means it is not available for pull requests from forks. Note this restriction applies to any CI/CD tool, not just github actions. So I cannot run my CI/CD pipeline for pull requests. I cannot tell if a pull requests passes the tests or even compile without a lot of manual work.

I would really like for a way to have unity command line run without having to provide secrets.

 

@JesseSTG So building the healdess server in Docker is your problem. Not releasing the headless build in a docker container to a server to be played on? Am I correct in my assumption of this?

 

My game doesn't run as a server, it's offline. I'm not sure I understand the question, can you clarify?

 

Digging this up. After 1 year, where are we on this ?

I'm currently setting up Github actions and it seems to run "okayish", but I still wish the process was simpler.

 

Hey there! Unity CICD is royal pain. Currently struggling to setup github actions on unity game. Do you have any working reference code then it would really help me.

Thank you.

 

Nope, sorry. I'm setting up a solution on AWS instead; Unity just takes too long to install and set up for it to be viable on GitHub Actions. When it's ready I'll be sure to share some code.

 

I have managed to automate the update license task, see the link https://github.com/jcs090218/JCSUnity/blob/master/.github/workflows/license.yml.

The license is updated every time in schedule (everyday in my workflow file), so you don't have to worry about license expiration anymore!