u3d an open source tool to Download, Install, Run Unity from the command line.

Hello,

the DragonBox team is happy to release u3d, an open source project to interact with Unity from the command line.

It currently allows to:

• download and/or install unity
• uninstall unity
• list the installed versions and their packages
• run unity
• prettify the output
• display your activated licenses
• integrates with FastLane (comes with its own action), and is Jenkins friendly

It knows about your project and can make decisions based on its configurations (which version you use)

It works on Mac, Linux, and Windows.

The latest version is v1.0.21, released 2018-04-27. Changelog

Some screenshots:



For more information about the features or question, check the project page on https://github.com/DragonBox/u3d

We would love your feedback!

Thanks,

Jerome on behalf of the DragonBox team

 

Could you use this to run Occlusion Culling, Lightmapping and Editor plugins on a server..in Editor Mode obviously?

 

Hey! I work with Jerome on u3d!

While I'm not sure what you mean by Occlusion Culling, I have answer for Lightmapping and Editor plugins: u3d has been designed to work for a build server, so it will definitly work for some sort of baking server. The idea nevertheless is that it is meant to make working with Unity easier, not improve his feature set.

While I don't know exactly the project that you have, here is what I would do if I wanted to bake lightmaps on a server using our tool; with no graphical interface, so only from command line:
1. Install our tool: gem install u3d
2. Install Unity with the version that you want: u3d install <the version that you use>
3. Clone, download... a project with some prepared scenes to bake and a script to run Bake. Say MyBakerClass.MyBakingMethod().
NOTE: To create such a script (useful to bake without going through GUI) you will probably want to have a look at https://docs.unity3d.com/ScriptReference/LightmapEditorSettings.html and https://docs.unity3d.com/ScriptReference/Lightmapping.html.
4. Go to your project folder, and launch your script: u3d -- -executeMethod MyBakerClass.MyBakingMethod() -batchmode -quit
This command would run your baking method in batchmode and exit when completing the task. More on that: https://docs.unity3d.com/Manual/CommandLineArguments.html

Similar steps could be taken to run Editor Plugins easily, and while I don't know much about Occlusion Culling, I guess that if that involves Unity's GUI you sure can create a script to bake that, and do the same.

I hope I answered your questions!

 

OC is built in to the engine. So..yeah..it is good to go. Neat toolset.

 

Hey!

I'm trying to use this with Jenkins, but I'm not sure how it's integration should be. I'm running it as a windows command (on jenkins), but it throws me an error, saying that it couldn't recognize 'u3d' as a command. When I launch it throught cmd it works correctly.

Any help here?

Thanks!!

 

Hello @dmontesdeocaSMK. We use u3d with Jenkins ourselves, so maybe we can help!

Note: our servers are Mac, Linux so far.

What you need is to install ruby on your Jenkins server, and run u3d within a Ruby environment. I recommend using RVM to install ruby on your server and use the ruby RVM plugin (https://wiki.jenkins.io/display/JENKINS/RVM+Plugin) to integrate it with jenkins. Then enable the "Run the build in a RVM-managed environment". After that you can use a shell plugin to run your u3d builds.

For example, here's the code I use to install unity across servers.

Code (CSharp):

1.  
2. if [[ ! `which u3d` ]]; then
3.   gem install u3d
4. else
5.   gem update u3d
6. fi
7.  
8. echo "${U3D_INSTALL_ARGS}"
9.  
10. PASS_KEY=U3D_PASSWORD_${NODE_NAME}
11. echo "PASS KEY: ${PASS_KEY}"
12. export U3D_PASSWORD=${!PASS_KEY}
13. u3d install --trace --verbose $U3D_VERSION $U3D_INSTALL_ARGS
14. u3d list

I also set up some environment variables such as

U3D_PASSWORD_name_of_my_slave -> points to the root password for the slave named "name_of_my_slave"

You can use the Build Secret plugin to store your passwords and configure your job to pass them to your build script. Or you use the keychain integration if on mac.

and then pass your u3d configuration like this:

U3D_INSTALL_ARGS=-p Unity,Android,iOS,Linux,Windows,WebGL
U3D_VERSION=2017.1.2f1

(these could be set using the Jenkins Parameter plugin).


As for running builds, we are in fact using fastlane to drive our builds, and u3d is used in one of our lane to build the app before we let other fastlane actions (supply, deliver, hockey ...) distribute the builds to Google, Apple, etc.

I plan on sharing our Fastfile (fastlane configuration file) in the near future.

A pure u3d Ruby Jenkins (https://github.com/jenkinsci/jenkins.rb) could have been possible, but I don't intend to work on that as we use u3d within fastlane ourselves. Also as I've never seen the pure ruby Jenkins plugins solution take off, I am uncertain as this would be useful.

 

@jerome-lacoste Thank you very much!! Let's do it. I'll keep you updated.

 

@dmontesdeocaSMK Did you manage to get something working?

 

u3d 1.0.13 is released with a few interesting features

• u3d/install: allow to specify packages download directory
  
• u3d/available: introduce a central cache
  
• u3d/available: allow to match using regular expression

This allows to do things like:

List the available versions matching 2017.3 and get the result almost instantaneously:



Full release changelog

 

u3d 1.0.15 released

• support downloading packages for another platform (already in 1.0.14)
• support installing the 2018.1 beta, including the new VisualStudio and Facebook modules.

Full release changelog

 

Thanks @jerome-lacoste great work! We are using it and it's a lifesaver.

 

Thanks for sharing @jerome-lacoste, this is super cool!

Are most others using Windows and OS X build machines? Do they require VNC-based GUI activation of licenses?

Would be really smooth if this could be used in Travis CI / Circle CI / GitLab CI to get build-and-upload setups going with the ease of cloud build. Linux instances tend to be quite a bit cheaper (.5x the cost of OS X / Windows VMs), and lots of great CI/CD tools are coming out that only support Docker/Linux-based containerization.

From what I can tell there is a breaking issue with headless license activation among the last year or two's worth of Unity Linux builds (even using the -serial $UNITY_SERIAL -username "$UNITY_USERNAME" -password "$UNITY_PASSWORD" options, it will complain in various forms)

 

@bcjordan we primarily use macs at our office because we target IOS heavily, but I've started experimenting with Linux editor builds on the side. If the licensing wasn't so in the way I would have done that a long time ago. Also we have not only to pay for a license per build server, but to also create a Unity account for each of them. And I also get various weird issues where the servers lose their licenses once in a while. This isn't very satisfying.

This is one project where I played with circleci: https://github.com/lacostej/u3d_ci_example

Given that it takes quite a bit to set things up, I wanted to play with docker instances (and generate all the configs) but didn't get to do it. Happy to discuss this further anytime.

I have seen the editor CLI registration issue reported earlier online, but I can't remember if I had the problem or not.

 

@jerome-lacoste have you seen this?

https://blogs.unity3d.com/es/2018/01/24/streamline-your-workflow-introducing-unity-hub-beta/

It would be great if Unity integrates u3d like tool.

My only concern is that they will do like always, have a team for a short period of time where they finish this project and then move on to another thing and have this Unity hub abandoned.

 

Have you folks been able to deploy Unity apps in docker containers?
NVM... That would require GPU support which is only in alpha so far on Linux and not planned for windows.

 

@N3uRo we added support to the latest package installer for Linux. Sometimes in the future, I will look into the hub to see what it provides.

 

Hi Jerome! I don't suppose you'd be willing to help me?

I currently have U3D setup and working which is awesome and I have it telling unity to make an xcode project in the builds folder. I can't seem to figure out how to make u3d get fastlane to then recognise that new xcode folder and run my lane functions there.

Do I need a whole separate fastlane folder in my builds folder?

 

@Evolht

I should probably share our own fastlane setup. Here's a small extract of our fastfile.

Code (csharp):

1.  
2. def u3d_run(args: nil, unity_version: nil)
3.   Dir.chdir('..') do
4.     path = Dir.pwd
5.     if(File.file? "#{path}/u3d_rules.json")
6.         ENV['U3D_RULES_PATH'] = "#{path}/u3d_rules.json"
7.     else
8.         UI.message 'Using default u3d logging rules'
9.     end
10.     u3d(
11.       run_args: "-logFile '#{path}/u3d.log' -projectpath \"#{path}\" #{args}",
12.       unity_version: unity_version
13.     )
14.   end
15. end
16.  
17. def run_simplebuild(method)
18.   # taken from https://docs.unity3d.com/Manual/CommandLineArguments.html
19.   valid_targets = %w[win32 win64 osx linux linux64 ios android web webstreamed webgl xboxone ps4 psp2 wsaplayer tizen samsungtv]
20.   # taken from SimpleBuild, list of execute methods that match Perform#{method}Build
21.   valid_methods = %w[Android AndroidSplit IOS Linux32 Linux64 LinuxUniversal MacOSX MacOSX64 MacOSXUniversal WebGL Win32 Win64]
22.  
23.   if !valid_methods.include? method
24.     UI.important "run_simplebuild: method '#{method}'' not found in list of known SimpleBuild methods '#{valid_methods.join(',')}'. Build might fail. Check your parameter"
25.   end
26.   active_target = method.downcase
27.  
28.   if !valid_targets.include? active_target
29.     if active_target.index('mac')
30.       active_target = 'osx'
31.     elsif active_target.index('android')
32.       active_target = 'android'
33.     elsif active_target.index('linux')
34.       if active_target == 'linux32'
35.         active_target = 'linux'
36.       else
37.         active_target = 'linux64' # doc unclear about whether LinuxUniversal should use activeTarget linux64
38.       end
39.     else
40.       raise ControlledAbortError.new("run_simplebuild: can not find a valid buildTarget for method '#{method}'")
41.     end
42.   end
43.   UI.message "Mapped method '#{method}' to active target '#{active_target}'"
44.   u3d_run(args: "-quit -batchmode -buildTarget #{active_target} -executeMethod WWTK.SimpleBuild.Perform#{method}Build")
45. end
46. desc "Slack something if slack is enabled (SLACK_URL defined). Options:\n"\
47.    "* @:message: the message\n"\
48.    "* @:success: true by default\n"\
49.    "* @:payload: defaults to BUILD_URL information"
50. lane :do_slack do |options|
51.   if ENV['SLACK_URL'].to_s.strip.length == 0
52.     UI.message("Missing SLACK_URL. No slack")
53.   else
54.     slack(
55.       message: options[:message],
56.       success: !options.include?(:success) || options[:success],
57.       payload: options[:payload] || {
58.         "Build URL" => ENV['BUILD_URL'],
59.         "Product info" => "BUILD_PRODUCT_ID: #{ENV['BUILD_PRODUCT_ID']} V:#{release_version_number}-#{ENV['BUILD_NUMBER']} --env=#{Actions.lane_context[Actions::SharedValues::ENVIRONMENT]}"
60.       }
61.     )
62.   end
63. end
64.  
65. platform :ios do
66.   before_all do
67.     check_env(keys: ['BUILD_PRODUCT_ID', 'BUILD_BUNDLE_ID',
68.                      'XCODE_VERSION', 'HOCKEY_API_TOKEN',
69.                      'FL_UNLOCK_KEYCHAIN_PASSWORD', 'MATCH_PASSWORD'])
70.     xcversion(version: ENV['XCODE_VERSION'])
71.   end
72.  
73.   desc "Build and deploy IOS binary to HockeyApp. Also slack."
74.   lane :all_Hockey do
75.     unity_ios
76.     xcode_dev
77.     hockey_ipa
78.     do_slack(message: "IOS build sent to HockeyApp")
79.   end
80.  
81.   desc "Build Unity for IOS"
82.   lane :unity_ios do
83.     simplebuild(target: "IOS")
84.   end
85.  
86.   desc "Build IOS for dev"
87.   # works from xcode 8 but only tested from xcode 9
88.   lane :xcode_dev do
89.     xcode9(profile_type: "adhoc")
90.   end
91.  
92.   # for development, prov profiles are of type adhoc as per Apple's rules
93.   lane :xcode9 do |options|
94.     profile_type = options[:profile_type]
95.  
96.     # password passed using standard fastlane environment variable
97.     # Note: by default no options passed to set-keychain-settings, meaning it is unlocked without timeout!
98.     unlock_keychain(path: "fastlane")
99.  
100.     match(readonly: true,
101.           keychain_name: "fastlane.keychain",
102.           shallow_clone: true,
103.           type: profile_type)
104.  
105.     # we don't use all of these, but good for documentation
106.     sigh_key="sigh_#{ENV['BUILD_BUNDLE_ID']}_#{profile_type}"
107.     env_pp_uuid="#{sigh_key}"
108.     env_pp_name="#{sigh_key}_profile-name"
109.     env_pp_path="#{sigh_key}_profile-path"
110.     env_pp_team_id="#{sigh_key}_team-id"
111.  
112.     pp_path=ENV[env_pp_path]
113.  
114.     raise "Couldn't find provisioning profile at #{pp_path}" unless File.file? pp_path
115.     UI.message "Using provisioning profile #{pp_path}"
116.  
117.     xcodeproject_path = "target/#{ENV['BUILD_PRODUCT_ID']}_ios/Unity-iPhone.xcodeproj"
118.  
119.     update_project_team(
120.       path: xcodeproject_path,
121.       teamid: ENV[env_pp_team_id]
122.     )
123.  
124.     update_project_provisioning(
125.       xcodeproj: xcodeproject_path,
126.       profile: pp_path,
127.     )
128.  
129.     gym(project: xcodeproject_path,
130.         scheme: "Unity-iPhone",
131.         archive_path: "target/#{ENV['BUILD_PRODUCT_ID']}_ios/build/Unity-iPhone.build",
132.         xcargs: "CODE_SIGN_IDENTITY='iPhone Distribution'",
133.         #export_options: {
134.         #  provisioningProfiles: {
135.         #    ENV['BUILD_BUNDLE_ID'] => ENV[env_pp_name]
136.         #  }
137.         #},
138.         buildlog_path: "#{project_dir}/target/xcode_build_logs",
139.         output_directory: "target/",
140.         output_name: "#{ENV['BUILD_PRODUCT_ID']}.ipa")
141.   end
142.  
143.   desc "Deploy IPA to Hockey"
144.   lane :hockey_ipa do
145.     markdown_type = 1
146.     hockey(
147.       ipa: "target/#{ENV['BUILD_PRODUCT_ID']}.ipa",
148.       api_token: ENV['HOCKEY_API_TOKEN'],
149.       notes: jenkins_dev_change_log,
150.       tags: 'qa',
151.       #notify: '2', # Notify all testers
152.       #status: '2', # Make available for download
153.       #release_type: '2' # 'alpha' release type
154.     )
155.   end

There are several things I would do differently, if I had to start today, in particular our SimpleBuild integration (our own Editor Build class). We haven't had to touch it much in years, so I don't bother. Let me know if that helps, or not!

 

As to the answer to your specific question, I specify the xcodeproject_path to gym in the above code extract

Code (csharp):

1.     xcodeproject_path = "target/#{ENV['BUILD_PRODUCT_ID']}_ios/Unity-iPhone.xcodeproj"
2.  

 

This is excellent! I've managed to get it working and now my team thinks I'm some kind of god! Thank you so much

 

For Linux builds, did you eventually get past the licensing issues?

 

I haven't looked at it. Got a new family member not long ago. I have to test them again. I added this issue to remind me of it: https://github.com/DragonBox/u3d/issues/282

Could you add some details to it?

 

@Face_Mcgace I tested Linux licensing and was able to get it to work as I want.

 

We just released version 1.0.21 with a few interesting new features:

*full Linux modular installation (with support for pkg)
*an interactive console to access the u3d API or modify the code dynamically
*a move command to help dealing with hotfix releases, and a way for you to mark installations you don't want u3d to touch

and of course a couple of bug fixes and minor improvements.

Edit:

The console command in action!

 

u3d is awesome. thanks for sharing!

 

+1

I'm using it every day on my build pipeline using jenkins.

 

I can't seem to build for Windows using the newer 2018 Versions anymore. Unsupported build target even tho I downloaded Windows-Mono. Got an error regarding a package info file tho.