How to improve scripts patching for faster development, especially for multiplayer games

I've recently decided to port my multiplayer game from Unreal Engine to Unity. One of the biggest issue in doing that is how slow is to test multiple network clients compared to UE.

Since Unity does not allow to have multiple Editor instances open for the same project, the only option is to build the game and run multiple instances of it. But that takes too much time when you're continuously doing script changes. There is the "Scripts Only Build" option, but that is too slow.

I was starting to regret my decision for leaving Unreal, until this week when I've finally managed to improve iteration time tremendously.

To improve the workflow, we should only patch the scripts for an existing build. We have three options for that, and I'll present them next, alongside with their respective time it takes to have the game open and ready for testing:

1. Scripts only build using the build settings window:
Windows: 28 seconds
Android ("Patch And Run" button): 42 seconds

2. Copy the DLLs for the compiled scripts from Editor into the build (my preferred option):
Windows: 1 second
Android: 4 seconds

3. Manually compiling the scripts and copying the DLLs into the build:
Windows: 6 seconds
Android: 9 seconds

Disclaimer:
- I'm using Mono scripting backend for development builds and patching
- I only tested on Windows and Android
- I'm using Unity 2020.1 Alpha

About Option 1 (Scripts Only Build)

I don't know what's going on when doing a "Scripts Only Build", but it's doing more than just scripts only build. It gets stuck for a good amount of time with a "Build Player" dialog open.

Less than a minute it's not that bad, but while continuously writing network code and testing multiple players, that is way too long, especially when I've been spoiled by UE.

About Option 2 (copy DLLs from Editor)

This is the fastest option and is the one I'm gonna use. But there is one big "gotcha" to it: the compiled scripts will contain editor code (when using #if UNITY_EDITOR preprocessor directive) that might call editor functions that are not available inside standalone builds.

To bypass the issue, I'm putting all the editor code inside Editor assemblies and restricting my usage of #if UNITY_EDITOR to only specific code that is not a problem if it will be executed without the editor.

How to patch:
Get a list of the project assemblies (using CompilationPipeline.GetAssemblies) and copy the compiled DLLs from the project's "Library/ScriptAssemblies" to "<build>/<project name>_Data/Managed".

To patch a build on Android, you'll have to copy the DLLs to "/storage/emulated/0/Android/data/<PackageName>/cache/ScriptOnly/<UnityVersion>/mono/Managed" (see https://docs.unity3d.com/Manual/android-AppPatching.html)

For Android, an extra file is needed for the game build to load the DLLs from different location:
"/storage/emulated/0/Android/data/<PackageName>/cache/ScriptOnly/<UnityVersion>/mono/patch.config".
The file should contain a patch date taken from DateTime.Now.Ticks: patchDate=637137256109191963

(code example at the end of the post)

About Option 3 (compile scripts first)

This option is also a faster alternative to "Scripts Only Build". Compared to option #2, the editor code is removed from DLLs, which is great. The only downside is that the time is dependent on how fast the compilation is. Plus, it's kinda redundant: you modify scripts, switch to the editor, wait for compile, then patch by compiling again.

To avoid the double-compilation, the auto-compile could be disabled for the editor. That I guess will give you a patch time similar to option #2. But of course, this could work only when you do not need your changes to be visible inside the editor (you would play-test the game using the patched builds).

The patching process is similar to option #2, and you can find a code example bellow. The actual compilation is being done with PlayerBuildInterface.CompilePlayerScripts.

Side Note: UI Elements made it really easy and fast to build a custom tool to automate the building and patching process.


Here's a code example on how the patch options could be implemented:

Code (CSharp):

1. public const string UnityPlayerActivity = "com.unity3d.player.UnityPlayerActivity";
2.  
3. public static BuildTarget selectedPlatform => EditorUserBuildSettings.activeBuildTarget;
4. private static BuildTargetGroup selectedPlatformGroup => BuildPipeline.GetBuildTargetGroup(selectedPlatform);
5.  
6. private static readonly string AdbPath = Path.Combine(
7.     EditorApplication.applicationContentsPath,
8.     "PlaybackEngines",
9.     "AndroidPlayer",
10.     "SDK",
11.     "platform-tools",
12.     "adb.exe"
13. );
14.  
15. private static string buildDirectory
16. {
17.     get
18.     {
19.         GameBuilderSettings settings = GameBuilderSettings.Get();
20.  
21.         switch (selectedPlatform)
22.         {
23.             case BuildTarget.StandaloneWindows64:
24.                 return Path.Combine(settings.buildPath, settings.windowsDirectory);
25.  
26.             case BuildTarget.Android:
27.                 return Path.Combine(settings.buildPath, settings.androidDirectory);
28.  
29.             default:
30.                 return "";
31.         }
32.     }
33. }
34.  
35. private static string appPath
36. {
37.     get
38.     {
39.         switch (selectedPlatform)
40.         {
41.             case BuildTarget.StandaloneWindows64:
42.                 return Path.Combine(buildDirectory, $"{Application.productName}.exe");
43.  
44.             case BuildTarget.Android:
45.                 return Path.Combine(buildDirectory, $"{Application.productName}.apk");
46.  
47.             default:
48.                 return "";
49.         }
50.     }
51. }
52.  
53. private static string patchDirectory => Path.Combine(buildDirectory, $"{Application.productName}_Data", "Managed");
54.  
55. private static string patchDirectoryAndroid
56.     => $"/storage/emulated/0/Android/data/{Application.identifier}/cache/ScriptOnly/{Application.unityVersion}/mono";
57.  
58. private static string patchTempDirectory => Path.Combine(buildDirectory, "CompiledScripts");
59.  
60.  
61. public static void Patch(PatchType patchType = PatchType.FromEditor, bool run = false)
62. {
63.     if (patchType == PatchType.FromEditor)
64.     {
65.         PatchFromEditor();
66.     }
67.     else if (patchType == PatchType.Compile)
68.     {
69.         PatchCompile();
70.     }
71.  
72.     Debug.Log($"patchDate={DateTime.Now.Ticks}");
73.  
74.     if (selectedPlatform == BuildTarget.Android)
75.     {
76.         Process process = new Process {
77.             StartInfo = {
78.                 FileName = AdbPath,
79.                 Arguments = $"shell echo patchDate={DateTime.Now.Ticks} > {patchDirectoryAndroid}/patch.config",
80.                 WindowStyle = ProcessWindowStyle.Hidden
81.             }
82.         };
83.  
84.         process.Start();
85.         process.WaitForExit();
86.     }
87.  
88.     if (run)
89.     {
90.         RunBuild();
91.     }
92. }
93.  
94. private static void PatchFromEditor()
95. {
96.     Assembly[] assemblies = CompilationPipeline.GetAssemblies(AssembliesType.PlayerWithoutTestAssemblies);
97.  
98.     foreach (Assembly assembly in assemblies)
99.     {
100.         if (assembly.sourceFiles[0].Contains("com.unity"))
101.         {
102.             continue;
103.         }
104.  
105.         PatchFile(assembly.outputPath);
106.         PatchFile(assembly.outputPath.Replace(".dll", ".pdb"));
107.     }
108. }
109.  
110. private static void PatchCompile()
111. {
112.     ScriptCompilationSettings compileSettings = new ScriptCompilationSettings {
113.         group = selectedPlatformGroup,
114.         target = selectedPlatform,
115.         options = ScriptCompilationOptions.DevelopmentBuild
116.     };
117.  
118.     PlayerBuildInterface.CompilePlayerScripts(compileSettings, patchTempDirectory);
119.  
120.     foreach (string filePath in Directory.GetFiles(patchTempDirectory))
121.     {
122.         if (Path.GetFileName(filePath).StartsWith("Unity", StringComparison.Ordinal))
123.         {
124.             continue;
125.         }
126.  
127.         PatchFile(filePath);
128.     }
129.  
130.     FileUtil.DeleteFileOrDirectory(patchTempDirectory);
131. }
132.  
133. private static void PatchFile(string filePath)
134. {
135.     string fileName = Path.GetFileName(filePath);
136.  
137.     if (fileName == null)
138.     {
139.         return;
140.     }
141.  
142.     if (selectedPlatform == BuildTarget.StandaloneWindows64)
143.     {
144.         FileUtil.ReplaceFile(filePath, Path.Combine(patchDirectory, fileName));
145.     }
146.     else if (selectedPlatform == BuildTarget.Android)
147.     {
148.         CopyFileToAndroidDevice(filePath, $"{patchDirectoryAndroid}/Managed/{fileName}");
149.     }
150. }
151.  
152. private static void CopyFileToAndroidDevice(string src, string dst)
153. {
154.     Process process = new Process {
155.         StartInfo = {
156.             FileName = AdbPath,
157.             Arguments = $"push {src} {dst}",
158.             WindowStyle = ProcessWindowStyle.Hidden
159.         }
160.     };
161.  
162.     process.Start();
163.     process.WaitForExit();
164. }
165.  
166. public static void RunBuild()
167. {
168.     Process process = new Process();
169.  
170.     if (selectedPlatform == BuildTarget.StandaloneWindows64)
171.     {
172.         process.StartInfo.FileName = appPath;
173.     }
174.     else if (selectedPlatform == BuildTarget.Android)
175.     {
176.         process.StartInfo.FileName = AdbPath;
177.         process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
178.  
179.         string closeCmd = $"am force-stop {Application.identifier}";
180.         string openCmd = $"am start {Application.identifier}/{UnityPlayerActivity}";
181.  
182.         process.StartInfo.Arguments = $"shell {closeCmd} && {openCmd}";
183.     }
184.  
185.     process.Start();
186. }

 

Or you could just use SyncToy to clone your project and run 2 instances of Unity.

Make any changes to the main project, hit sync and play on both instances.

 

This is an amazing idea brav, let me give it a try and get back to you on how it goes

 

As convenient as this is, this and tools similar to Parallel Sync all have the problem of running two versions of the editor that both need to update and maintain compiled script assemblies, this increases the RAM and total time taken to recompile scripts. In my project, with one editor open it takes 30s to recompile, with 2 open it take 3 minutes to recompile. That's an issue. It is very convenient, but with the downside of greatly increasing development time.

 

I have tried it, and yes, it works amazing. I think unity really should implement such tools out of the box