Incremental builds for IL2CPP

When is the 4.6.9p2 patch coming up?

 

Hi Josh in the recent version of unity patch 4.6.9p1 I have an issue loading a level after I loaded a level already, which means the game crash after loading a level twice in iOS. Is anyone experiencing this issue after loading a level twice in iOS?

Android works fine

 

Hi Josh,

Thanks very much for 4.6.9p2. Is there a way of setting the Append or Replace option from script? We have our own build window which ultimately calls BuildPipeline.BuildPlayer directly, so we won't see the option that's built in to the regular build window.

Thanks again.

 

Hi Josh,

So as I read, this setting can only be set manually through UI before building? Can this also be set through code, like other BuildOptions? As we'd like to integrate it into our building process, which is written in C#.

Thanks.

 

Oh, didn't realise that old setting value gets new functionality. Thanks.

 

I'm having problems with incremental builds. I'm not sure if I'm missing something. Here's a simple case (tested with Unity 5.3.4f1 and Xcode 7.2.1):

1. Create a new iOS project.
   • Create 2 test Monobehaviour files. I just used the default templates created by MonoDevelop.
   • Add the behaviors to a GameObject.
   • Save the scene and add it to the build settings.
2. Build the project in Unity with IL2CPP.
3. Open in Xcode and build.
   • This first build will compile 158 c++ files (156 from the Unity core, plus the 2 custom test behaviors).
4. Go back to Unity, make a change to one of the test files.
5. Build the Unity project, selecting Append.
6. Reopen in Xcode and build.
   • This 2nd build will still compile all 158 c++ files. My expectation was that it would only need to recompile the 1 file that was altered.

I tried the above with and without "Strip Engine Code" enabled in the build settings, with the same results. Even if I do an Append build without making any changes to the Unity project, Xcode will still recompile every file.

Am I missing a step or a build setting to make this work?

 

The only change I made was to add an empty Awake method to the behaviour.

After looking into this more, it appears that incremental builds are working, but maybe my expectations are too high?

I did a diff to check what files were being changed in the Xcode project folder after an Append build. They were as follows (note that TestScript was the name of my behaviour file):

Classes/Native/AssemblyU2DCSharp_TestScript13201784756MethodDeclarations.h
Classes/Native/Bulk_Assembly-CSharp_0.cpp
Classes/Native/Il2CppCodeRegistration.cpp
Classes/Native/Il2CppMethodPointerTable.cpp
Classes/Native/UnityICallRegistration.cpp
Data/Managed/UnityICallRegistration.cpp
Data/Managed/Metadata/global-metadata.dat
Unity-iPhone.xcodeproj/project.pbxproj

I could see the addition of the Awake method, and other changes seemed minimal.

When compiling the incremental build (and paying extra attention), I noticed the compiler skipping over some files faster than before (most notably the first 42 c++ files go by instantly, but most of the others still take about the same amount of time). So, it is definitely a difference from a clean build.

However, the incremental build is only trimming about ~10% from the total compile time for this test project. That was a small enough savings that I didn't notice and assumed it wasn't working. FYI - I have never noticed a great time savings when compiling our actual game projects (which on average take about 20 minutes), so I'm guessing if I measure them I'll find about the same % of savings.

Note: if I go and modify some of the c++ files by hand in Xcode, the build is basically instant. This is more what I was expecting, based on when I code natively in Xcode. I know this probably isn't a relevant test case, but I wanted to verify that Xcode wasn't corrupt or causing some trouble on its own.

Should I be seeing a greater time savings from incremental builds, or does this seem normal?

 

The 10% was just the compile step in Xcode (from when the build starts until linking starts).

Release builds (but not Archive builds) with no modifications to the project settings exported from Unity. I would just hit the play button on the Xcode toolbar.

I ran a few tests related to this with strange results.

1st Test: Unity Incremental Build

1. Start with a fresh Unity test project, as described above.
2. Do a clean build from Unity, into a folder named Test.
3. In Finder, copy the Test folder and name it Clean (for use later).
4. Open the Xcode project inside of Test.
5. Build the project, noting the compile time. This will be our baseline.
6. Do another build, without changing any files. Notice that it finishes almost instantly. (Keep this in mind for later during the 2nd test)
7. Quit Xcode.
8. Open Unity. Add an empty Awake method to one of the behaviours. Do an append build into the Test folder.
9. In Finder, copy the Test folder and name it Append (for use later).
10. Open the Xcode project inside of Test.
11. Build the project, noting the compile time. As an incremental build, the expectation is this should be much faster than baseline, but that isn't the case for me. We'll call this baseline 2.

2nd Test: Manual Incremental Build

1. Do a diff of the Clean and Append folders (I used FileMerge). Note which files changed. We’ll refer to these as the Changed Files.
2. In Finder, copy the Clean folder and name it Test2.
3. (Optional) Delete Xcode’s Derived Data folder to ensure everything is 100% clean.
4. Open the Xcode project inside of Test2.
5. Build the project. No need need to time this one, it should be similar to baseline.
6. Quit Xcode.
7. In Finder, copy all Changed Files from the Append folder into Test2, replacing the Test2 versions. This gives us a manual recreation of the 1st append test folder. You can verify this with FileMerge.
8. Open Test2 in Xcode. Build the project, noting a few things.
   1. The compile time is much faster than baseline 2. On my machine, it only takes 0.1% of the time.
   2. It isn't as fast as the reference build from step 6 in the 1st Test. You can see Xcode's progress pause when processing the changed files.

Result:
Seemingly identical build folders take significantly different times to compile.

Theories:

• My testing methodology is flawed.
• My test environment is corrupt in some way.
• The Unity Append process does something that breaks Xcode’s incremental build system.
  • Maybe it touches files in some way that FileMerge doesn’t consider a change but Xcode does?
  • Maybe there are some hidden files causing trouble that I didn’t take into account?

I could not narrow down why this happened, but I can repeat the results. Hopefully there is something useful here, not just a wild goose chase.

 

Hi Josh!

Could you please let me know what the issue number is for this? And is it in any way related to
"(none) - iOS/IL2CPP: Prevent unnecessary changes to the timestamps of the libil2cpp headers during a build. This allows incremental builds to work correctly in Xcode."
in 5.4.3p2?

Also, two questions regarding incremental builds:
1) We make builds on a Jenkins instance, where the CI builds are optimised for speed and utilise Mono/32bit for iOS. Adding BuildOptions.AcceptExternalModificationsToPlayer to our Unity build script results in the following error:
"CopyStringsFile My.app/HelpshiftLocalizable.strings /Users/Me/Project/Assets/Lib/Helpshift/Plugins/iOS/HSLocalization/zh-Hant.lproj/HelpshiftLocalizable.strings
cd /Users/Me/Project/Build
export PATH=“/path/to/xcode711/7.1.1/Xcode_7.1.1.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/path/to/xcode711/7.1.1/Xcode_7.1.1.app/Contents/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin"
builtin-copyStrings --validate --inputencoding utf-8 --outputencoding binary --outdir /Users/Me/Project/Build/My.app -- /Users/Me/Project/Assets/Lib/Helpshift/Plugins/iOS/HSLocalization/zh-Hant.lproj/HelpshiftLocalizable.strings
error: failed to remove /Users/Me/Project/Build/My.app/HelpshiftLocalizable.strings: “HelpshiftLocalizable.strings” couldn’t be removed."

It works without BuildOptions.AcceptExternalModificationsToPlayer. Is BuildOptions.AcceptExternalModificationsToPlayer not supported with Mono, or is this a bug? We are using Perforce but with the "allwrite" option, so the files should not have write protection on.

Of note is that the other CopyStringsFile steps succeed. The following warning is also generated:
"xcodebuild[22197:5389513] DVTAssertions: Warning in /Library/Caches/com.apple.xbs/Sources/IDEXcode3ProjectSupport/IDEXcode3ProjectSupport-9061/Xcode3Core/LegacyProjects/Frameworks/DevToolsCore/DevToolsCore/BuildSystem/Runtime/PBXTargetBuildContext.mm:739
Details: unexpected successful exit code from cancelled command <C0002:'CpResource Data'4>
Object: <PBXTargetBuildContext: 0x7f9edfd3e260>
Method: -createCommandInvocationRecordFromInvocation:
Thread: <NSThread: 0x7f9edfb94f60>{number = 10, name = (null)}
Please file a bug at http://bugreport.apple.com with this warning message and any useful information you can provide."

2) 5.4 has incremental build support for WebGL. Do we need to use the BuildOptions.AcceptExternalModificationsToPlayer option to take advantage of it, or is it enough to not clear the cache folder under Library? (I have not tested 5.4 yet but want to prepare).

Thanks a lot,
- Tommi

 

Josh, I just found a post by you from August that instructs to use
PlayerSettings.SetPropertyInt("EnableIncrementalBuildSupportForIl2cpp", 1, BuildTargetGroup.iOS);
for enabling incremental builds with IL2CPP for iOS.

I will extend my questions to include this as well. I take that the above is not supported with Mono/iOS, but is it required for incremental WebGL builds in batch mode with Unity 5.4? And is that still a requirement for IL2CPP/iOS?

There's quite a lot of overlapping concepts here. Even when using mono you can append to the Xcode project - is BuildOptions.AcceptExternalModificationsToPlayer the programmatic equivalent to append (when building in batch mode), and does it even save on build times, or is it just for preserving the Xcode project?

We are very dependent on batch mode builds so it is absolutely necessary that incremental building can be done from the command line as well. Also, the documentation for these features is very spread out and it's hard to know what is required at the moment.

Thanks again,
- Tommi

 

Thank you so much for the clarifications, Josh! Greatly appreciated.

(You may want to specify the functionality of BuildOptions.AcceptExternalModificationsToPlayer in regard to IL2CPP builds in the documentation as well, as the relationship is not clear without your clarifications)

- Tommi

 

Hello,

I am writing an iOS app and I am using an OS X virtual machine on Windows to compile the XCode projects files generated from Windows.

So far so good, my setup works lovely, however I cannot benefit from incremental build when compiling the C++ files with XCode when the Unity project was previously build from Windows.

As a result, each minor change requires a full recompilation of the XCode proejct which is a bit lengthy.

When building the project on Windows, I do not get prompted the "append" option like on OS X, instead I only see one popup which says.

"WARNING: all files and folders located in target folder will be deleted by build process."

Is there a work-around to get this feature working on Windows too, if not is it in the roadmap ?

Thanks in advance.

 

Wow thanks for asking this, will keep waiting then.

Also, do you know know if incremental builds would work on Mac OS X when building a project via script with "BuildPipeline.BuildPlayer" too ?

If not, do you know if this feature will ever be in the pipeline someday ?

@JoshPeterson

Cheers and thanks,
Mat.

 

Thanks a lot !!