Showcase Unity as a Library rendered on partial screen with SwiftUI

I tweaked some of the Unity iOS support code to make it work with SwiftUI as a UIViewRepresentable. Will share the process of getting there if people are interested.

 

Yes Please!
Can you put this into a github and share it? I've done a bunch of work to get this to work as a window above a swift UI, but it has it's issues. This seems much more seamless.

 

@Neonlyte maybe also post it into the UAAL forum page. But please show us a github for it!

 

I checked my changes and it's bits and pieces everywhere that I don't think I can just share a git repo and it'll be clear by itself. I'll do a write-up here instead and update the main post.

The anatomy of an iOS app is essential to understand what we need to do. An iOS app would consist of one or more UIWindow, in each window, one or more UIViewController will be presented, and in each view controller, one or more UIView will be presented.

Unity's UaaL initialization for iOS is essentially the same as if it was running as a standalone application, where on initialization, Unity's iOS "trampoline" code would create one of each UI components internally so that it can have exclusive control over those components, rather than reusing any of those from the host application.

What we need to do is to hijack the creation of these UI components, and along the way, we will also need to move around some code to make things visible under Swift, so that we can embed the Unity View into SwiftUI.

1. Prevent Unity creating its own window and view controller. A newly UIWindow will cause it to be placed on top of the host application, and you would have to manually place the SwiftUI app window in front of it again, or all touch events will be blocked by that new window. The UnityViewControllerBase contains code related to auto screen orientation, status bar and home bar, which we want to avoid letting Unity changing those.

The following changes also removes snapshots VC and splash screen usage. My anecdotal observation is that removing the splash screen code actually makes the app launches perceivably faster even for non-UaaL purpose, as it seems that Unity would manually present the splash screen VC, even though iOS would present those automatically.

 

This flag enabled auto-rotation which would try to access Unity's own view controller that would not exist, plus we automatically get auto-rotation when embedded into other view controllers.


This change may be optional as it's part of the auto-rotation code.

 

2. Make Unity View accessible in Swift. By default, Unity does not export the UnityView class, and its header file structure prevents Swift from importing the relevant methods and symbols.


Create a new header file called "UnityView+Private.h", and move the 3 private fields deleted in the previous image into here.


Then, update header includes

 

Finally, export the header file in the framework umbrella header, and configure Xcode project to copy the header into the framework:

 

3. Integrate with SwiftUI in a way that SwiftUI Preview still works.

Code (Swift):

1. import SwiftUI
2.  
3. #if targetEnvironment(simulator)
4. #else
5. import UnityFramework
6. import MachO
7.  
8. private var isUnityStarted: Bool = false
9. func initializeUnityIfNeeded() {
10.     guard !isUnityStarted else {
11.         return
12.     }
13.  
14.     UnityInstance.setDataBundleId("com.unity3d.framework")
15.     UnityInstance.runEmbedded(withArgc: CommandLine.argc, argv: CommandLine.unsafeArgv, appLaunchOpts: [:])
16.  
17.     isUnityStarted = true
18. }
19.  
20. private let UnityBundle = Bundle(path: Bundle.main.privateFrameworksPath! + "/UnityFramework.framework")!
21. let UnityInstance = {
22.     guard UnityBundle.load() else {
23.         fatalError("Unity Bundle failed to load")
24.     }
25.  
26.     let unityFrameworkInstance = UnityBundle.principalClass!.getInstance()!
27.  
28.     if unityFrameworkInstance.appController() == nil {
29.         let machineHeader = UnsafeMutablePointer<MachHeader>.allocate(capacity: 1)
30.         machineHeader.pointee = _mh_execute_header
31.         unityFrameworkInstance.setExecuteHeader(machineHeader)
32.     }
33.  
34.     return unityFrameworkInstance
35. }()
36. #endif
37.  
38. @main
39. struct Partial_View_TestApp: App {
40.     var body: some Scene {
41.         WindowGroup {
42.             ContentView()
43.         }
44.     }
45. }
46.  

Code (Swift):

1.  
2. import SwiftUI
3.  
4. #if targetEnvironment(simulator)
5. // Does not import UnityFramework for simulator
6. #else
7. import UnityFramework
8. #endif
9.  
10. struct ContentView: View {
11.     var body: some View {
12.         VStack {
13.             Text("Hello from Swift UI!")
14.                 .padding()
15.             UnityWrapperView()
16.                 .aspectRatio(16/9, contentMode: .fit)
17.             Button("Render") {
18. #if targetEnvironment(simulator)
19.                 // Does nothing if simulator
20. #else
21.                 UnityInstance.sendMessageToGO(
22.                     withName: "Message Receiver",
23.                     functionName: "Receive",
24.                     message: "C"
25.                 )
26. #endif
27.             }
28.             .padding()
29.         }
30.         .padding()
31.     }
32. }
33.  
34. #if targetEnvironment(simulator)
35. struct UnityWrapperView: View {
36.     var body: some View {
37.         Rectangle()
38.             .foregroundColor(.blue)
39.     }
40. }
41. #else
42. struct UnityWrapperView: UIViewRepresentable {
43.     func makeUIView(context: Context) -> UIView {
44.         initializeUnityIfNeeded()
45.      
46.         let view = UnityInstance.appController().unityView!
47.         view.removeFromSuperview()
48.      
49.         return view
50.     }
51.  
52.     func updateUIView(_ uiView: UIView, context: Context) {
53.     }
54. }
55. #endif
56.  
57. struct ContentView_Previews: PreviewProvider {
58.     static var previews: some View {
59.         ContentView()
60.     }
61. }
62.  

Make sure that the framework is not linked via the build phase.


Linker flag:

 

@Neonlyte ! There's some great info there!
This is a completely different setup than how I have it. I'll give it a spin soon and see if that helps solving some things.
One question though, how does your UnityInstance object / file looks like? I'm using a version of the UnityBridge idea from the other thread. or is UnityInstance at that point imported from the library?
I feel just by reading this that I'm missing something.
Thank you for taking the time to write this up!

 

I've tried replicating this in my own project, but i'm getting some odd error in an unrelated class (lifecycle).
There are also some lines that weren't in my project, so my questions are:
1. Can you please share this as a repo? This will be easier to compare where discrepencies are.
2. Which version of Unity is this based off? I'm using 2021.3

 

This is based off Unity 2022.2.10.

I cannot share this as a repository because of the changes I have made is very fragmented, and since I can't reasonably bundle the IL2CPP sources and Unity library, you won't be able to build it to run anyway. All the screenshots above contained all of my changes from a clean export.

The UnityInstance global variable is an instance of UnityFramework. You get that by calling [UnityFramework getInstance] in Objective-C. The UnityFramework class is the principal class of the UnityFramework.framework. See the first code chunk of this post on Wednesday at 6:26 AM.

 

Ok thanks. Might have to try a clean export out into a new project or something. I got all the changes down yet it won't compile due to some wierd error happening on the LifeCycleListener protocol. This happens after I add the last parts, of the view / view+private. All of a sudden I'm getting some errors in that class.



But, unfortunatly those errors don't make any sense, especially as this is a class that hasn't even been edited, and compiles correctly without the other changes...

@Neonlyte. Thanks again for this write up and your help. hopefully this will benefit other users in the future. I'll post updates if I manage to get this working.

Update : Did a clean XCode 14.3 project, with a clean Unity Project 2022.2.11, getting the same error. Looks like a dead cause atm unfortunately.

 

You may have messed up a header file somewhere which can cause this kind of bogus errors. These changes are indeed not easy to replicate due to all the moving pieces and I was only comfortable doing it because my day job is iOS Development.

 

Yeah we use this in an IOS app ourselves however I'm new to it and never worked with ObjC. What irks me is that it happens with a file that isn't releated and I've double and triple checked. It has to be some odd order of includes that breaking it or something along those lines.

A clean project and build also didn't work and i've gone through this multiple times, commiting each step and seeing where it breaks.

Could be some config thing, xcode issue or god knows what, and at this point, I can't spend anymore time troubleshooting it. Unity really needs to step up and update their codebase at this point so we aren't relying on the community to solve these issues.

In any case, @Neonlyte, thanks for the effort, maybe someone else will have better success and will be able to get it working.

 

It's alive! It is working. @Neonlyte Thank you many times.
One thing is that you missed, you have to add UnityFramework in General settings Frameworks, Libraries, and Embedded Content.
And probably there a bit more things that can be cleaned up before making Framework, I will try and maybe post a github link with changes on empty project

 

Oh. I forgot to mention that I have a small problem, probably it is related to headers as well. When I follow exact steps from your instruction I am getting an error (on the screen)

If I comment that line - everything builds fine.
BUT in Native iOS project where I am trying to use resulted framework I am getting an opposite error in that file that Screenorientation can’t be found. I have to go inside the framework (fortunately it's just a folder basically) and uncomment that line manually.

May be the order of headers or quirks of compiler idk. But for future followers - worth mentioning

 

Hello, I followed along and got stuck on the LifeCycleListener.h warnings identical to @SKArctop, how were you able to get past those?

 

Glad you were able to make it work. I hope my posts have served the necessary inspiration.

One comment on “Frameworks, Libraries, and Embedded Content”. The reason that I did not set it there is because I manually configured the build settings to achieve the same effect. That setting is usually convenience, but in my case it interferes with building SwiftUI in Xcode Preview, since the Unity Xcode project can only link with either Simulator or Device SDK at one time, so if I export the framework with Device SDK, it can’t link with Simulator targets, which is what Xcode Preview is underneath.

 

I am not sure. I just followed the steps ¯\_(ツ)_/¯
The only difference is that I made a universal framework (xcframework) which includes a build for simulator and device.
I made two separate builds and after that, I joined the results via command:

xcodebuild -create-xcframework -framework <deviceFrameworkPath> -framework <simulatorFrameworkPath> -output <xcframeworkPath>