Integration Unity as a library in native iOS app

If you use SwiftUI, I made it work:
https://stackoverflow.com/questions/58893862/swiftui-and-unity/65827912#65827912

It's a gross hack for now.

EDIT:

I created a public example showing how to do a full integration:
https://github.com/DavidPeicho/unity-swiftui-example

 

Your example worked perfectly - it's something that I've been trying to crack for quite some time.

However, when I tried to replicate it using my Unity project and a new xcode project, I encountered the following issues - see screenshot.

I've made sure the header file was public and have cleaned and restarted xcode several times but still won't go away.
Strange thing is that when I right click on 'NativeCallsProtocol' and 'Jump to Definition', it finds it in the header file, and it struggles to see the 'FrameworkLibAPI' interface at build time.

If you could suggest anything else for me to try then that'll be greatly appreciated!

So close!

 

My bad, turns out, I had forgotten to set the 'Objective-C Bridging Header' to the header file itself. Once I set it to '$(PROJECT_DIR)/$(PROJECT_NAME)/NativeCallProxy-Bridging-Header.h' , then a clean and rebuild works.

Thanks for posting up your working project @DavidPeicho, it really has helped me a lot.

 

Has anyone successfully been able to run the various custom bridge/proxy plugins included from these examples in the simulator?

Details --> Here

 

Hi
I used unity as framework in my ios(swift) framework and used thus framework in react native. I added framework's view to my viewcontroller as subview. But I am not getting touch in my native or unity. Did I missed anything. Any help is appreciated
Thanks
Soumya

 

Anyone managed to make this run with Kotlin?

I tried following this: https://medium.com/swlh/integrate-and-embed-unity-with-android-for-kotlin-af0766b17e9c
but the "tutorial" is all over the place really and it results in a crash.

I did manage to have the project running with Swift on iOS tho

 

Hello Guys,
I am trying to communicate between Unity to iOS and vice versa.
Here is my piece of code:
In C#-

Code (CSharp):

1. public class NativeLaodSceneAPI {
2.     [DllImport("__Internal")]
3.     public static extern void LoadScene(string sceneId);
4.     [DllImport("__Internal")]
5.     public static extern void sendUnityStateUpdate(string state);
6. }
7.  
8. void Update()
9.     {
10.         NativeLaodSceneAPI.sendUnityStateUpdate(sceneLoader.loadingStatus);
11.     }

In NativeCallProxy.h-

- (void) onUnityStateChange:(const NSString*)state;

NativeCallProxy.mm(Objective-C++)-

Code (Boo):

1. extern "C" {
2.  
3.     void sendUnityStateUpdate(const char* state)
4.     {
5.         const NSString* str = @(state);
6.         [api onUnityStateChange: str];
7.         NSLog(@"This is Unity State: %@", str);
8.         return [api onUnityStateChange:[NSString stringWithUTF8String:state]];
9.     }

Query:
I got the status from Unity side on Xcode console from

NSLog(@"This is Unity State: %@", str);

Now, I have a label on a UnityViewController and wants to print the status on the statusLabel.
Can someone please help me to do this?

 

You need to register a delegate that will listen to `onUnityStateChange`. The call to:

Code (CSharp):

1. [api onUnityStateChange:[NSString stringWithUTF8String:state]];

calls a delegate that implements the `NativeCallsProtocol` protocol.

Unrelated, but your `return` is useless on the above example.

 

Thanks @DavidPeicho for the solution. I have tried with the NSNotification and it's work fine for me but I will try with your approach as well if it will work.

 

Now, I have one more question here:
On my Unity View have some Button, TextField etc basically we can say UI part. From the button action I am loading the scene from Unity.
Question: Is there any possibility to give the gestures(Pinch, Zoom In, Zoom Out etc etc) from the native iOS(Swift) side?
I have already tried with the gestures on Unity View and it's working only on view. Of course because I add the gestures on Unity View. Therefore, it should be possible if I can access the GameObject from Unity into the Native side.

Please let me know if there is any way / approach.

 

Is there anyone who work on Gestures on Native side? I mean write a code in Swift for gestures for the Unity Game Object.
Looking forward to your reply guys!

 

I'm getting this error:
Type 'UnityEmbeddedSwift' does not conform to protocol 'NativeCallsProtocol'

I think its because I haven't done the first part of the Calling stuff from Unity (C#) to xcode (Swift) from your post, can you expand what you did her? Thanks!

 

It's been so long and I haven't touched iOS since. So I no idea what I did, darn brain can't remember stuff

 

No problem, I followed the example on page 6 that is based off of your answer and have it mostly working, cheers anyhow!

 

puneetmahali You can do that by exposing Unity delegate to iOS, that you call with Swift Gesture. However, I would recommend against it. To me, it makes the most sense to keep as much as possible interactions / rendering on the Unity side.

Use the bridging capabilities to send state and initialize data. Try to do as much as possible in Unity if you can

 

Thanks @DavidPeicho , Sure I will try this and also try to keep into the Unity side.

 

Today I got one more issue:
I have the UnityEmbeddedSwift class and from there I am calling showUnity method as I did it previously on my demo app. But in my new app error says: "image not found"

Xcode Debugger:
Reason: image not found

Fatal error: Unexpectedly found nil while unwrapping an Optional value: file App Name/UnityEmbeddedSwift.swift, line 157

Fatal error: Unexpectedly found nil while unwrapping an Optional value: file App Name/UnityEmbeddedSwift.swift, line 157

The line 157 is:

Code (CSharp):

1. ufw!.setExecuteHeader(machineHeader)

Anyone have an idea on this issue. Seems like have and issue with Unity Framework but don't no solution worked for me.
I have tried: this, this & this.

 

Hi all,
I have the c# code script like:

Code (CSharp):

1. public class NativeAPI
2. {
3.     [DllImport("__Internal")]
4.     public static extern string[] getAllXRCloudScenes();
5.     [DllImport("__Internal")]
6.     public static extern string[] getAllSmartAssetsForXRCloudScene(string sceneId);
7.     [DllImport("__Internal")]
8.     public static extern void loadSmartAsset(string binaryId);
9. }

So, now I want to write the plugins in Objective-c++(.mm). Could anyone let me know how can I call the C# methods into the .mm plugins to get the array strings.

Basically, We almost have the void function like: "public static extern void sendUnityStateUpdate(string state);"
But I have now array of strings and unable to get the array if I tried like this:
In plugin .h file

Code (CSharp):

1. - (void) getAllXRCloudScenes:(NSMutableArray*)getAllScenes;

In plugin .mm file

Code (CSharp):

1. void getAllXRCloudScenes(const NSString* getAllScenes) { return [api getAllXRCloudScenes:[NSMutableArray arrayWithObjects:getAllScenes, nil]]; }

 

I make the void function and after that it works!

 

Hi Guys, Again with new query:
How can I display the Native Alert View on Unity view.
I am trying:

Code (CSharp):

1. var unityView = UnityFramework.getInstance()?.appController()?.rootViewController.view
2. unityView?.addSubview(nativeAlertView)

Also native alert add from this line:

Code (CSharp):

1. // show the nativeAlert
2. self.present(nativeAlert, animated: true, completion: nil)

Unity view doesn't allow for this. Any Idea how can I achieve that.

 

Hi, we have been using Unity as Library in a project that was working on iOS until two weeks ago. We had made no serious changes and now it crashes:

validateRenderPassDescriptor:702: failed assertion `Texture at colorAttachment[0] has usage (0x01) which doesn't specify MTLTextureUsageRenderTarget (0x04)'

Versions:
- Unity 2020.1.13f1
- XCode 12.4, swift to integrate Unity Library to the app.

Packages:
- Wikitude
- ARFoundation
- Firebase
- Unity UI Extensions (Particle System)
- ARFoundation, plane detection.

We have tried:
- Disabling metal validation
- XCode setting: OpenGL ES instead of metal
- Crashes after loading the scene with wikitude camera (no use of ARFoundation in this scene), after a scene where we check ARFoundation support and camera permission on the device.

 

Hey everyone,
im trying to integrate a unity project into a Native iOS App.
The integration worked fine and I can initialize unity on button click, but the unity view isn't showing.
I can see the debug outputs of ARKit in the Console, but the view won't show.

On Button Click im calling InitUnity() and ShowMainView()

Code (CSharp):

1. - (void)initUnity
2. {
3.     if([self unityIsInitialized]) {
4.         NSLog(@"Unity already initialized, Unload Unity first");
5.         return;
6.     }
7.  
8.     [self setUfw: UnityFrameworkLoad()];
9.     // Set UnityFramework target for Unity-iPhone/Data folder to make Data part of a UnityFramework.framework and uncomment call to setDataBundleId// ODR is not supported in this case, ( if you need embedded and ODR you need to copy data )
10.     [[self ufw] setDataBundleId: "com.unity3d.framework"];
11.     [[self ufw] registerFrameworkListener: self];
12.     //[NSClassFromString(@"FrameworkLibAPI") registerAPIforNativeCalls:self];
13.  
14.     [[self ufw] runEmbeddedWithArgc: gArgc argv: gArgv appLaunchOpts: appLaunchOpts];
15.     self.window= [[[self ufw] appController] rootView];
16.  
17. }
18.  
19. - (void)ShowMainView
20. {
21.     if(![self unityIsInitialized]) {
22.         NSLog(@"Unity is not initialized Initialize Unity first");
23.     } else {
24.         [[self ufw] showUnityWindow];
25.      
26.     }
27.  
28. }

Does anyone has a suggestion on how to change the view to the Unity Window?

 

In our project we had to set the window.windowScene of ufw:

Code (JavaScript):

1.         let delegate = UIApplication.shared.connectedScenes.first?.delegate as? SceneDelegate
2.          let currentWindow = delegate?.window
3.          let currentWindowScene = currentWindow?.windowScene
4.  
5.          ufw?.appController()?.window.windowScene = currentWindowScene

 

Hey everyone,

I previously had Unity as a Library integrating into a SwiftUI application, but recently I have had trouble. I keep getting two errors when compiling that there are "undefined symbols for architecture arm64."

Xcode Version: 12.4
Unity Versions: 2020.3.5f1, 2021.1.5f1

I've attached screenshots of the error. Let me know if anyone else has experienced this issue and fixed it.

 

I think you have these two methods into Unity side - C# Script. You need to write those into the Plugins file with extern c. I hope then it should solve the error.

 

Hi,

I have started implementing Unity framework. In the tutorial, seems they are passing string. Is it possible to pass custom objects from Xcode to Unity, and vice versa? If so, please show how. Thanks in advance

 

Not you can not pass any objects to Unity. But you can return/out it to Unity. To read more about marshall.

 

When I try to build Unityframework after updating to Xcode 12.5, I get the following error:

Code (CSharp):

1. StandaloneFileBrowser.mm
2. Cannot initialize a parameter of type 'id<UIDocumentPickerDelegate> _Nullable' with an lvalue of type 'const Class'

I haven't changed the code at all, and until Xcode 12.4 I didn't get such an error.
Does anyone have the same problem?

 

Hi guys,

I am working on a Unity project embedded in a Swift app.

I follow this tutorial (https://medium.com/mop-developers/c...me-embedded-in-a-swiftui-ios-app-1cefb38ff439) and many of your suggestions.

I can start Unity, having communication between Unity and iOS (both side) and unload properly, everything seems to work really find.

Except that we have a freeze when we launch a second time Unity from the iOS app.

We have a screen of the first scene in Unity and it freezes...

Any suggestions?

 

Hi FermerBZH! I'm the author of that article, I mostly based it on all the discussions in this topic, glad you found it useful!

The issue you're having the second time you launch the Unity game from your app is probably due to Unity's memory leak bug. Unity framework is unable to properly release its memory after you call Application.Unload(), which results in your game retaining the state it was in when you first launched the game.

My team has been having these problems for a long time, and we have been in email correspondence with Unity in order to have this bug fixed. We got a response that Unity's R&D has looked into the issues with memory and releasing memory and has determined this is definitely something for them to look into.

I don't think this issue has been fixed on the Unity side, but the workaround you can implement on your side is to load an empty, completely clean-slate scene each time you launch Unity, which will force the game to have a fresh start every time. Or you can reset all the game data when the user unloads the game. The point is that you basically can't expect that Unity will do this cleanup work for you, but you have to do it manually instead.

I hope this solves your problem!

 

@NSWell
Thanks for ur reply.
would u be able to to show basic code how to use marshal in achieve passing as parameter and return type of functions.
I went into the link u have shared, unfortunately i didnt understand it. Im a beginner in unity.

@dinaga
Thanks for the guide.
Could u pls share how are u passing more complex data types instead of string. From unity to ios, or vice versa. Or u r just passing string always?
Thanks in advance.

 

I am always passing a string. If you want to pass more complex data structures as a string, one very convenient way to do that is to wrap your data in Protobuffers (https://developers.google.com/protocol-buffers), and just decode it on the Unity side.

 

Any tips on how to include other libraries (frameworks) in an IOS build bundled in a native app?
Having trouble with a framework in our build not being found by Unity when it tries to startup from the Initialise unity button in the native app.

 

Our application (swift + unity UaaL) no longer works with the fix in 2020.3.20f1 LTS :

UnityAppController.mm was modified (#if => #endif) :

Code (CSharp):

1. #if (PLATFORM_IOS && defined(__IPHONE_13_0)) || (PLATFORM_TVOS && defined(__TVOS_13_0))
2.     if (@available(iOS 13, tvOS 13, *))
3.         _window = [[UIWindow alloc] initWithWindowScene: [self pickStartupWindowScene: application.connectedScenes]];
4.     else
5. #endif
6.     _window = [[UIWindow alloc] initWithFrame: [UIScreen mainScreen].bounds];

It seems that "pickStartupWindowScene" return the wrong window.

Does anyone have the same problem?

 

Hey @NSWell , Did you find a fix to this? Thanks!

 

Yes, I sovled.

Code (CSharp):

1.         if (@available(iOS 13.0, *)) {
2.             if(originWindow.windowScene != nil){
3.                 [[[[self ufw] appController]  window] setWindowScene:originWindow.windowScene];
4.                 [[[[self ufw] appController] window] makeKeyAndVisible];
5.             }
6.         }

originWindow is native app window.
And Unity app has not the window in UAAL model(I think), so we need connect them.

 

We were using unity as a library even before it was officially launched from unity, we used unity 2018.2 in a hacky way to integrate it in the iOS native app but now it's good to know that Unity itself allows us to do this in an official way.

We are still running our old hacky way in our app but after looking into this thread it looks like we should update to this new architecture but before that, I have a few questions here so please someone spend some time to answer it.

Earlier in our hack, we used unity as a static library so that means once the unity is launched it will stay in RAM throughout the app session, so in our case, we were not able to kill/unload unity completely just were able to play with the view controller to hide unity view and we were able to pause unity but unity will be always there in the background in hibernate state.
So in that case lot of memory management issues were appearing which we handled manually and also once the unity is launched it was consuming around 150-200MB RAM which will not be released until we kill the app as it will always stay in memory even though we are not using it.

So now in the UAAL concept are we able to kill/unload unity and relaunch a fresh unity instance again in the same app session?
for example: Opened my native app (unity not launched yet) and clicked on a button to launch unity and in the unity environment, I played a game, and then when I come back to native will the whole unity engine will be completely unloaded from RAM and once again if I tap on the native UI button will it relaunch a fresh unity environment again?

If that is "yes" then I wanted to give it a try, so I see here that so many of you have already tried it so can anyone confirm this?

 

No, it still occupying some memory(100M+)

 

I was wondering if anyone had any suggestions on how to control the size of the Unity View on Android and IOS, for example to make the unity view a small widget within a larger native app view.

Is this possible?

 

When using your project I try to build it and get

Code (CSharp):

1. /Users/####/Library/Developer/Xcode/DerivedData/Unity-Bridge-bdzbttetqckbuqhgzxsfrpaecgjq/Build/Products/Debug-iphoneos/NativeApp.app is not a valid path to an executable file. Please rebuild the project to ensure that all required executables are created. Check your project settings to ensure that a valid executable will be built.
2.  

 

I had one more doubt,
In this architecture(what everyone is talking about here), does unity load as a fresh instance every time you launch it from native?
i.e
The static variables initialized in the previous unity session have to be manually reset again in the next Unity session or does unity automatically resets it?

In my current architecture, the values of the static variables have to be manually cleared while going back to native or else in the next Unity session all my static variables hold the previous unity session values itself even though I reload the scene.
But we don't have this issue in Android because In android we are able to kill unity activity completely once we make the transition from Unity to native so on the relaunch, we initiate a new activity for launching unity again so there these static variables issues don't occur but in iOS, we were not able to kill the unity activity so we were reusing same activity in subsequent launches.
So just wanted to know what happens to static variables in your architecture?

 

@ROBYER1 maybe someone can correct me, but with Unity > 2020 I don't think you can do that anymore. I just run it in fullscreen and my camera takes that into account.

 

Yep we got Unity on IOS running in a subview which can be resized and moved around. I just need to solve the subview not receiving touchscreen input which there should be a hack for.

I'll try to write up how we got Unity running in a subview on Android and IOS sometime if others would find it useful as I found 0 documentation on how to do exactly that.

 

Because the window receiving the events for Unity is the background window. I don't know if it will work when unity is added as a subview, but it should: https://davidpeicho.github.io/blog/unity-swiftui-integration-revisited/

I basically forward the events from the view to the background windows (which then communicate it to Unity).

 

That's the solution our native IOS developer is investigating, as we are not using swift UI we are just doing a regular native IOS app in Xcode trying to do the same thing.

 

In iOS does anyone know how to relaunch unity after calling

Code (CSharp):

1. void    UnityQuitApplication(int exitCode);

basically, I want to start a clean new unity session on subsequent launches from iOS native, is that possible?

 

Hello, guys.
is it possible to embed unity as a subview on iOS?
I managed to render unity view in full screen, but I'd like to know if it's possible.
Thanks in advance.

 

Yes it is! I might have a working sample I can share, I might see if it is something I can share on github sometime.

It was just something I got our IOS native developer to look into and it wasn't too hard for them, just a heads up! But I am not an IOS Native developer myself so how it works is a bit of a mystery.

 

Thank you.
It would be appreciated if you can share sample project. I’m struggling it the whole day.

 

it seems that I managed to render Unity as a subview on iOS, but I think there are still many stuffs to tweak.