Integration Unity as a library in native iOS app

One question, I'm a little new to unity, I've already exported my unity project but it does not include the Unityframewrok, is there a series of specific steps to export the project correctly? Greetings!

 

It is true I have a version below, I had that it was the most stable version since the ones you mention is an alpha, but then I will update and follow the tutorial, thanks for the help

 

I experience this issue https://github.com/yurykorzun/unity-react-native-demo with XCode project generated with Unity >2017, I am going to try the library approach, hopefully it solves it

 

Hi bosses ,if my Unity3D project change，Which files I need to replace in the IOS project？or I need intergation again?

 

Thank you for your answer. I mean after integration, what should I do if my Unity needs to add new content? Replace or reintegrate

 

Thank you very much, hope you give more detailed tutorials!

 

I tried to follow this but I can't get it to work.

First of all my unity xcodeproj is missing the NativeProxyCall. In fact it is missing the whole Plugins folder, there is only Libraries and that's it.

Second, I don't understand how I can present a second ViewController as my Unity project.
I was expecting something similar to the Android one, which I got running just fine, where I "just" have to integrate the generated framework and then can push to UnityViewController (Similar to Activity under Android) but that seems not to be the case?

 

I got the issue with unity2019.a7 version.
MTLTextureDescriptor has width of zero.

 

So I am an iOS developer more than I am a Unity developer. This sounds great but can you explain what functionally "Unity as a Library" provides? Maybe explain it with some use cases? At first glance this sounds too good to be true!

For example, can I switch from my natively running iOS Apps view to a Unity running view and then back to my native iOS view again? I think the AR Ads use case was mentioned... Can I have my natively running iOS app switch to a visible Unity Engine C# view and then when the Unity AR Ad is finished can I switch back to my view running Objective C native code programmatically? Also, can those 2 views pass data back and forth to each other?

 

validateTextureDimensions, line 1074: error 'MTLTextureDescriptor has width of zero.'

Code (CSharp):

1. extern "C" void CreateSystemRenderingSurfaceMTL(UnityDisplaySurfaceMTL* surface)
2. extern "C" void CreateRenderingSurfaceMTL(UnityDisplaySurfaceMTL* surface)
3.  
4. so i add code
5. surface->targetW=[UIScreen mainScreen].bounds.size.width*2;
6.  surface->targetH=[UIScreen mainScreen].bounds.size.height*2;
7.  surface->systemW=[UIScreen mainScreen].bounds.size.width*2;
8.  surface->systemH=[UIScreen mainScreen].bounds.size.height*2;
9.  
10. 2==[UIScreen mainScreen].scale
11.  
12. why width and height==0?
13.  
14.  

 

Hi Friends,

I am checking with full project and running it on Xcode 10.1. When I run NativeIosApp scheme. It shows

error
'UnityFramework/UnityFramework.h' file not found

Its not loading UnityFramework class properly. I did all steps of loading UnityFramework.framework .

Can anybody knows how to fix this?

Thanks

 

Hey Everyone , i am trying to do all the needful with this project but facing the issue of
'MTLTextureDescriptor has width of zero.'

Can anyone help me out with this .
Thanks in advance.

 

me to ; look 16#,but，don't know why?

 

Thank you @levaUnity for the quick response , i have a question this library can only be used to initialise Unity into iOS or can we load a particular scene as well , like if i want to load only a scene which is on 3rd index , is this possible with this library ?

 

I have the same error... Can anybody knows how to fix this?

 

Thanks for the detailed explanation but I have the same problem using your full example:

'UnityFramework/UnityFramework.h' file not found

Any ideas?

Update: OK, seems to be my fault. Your example is running fine on a device but not in simulator.

 

How about integrating Vuforia AR framework into that workflow? What additional steps would I have to take?

Update: I managed it to embed easyAR without any problems, camera view und target tracking, but no success with Vuforia so far.

 

As you can use SendMessage to public receivers methods it should not be a problem to start your Unity project in a loading scene, wait for that call with a parameter and load the scene you like after Unity was loaded successfully.

 

This is exactly what you can do. Have a look at the full example and you'll find out, that you start a native written app than load and switch to Unity and go back to your native entry point. You can pass messages with params from native to Unity but I don't know if that is possible from Unity to native (but I'll find out).

 

Of course you can call native functions with params from Unity. Unless something else is meant/desired.

 

Some questions @PavelLU
1. What was behind the decision to change the usual target Unity-iPhone to use UnityFramework?
2. Why not have Data be in UnityFramework's PBXResourcesBuildPhase by default? It would pretty much remove the step 6. Make Data folder to be part of the UnityFramework thus making it easier for users. Only because it's mentioned, I'm guessing to allow On Demand Resources for Data, but that seems like an obscure use case.
3. Regarding bundle and path stuff, why not automate it? I filed bug 1169886 for this.

 

Once the unity view is started by
[[self ufw] runEmbeddedWithArgc:

how can we navigate back to the previous ViewController?

 

Great post! Looks like using the ARFoundation and AR Kit XR plugins into the sample project linked above) generates what seems to be IL2CPP related link errors in XCode when building.

When the plugins aren't installed, the sample project seems to work fine on the iOS device .

Versions being used:

Code (CSharp):

1. Unity: 2019.3.0a2
2. ARFoundation: 2.0.2 (also tried 1.1.0 and 1.0.0)
3. ARKit XR plugin: 2.0.1 (also tried 1.0.0)
4. XCode: 10.1
5. iPhone 8 running iOS 12.2

Code (CSharp):

1. Undefined symbols for architecture arm64:
2.   "_FrameReceivedEventArgs_t4637B6D2FC28197602B18C1815C4A778645479DD_marshal_pinvoke_cleanup", referenced from:
3.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
4.   "_GestureManipulationEvent_t80F0163CC6437F2E1BC5B773E9599AF499D08E4F_marshal_pinvoke", referenced from:
5.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
6.   "_GestureManipulationEvent_t80F0163CC6437F2E1BC5B773E9599AF499D08E4F_marshal_pinvoke_cleanup", referenced from:
7.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
8.   "_GestureNavigationEvent_t3718B4FDF2C71A07DFAB23CE2E27D1D022417866_marshal_pinvoke", referenced from:
9.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
10.   "_GestureRecognitionEvent_tBB5EFD6FF7697E5F8FDFB64BA9A51003282AEE68_marshal_pinvoke", referenced from:
11.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
12.   "_GestureRecognitionEvent_tBB5EFD6FF7697E5F8FDFB64BA9A51003282AEE68_marshal_pinvoke_back", referenced from:
13.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
14.   "_GestureTappedEvent_t793E61A597E40572998004754809BBF49D738535_marshal_pinvoke_back", referenced from:
15.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
16.   "_GestureTappedEvent_t793E61A597E40572998004754809BBF49D738535_marshal_pinvoke_cleanup", referenced from:
17.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
18.   "_MeshGenerationResult_t24F21E71F8F697D7D216BA4F3F064FB5434E6056_marshal_pinvoke", referenced from:
19.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
20.   "_MeshGenerationResult_t24F21E71F8F697D7D216BA4F3F064FB5434E6056_marshal_pinvoke_back", referenced from:
21.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
22.   "_PlaneAddedEventArgs_t06BF8697BA4D8CD3A8C9AB8DF51F8D01D2910002_marshal_pinvoke_back", referenced from:
23.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
24.   "_PlaneRemovedEventArgs_t21E9C5879A8317E5F72263ED2235116F609E4C6A_marshal_pinvoke_cleanup", referenced from:
25.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
26.   "_PlaneUpdatedEventArgs_tD63FB1655000C0BC238033545144C1FD81CED133_marshal_pinvoke", referenced from:
27.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
28.   "_PointCloudUpdatedEventArgs_tE7E1E32A6042806B927B110250C0D4FE755C6B07_marshal_pinvoke", referenced from:
29.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
30.   "_PointCloudUpdatedEventArgs_tE7E1E32A6042806B927B110250C0D4FE755C6B07_marshal_pinvoke_back", referenced from:
31.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
32.   "_SessionTrackingStateChangedEventArgs_tE4B00077E5AAE143593A0BB3FA2C57237525E7BA_marshal_pinvoke", referenced from:
33.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
34.   "_XRRenderPass_tEFEBF289C538809498F1828580181B42CE0FD7C8_marshal_pinvoke_back", referenced from:
35.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
36.   "_XRRenderPass_tEFEBF289C538809498F1828580181B42CE0FD7C8_marshal_pinvoke_cleanup", referenced from:
37.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
38.   "_IntegratedSubsystemDescriptor_tCC675C388E8A720B9467FCE039B00E7BE7693812_marshal_pinvoke", referenced from:
39.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
40.   "_XRCameraSubsystemCinfo_t2DEC704C4D0CB1EC17BB8521B3358EE08C878BCC_marshal_pinvoke", referenced from:
41.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
42.   "_XRCameraSubsystemCinfo_t2DEC704C4D0CB1EC17BB8521B3358EE08C878BCC_marshal_pinvoke_back", referenced from:
43.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
44.   "_PlaneRemovedEventArgs_t21E9C5879A8317E5F72263ED2235116F609E4C6A_marshal_pinvoke", referenced from:
45.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
46.   "_XRCameraSubsystemCinfo_t2DEC704C4D0CB1EC17BB8521B3358EE08C878BCC_marshal_pinvoke_cleanup", referenced from:
47.       _g_Il2CppInteropData in Il2CppInteropDataTable.o
48.  

I initially posted my question here , but thought it would be more relevant to post it here. Appreciate any help. Thank you for your work!

 

Not sure if someone mentioned that already but there is a typo in your generated iOS Framework.
The method UnityFramework.unloadApplication is named unloadApplicaion, it is missing the t.

 

It is for Objective-C only.How do we get it for Swift?

 

I got to step 5 but when I look at the Unity-IPhone folders in Xcode and go to libraries/plugins/ios/ there is no file called NativeCallProxy.h - is there some place I need to get this from? I only see some unity files here but nothing called nativecallproxy.h

Here is a photo of what I see:

any help on what part I am missing here would be great!

 

My error seems similar to this - but I dont think it is exactly the same.

My error is:
Thread 1: signal SIGABRT

and is with this line:
surface->drawableProxyRT = [surface->device newTextureWithDescriptor: txDesc];

This is the entire block of code it is in:

@Synchronized(surface->layer)

{

OSAtomicCompareAndSwap32Barrier(0, 0, &surface->bufferCompleted);

OSAtomicCompareAndSwap32Barrier(0, 0, &surface->bufferSwap);



for (int i = 0; i < kUnityNumOffscreenSurfaces; i++)

{

// Allocating a proxy texture is cheap until it's being rendered to and the GPU driver does allocation

surface->drawableProxyRT = [surface->device newTextureWithDescriptor: txDesc];

}

}


in the metalhelper.mm file.

I keep getting this error when trying to make any changes to the unity project. The error has happened a few times in different unity tests and it is always the same error.
Any help would be great.

Here

 

I have begun building my own way of communication between the unity and native app - but I wanted to know is it possible to create buttons for the view controller that only appear in the end of the unity game - ie in the last scene of your game?

 

The best would be if the app could be displayed and interacted from Firemonkey (Delphi) on a Canvas :...

 

How to do it for swift

 

hi, Application.Unload() doesnt work for ios and when we switch to ios view controller after loading unity scene, how does memory be released for the framework , is their a workaround for unloading unity framework and loading again when required to handle memory issues ?

 

unity_framework.unloadApplication(true) got crash in UnityRepaint().
And unity_framework.unloadApplication(false) got crash when I try to relaunch the game.

 

Crashing when unloading through unity framework.
Trying to call unityframework UnloadApplication(tried both true and false). And getting error code:
("GraspFinalTest" is the name of the iOS application)

DEBUG LOG: unloading unity from NativeCallsProxy.h. Message: red



Setting up 1 worker threads for Enlighten.

  Thread -> id: 16fc1f000 -> priority: 1

2019-08-20 10:30:02.727453+0200 GraspFinalTest[328:35902] Task <967CA1F7-14E2-47F1-A85C-E792FB21D7DD>.<1> finished with error - code: -999

2019-08-20 10:30:02.727966+0200 GraspFinalTest[328:35971] Task <967CA1F7-14E2-47F1-A85C-E792FB21D7DD>.<1> load failed with error Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=https://config.uca.cloud.unity3d.com/, NSErrorFailingURLKey=https://config.uca.cloud.unity3d.com/, _NSURLErrorRelatedURLSessionTaskErrorKey=(

    "LocalUploadTask <967CA1F7-14E2-47F1-A85C-E792FB21D7DD>.<1>"

), _NSURLErrorFailingURLSessionTaskErrorKey=LocalUploadTask <967CA1F7-14E2-47F1-A85C-E792FB21D7DD>.<1>, NSLocalizedDescription=cancelled} [-999]

2019-08-20 10:30:02.731281+0200 GraspFinalTest[328:35970] [BoringSSL] nw_protocol_boringssl_get_output_frames(1301) [C1.1:2][0x14dec3b30] get output frames failed, state 8196

2019-08-20 10:30:02.731857+0200 GraspFinalTest[328:35970] [BoringSSL] nw_protocol_boringssl_get_output_frames(1301) [C1.1:2][0x14dec3b30] get output frames failed, state 8196

2019-08-20 10:30:02.733650+0200 GraspFinalTest[328:35970] TIC Read Status [1:0x0]: 1:57

2019-08-20 10:30:02.734007+0200 GraspFinalTest[328:35970] TIC Read Status [1:0x0]: 1:57

UnityFramework was compiled with optimization - stepping may behave oddly; variables may not be available.

(lldb)

EDIT: Also, I am calling the unloadApplicaiton function from swift. But shouldn't be a problem with that.

 

Just to confirm, is the expected behaviour that this will _only_ work on a device and not in simulator? Is there any way of making it work with simulator? Thank you.

 

How I did Swift integration

EDIT: Updated solution can be found here: https://forum.unity.com/threads/int...-in-native-ios-app.685219/page-2#post-5038628


Ok, so I think I got a swift integration working (it's working on my end at least ). Here you can init and show embedded unity. And send messages to C# scripts. (unloading unity doesn't work yet, and I am too inexperienced as to understand why).

Please understand that I have never ever touched iOS development, nor any other language than Java and C#. So I am not sure if this is a good way to do it. Use at your own risk

How to use it
Create a swift project and workspace:
Follow the guide in the original post as normal. But instead, make a swift project. And import that swift project into the workspace. Remember to follow the steps with adding the unity framework to the xcode project, and setting data folder to be a part of Unity Framework.

Import the UnityEmbeddedSwift.swift file
Just drag and drop the file into the native swift iOS xcode project.
Warning: I had issues where the xcode project didn't find the UnityFramework. I solved it by building the UnityFramework first, then clean the project (Command + shift + K).

Set launch options:
In my AppDelegate.swift file, under the "func application" method, I call UnityEmbeddedSwift.setLaunchOptions(launchOptions)

class AppDelegate: UIResponder, UIApplicationDelegate {

    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {

        // Override point for customization after application launch.

        UnityEmbeddedSwift.setLaunchOptions(launchOptions)

        return true

    }

    (...)

}

Then I created some buttons that calls the different methods to launch Unity and send message to it:
This is done in my ViewController

@IBAction func initUnityBtn(_ sender: UIButton) {

   let instance : UnityEmbeddedSwift = UnityEmbeddedSwift()

   instance.initUnity()

}



//Setting unity message, this is called when unity is initialized or shown.

//You can also call sendUnityMessage() to send it right away (unity needs to be initialized first)

@IBAction func setUnityMsgBtn(_ sender: UIButton) {

   UnityEmbeddedSwift.setUnityMessage("Some GameObject name", methodName: "SomeCSharpMethodName", message: "string parameter")

}

Here is my Swift integration file:

Code (Boo):

1. //
2. //  Created by Simon Tysland on 19/08/2019.
3. //  Copyright © 2019 Simon Tysland. All rights reserved.
4. //
5.  
6. import Foundation
7. import UnityFramework
8.  
9. class UnityEmbeddedSwift: UIResponder, UIApplicationDelegate, UnityFrameworkListener {
10.  
11.     private var ufw : UnityFramework!
12.     private static var launchOptions : [UIApplication.LaunchOptionsKey: Any]?
13.     private static var unityMessageObjectName : String = "default value"
14.     private static var unityMessageMethodName : String = "default value"
15.     private static var unityMessageBody : String = "default value"
16.     private static var unityMessagesQueued : Bool = false
17.  
18.     func unityIsInitialized() -> Bool {
19.         return ufw != nil && (ufw.appController() != nil)
20.     }
21.  
22.     static func setLaunchOptions(_ launchOpts : [UIApplication.LaunchOptionsKey: Any]?){
23.         UnityEmbeddedSwift.launchOptions = launchOpts
24.     }
25.  
26.     static func setUnityMessage(_ objectName : String, methodName : String, message : String) {
27.         UnityEmbeddedSwift.unityMessageObjectName = objectName
28.         UnityEmbeddedSwift.unityMessageMethodName = methodName
29.         UnityEmbeddedSwift.unityMessageBody = message
30.         UnityEmbeddedSwift.unityMessagesQueued = true
31.     }
32.  
33.     func sendUnityMessage() {
34.         if(UnityEmbeddedSwift.unityMessagesQueued && unityIsInitialized())
35.         {
36.             ufw.sendMessageToGO(withName: UnityEmbeddedSwift.unityMessageObjectName, functionName: UnityEmbeddedSwift.unityMessageMethodName, message: UnityEmbeddedSwift.unityMessageBody)
37.             UnityEmbeddedSwift.unityMessagesQueued = false
38.         }
39.     }
40.  
41.     func showMainView() {
42.         if unityIsInitialized() {
43.             ufw.showUnityWindow()
44.             sendUnityMessage()
45.         }
46.     }
47.  
48.     func initUnity() {
49.         if unityIsInitialized() {
50.             showMainView()
51.             return
52.         }
53.  
54.         self.ufw = UnityFrameworkLoad()!
55.         ufw.setDataBundleId("com.unity3d.framework")
56.         ufw.register(self)
57.  
58.         ufw.runEmbedded(withArgc: CommandLine.argc, argv: CommandLine.unsafeArgv, appLaunchOpts: UnityEmbeddedSwift.launchOptions)
59.  
60.         sendUnityMessage()
61.     }
62.  
63.     func unloadUnity() {
64.         if unityIsInitialized() {
65.             ufw.unloadApplication(true)
66.             ufw = nil
67.         }
68.     }
69.  
70.     func UnityFrameworkLoad() -> UnityFramework? {
71.         let bundlePath: String = Bundle.main.bundlePath + "/Frameworks/UnityFramework.framework"
72.  
73.         let bundle = Bundle(path: bundlePath )
74.         if bundle?.isLoaded == false {
75.             bundle?.load()
76.         }
77.  
78.         let ufw = bundle?.principalClass?.getInstance()
79.         if ufw?.appController() == nil {
80.             // unity is not initialized
81.             //            ufw?.executeHeader = &mh_execute_header
82.      
83.             let machineHeader = UnsafeMutablePointer<MachHeader>.allocate(capacity: 1)
84.             machineHeader.pointee = _mh_execute_header
85.      
86.             ufw!.setExecuteHeader(machineHeader)
87.         }
88.         return ufw
89.     }
90. }
91.  

So this means that you do not need to use the mainViewController.mm file provided in the original post (nor a customized Obj-C + header file for that matter), if you are using swift.

Hope you guys find this useful.