iOS plugin runtime loading using dlopen and dlsym

Hy everyone,

Everyone who worked with plugins known that it a pain that unity editor never releases them (they cant be recompiled).

I managed to load plugin manually using function dlfcn.h of the c standard library on Linux and Mac OSX just fine. Same on Windows using kerner32.dll functions. (see ref 4 for examples)

Then i tried to reproduce the same on iOS but couldn't get it to work. dlsym does not find my function symbole.
Sure i could go back to using DllImport on that platform (and it works), but i wanted to take the challenge on.

I understood that on iOS the project has to be built into one big executable (no dylib, bundles, frameworks) can be included in the .app bundle in order for the program to be able to be published on the apple iphone store.
Anyway their is a nice feature with ldopen and dlsym function:
if you pass a "NULL" filename to dlopen it will return a special handle pointing the the main executable, this handle is defined as RTLD_SELF ((void *) -3) in dlfcn.h. (see ref 7)
Then you can look for function in you own program using dlsym:
void* addr = dlsym( RTLD_SELF, "foo" );

I validated this approche by creating a "Single View App" project on "XCode" with two modifications:

1. I added a cpp file as follow:
   
   Foo.cpp:
   
   Code (CSharp):

   1.  
   2. extern "C" const char* foo () { return "bar"; }
   3.  
2. In ViewController.m i edited the code of viewDidLoad
   
   
   Code (CSharp):

   1.  
   2. - (void)viewDidLoad {
   3.     [super viewDidLoad];
   4.    
   5.     void* addr = dlsym( RTLD_SELF, "foo" );
   6.     if ( addr == NULL )
   7.     {
   8.         fprintf( stderr, "Failed to resolve function \"foo\": \"%s\"", dlerror() );
   9.         exit(0);
   10.     }
   11.    
   12.     const char* (*foo_ptr)(void) = addr;
   13.     fprintf( stdout, "foo_ptr(): \"%s\"", foo_ptr() );
   14. }
   15.  

This worked fine. So their is no reason that the same would not work in C# and Unity!
So I created a new Unity Project targetting iOS platform with two added files:

1. Assets/Plugins/iOS
   
   This folder is recognized by unity as containing native code that should be compiled in the XCode project (Library/Plugins/iOS) and added to the executable (see ref 1,2)
   
   So i added a file name Foo.cpp:
   
   Code (CSharp):

   1.  
   2. #define DLLEXPORT __attribute__((visibility("default"))) // here to ensure symbole is exported, see ref 6.
   3. #include <string>
   4.  
   5. extern "C" DLLEXPORT char* foo ()
   6. {
   7.         const char bar[] = "bar";
   8.         // returned string must be dynamically allocated since Marshal will try to release it
   9.         char* str = new char[4];
   10.         strcpy( str, bar );
   11.         return str;
   12. }
   13.  
2. Then i created a behaviour to test it
   
   
   Code (CSharp):

   1.  
   2. using UnityEngine;
   3. using System;
   4. using System.Runtime.InteropServices;
   5.  
   6. public class Bhv : MonoBehaviour
   7. {
   8.     [DllImport( "__Internal" )]
   9.     private static extern string foo ();
   10.  
   11.     [DllImport( "__Internal" )]
   12.     private static extern IntPtr dlopen ( IntPtr path, int mode );
   13.  
   14.     [DllImport( "__Internal" )]
   15.     private static extern int dlclose ( IntPtr handle );
   16.  
   17.     [DllImport( "__Internal" )]
   18.     private static extern IntPtr dlsym ( IntPtr handle, string symbol );
   19.  
   20.     [DllImport( "__Internal" )]
   21.     private static extern IntPtr dlerror ();
   22.  
   23.     private static IntPtr RTLD_NEXT      = new IntPtr(-1);
   24.     private static IntPtr RTLD_DEFAULT   = new IntPtr(-2);
   25.     private static IntPtr RTLD_SELF      = new IntPtr(-3);
   26.     private static IntPtr RTLD_MAIN_ONLY = new IntPtr(-5);
   27.  
   28.     private static string GetLastError ()
   29.     {
   30.         // Get the char* text
   31.         IntPtr ptr = dlerror();
   32.         // Check if its null (NULL == no error)
   33.         if ( ptr == IntPtr.Zero )
   34.         {
   35.             // no error: empty text
   36.             return string.Empty;
   37.         }
   38.         else
   39.         {
   40.             // convert to managed string
   41.             return Marshal.PtrToStringAnsi( ptr );
   42.         }
   43.     }
   44.  
   45.     delegate string FooDelegate ();
   46.  
   47.     void Start ()
   48.     {
   49.         Debug.LogFormat( "Test 1 foo(): \"{0}\"", foo() );
   50.  
   51.         IntPtr handle = dlopen( IntPtr.Zero, 0 );
   52.         IntPtr addr = dlsym( RTLD_SELF, "foo" );
   53.         if ( addr == IntPtr.Zero )
   54.         {
   55.             Debug.LogErrorFormat( "Failed {0}", GetLastError() );
   56.             // return;
   57.         }
   58.         else
   59.         {
   60.             FooDelegate funfoo = Marshal.GetDelegateForFunctionPointer( addr, typeof(FooDelegate)) as FooDelegate;
   61.             Debug.LogFormat( "Test 2 foo(): \"{0}\"", funfoo() );
   62.         }
   63.         dlclose( handle );
   64.     }
   65.  
   66. }
   If foo is defined as static extern and decorated with DllImport, then both test pass. But if is not defined, the second test fails. This is similar to what is reported in ref 5.
   
   My guess is that the project (behaviours...) are built in Library/libil2cpp.a and Foo.cpp is built later so they are not in the same translation unit, or symboles are not shared/exported between the two. but i'm not sure.

Does anybody knows how to solve this because i have no ideas left ?

References:
[1] https://docs.unity3d.com/Manual/StructureOfXcodeProject.html
[2] https://docs.unity3d.com/Manual/PluginsForIOS.html
[4] https://jacksondunstan.com/articles/3938 (see part 2)
[5] http://sharkman.asuscomm.com/blog/the-curious-case-of-the-function-that-shouldnt-exist/
[6] https://gcc.gnu.org/onlinedocs/gcc-4.0.0/gcc/Function-Attributes.html (see visibility)
[7] https://opensource.apple.com/source/dyld/dyld-195.5/include/dlfcn.h.auto.html

 

If you are doing mostly work in the DLL codebase itself (as compared with some in DLL, some in Unity), the best way I found is to build either a native app to some known location on disk, or else export the XCode Project (for iOS), and then make yourself some scripts (or custom post-build copy commands in XCode) and then work entirely outside of Unity.

The workflow I use on my KurtMaster2D game when I'm actually iterating on the plugins themselves is like this. I basically have a post-build copy step in XCode (either MacOSX target or iOS target) that overwrites the .a or .bundle (or whatever it is called) in a fixed Unity binary, which I generally make changes to fairly infrequently, as the two applications are quite loosely coupled: in my case Unity is just a shell that pumps the native code once a frame to get updates.

In the case of the MacOSX app, just exit and relaunch the app, and it will remount the right .bundle.

In the case of iOS, you would just hit compile again, Xcode would see that the .a file is newer, and off you go.

The nice part of this is if you DO need to do tweaks in Unity, you do the tweaks, rebuild to this interim binary (or project), and rerun. The iteration is fairly smooth, if not quite ideal.

 

This is actually an interesting idea i didn't though of and didn't see in my researchs.
Thank you @Kurt-Dekker

But actually my question is more centered on the behaviour of the dlopen/dlsym function as part of a unity C# project when targetting iOS and resolving symboles that are compiled with the Unity XCode project (Assets/Plugins/iOS)