Search Unity

IL2CPP - Accessing WinRT Managed Component from an external, non-Unity process

Discussion in 'Windows' started by heffendy, Nov 22, 2019.

  1. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    Hello,

    We're migrating our application from the .NET backend to IL2CPP and ran into a road block.

    Our application employees a two process model where the foreground application is a Unity app but our background process is plain old C++/CX based exe.

    We have modules consumed by both the processes that's written in C# and projected to WinRT as managed WinRT components.

    The Unity app can access these components just fine. However, its unclear what is required for our background process to also consume this managed WinRT component that has now been generated to C++ (I double checked and confirmed that the IL2CppOutputProject contains code for the class).

    Specifically, we're getting a 0x8007007E ('The specified module could not be found.') error when code in the background process attempts to activate an instance of `MyWinRTClass`.

    When using the .NET backend, the managed WinRT types would get registered in the package.appxmanifest/appxmanifest.xml to be hosted by CLRHost.dll:

    Code (CSharp):
    1.  
    2. <Extension Category="windows.activatableClass.inProcessServer">
    3.   <InProcessServer>
    4.     <Path>CLRHost.dll</Path
    5.       <ActivatableClass ActivatableClassId="MyManagedComponentNamespace.MyWinRTClass" ThreadingModel="both"/>
    6.   </InProcessServer>
    7. </Extension>
    8.  

    What would the equivalent of `CLRHost.dll` be in IL2CPP?

    I tried specifying `GameAssembly.dll` but that simply returned 0x800401F9 ('Error in the Dll').

    Attempting to use `UnityPlayer.dll` also didn't work (not that I expect it to) but that returned 0x80040111 ('ClassFactory cannot supply requested class').

    Any ideas or thoughts?
     
  2. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Hi,

    IL2CPP does not support this out of the box, but you should be able to write code to expose generated code to these things. This is one of the things we didn't implement because I thought nobody would need it... OOPS.

    You need to implement these two methods:

    Code (csharp):
    1. #include <activation.h>
    2. #include <hstring.h>
    3. #include <wrl.h>
    4.  
    5. STDAPI DllCanUnloadNow()
    6. {
    7.     return S_FALSE;
    8. }
    9.  
    10. STDAPI DllGetActivationFactory(HSTRING activatableClassId, IActivationFactory** factory)
    11. {
    12.     *factory = Microsoft::WRL::Make<MyActivationFactory>(activatableClassId).Detach();
    13.     return S_OK;
    14. }
    The first one is pretty straightforward: it's supposed to return S_OK if there are no objects allocated, and S_FALSE if there are objects allocated. If you don't support unloading, you don't even have to ever return S_OK and that should be perfectly fine.

    The second one is a bit more tricky. It's supposed to return "IActivationFactory" interface for specific classes by name.

    That means you have to implement a class that implements IActivationFactory. In both cases (when it's called in your game and when it's called in the other process), you need to create a managed IL2CPP object, and return it inside a "COM Callable Wrapper". Let's assume that you don't need to call it in your game through this mechanism since you can call it directly. If that's the case, when this function gets called, IL2CPP is not initialized and you need to initialize it manually. The full picture would be something like this:


    Code (csharp):
    1. #include "il2cpp-api.h"
    2. #include "vm\Assembly.h"
    3. #include "vm\Class.h"
    4. #include "vm\Image.h"
    5. #include "vm\Object.h"
    6. #include "vm\Runtime.h"
    7. #include "vm\CCW.h"
    8. #include "utils\StringUtils.h"
    9. #include "codegen\il2cpp-codegen.h"
    10.  
    11.  
    12. INIT_ONCE g_InitOnce = INIT_ONCE_STATIC_INIT; // Static initialization
    13.  
    14. static BOOL CALLBACK InitializeIL2CPP(PINIT_ONCE InitOnce, PVOID Parameter, PVOID* lpContext)
    15. {
    16.     const wchar_t* kArgs[] = { L"mygame.exe" }; // Make sure this matches your game name. You can probably compute it.
    17.     il2cpp_set_commandline_arguments_utf16(1, kArgs, nullptr);
    18.     il2cpp_init("background job domain");
    19.  
    20.     il2cpp_set_config_utf16(kArgs[0]);
    21. }
    22.  
    23. class MyActivationFactory : public Microsoft::WRL::RuntimeClass<Microsoft::WRL::RuntimeClassFlags<Microsoft::WRL::WinRtClassicComMix>, Microsoft::WRL::FtmBase, IActivationFactory>
    24. {
    25. public:
    26.     MyActivationFactory(HSTRING className)
    27.     {
    28.         m_ClassName.Set(className);
    29.     }
    30.  
    31.     virtual HRESULT __stdcall ActivateInstance(IInspectable** instance) override
    32.     {
    33.         InitOnceExecuteOnce(&g_InitOnce, InitializeIL2CPP, nullptr, nullptr);          // Receives pointer to event object stored in g_InitOnce
    34.      
    35.         unsigned length;
    36.         auto buffer = m_ClassName.GetRawBuffer(&length);
    37.         auto classNameUTF8 = il2cpp::utils::StringUtils::Utf16ToUtf8(buffer, length);
    38.  
    39.         auto lastDot = classNameUTF8.find_last_of('.');
    40.         if (lastDot == std::string::npos)
    41.             return REGDB_E_CLASSNOTREG;
    42.  
    43.         auto namespaze = classNameUTF8.substr(0, lastDot);
    44.         auto name = classNameUTF8.substr(lastDot + 1);
    45.  
    46.         for (auto assembly : *il2cpp::vm::Assembly::GetAllAssemblies())
    47.         {
    48.             auto image = il2cpp::vm::Assembly::GetImage(assembly);
    49.             auto klass = il2cpp::vm::Image::ClassFromName(image, namespaze.c_str(), name.c_str());
    50.             if (klass != nullptr)
    51.             {
    52.                 auto constructor = il2cpp::vm::Class::GetMethodFromName(klass, ".ctor", 0);
    53.                 if (constructor == nullptr)
    54.                     return E_FAIL; // class has no constructor
    55.  
    56.                 auto instantiatedObject = il2cpp::vm::Object::New(klass);
    57.  
    58.                 Il2CppException* exception;
    59.                 il2cpp::vm::Runtime::Invoke(constructor, instantiatedObject, nullptr, &exception);
    60.                 if (exception != nullptr)
    61.                     return E_FAIL; // constructor threw an exception
    62.  
    63.                 *instance = reinterpret_cast<IInspectable*>(il2cpp::vm::CCW::GetOrCreate(instantiatedObject, Il2CppIInspectable::IID));
    64.                 return S_OK;
    65.             }
    66.         }
    67.     }
    68.  
    69. private:
    70.     Microsoft::WRL::Wrappers::HString m_ClassName;
    71. };
    Note I did not test this code (but verified it compiles), so it might have issues. Furthermore, this will only allow instantiating classes which have constructors with no parameters. Let me know if you run into any issues with this approach.

    Can you file a bug so we could fix it in IL2CPP and you wouldn't have to do this in the future?
     
  3. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    Thanks for the super prompt response @Tautvydas-Zilys!

    I think this make sense. Essentially, we need to expose an activation factory entry point & a corresponding activation factory implementation that is able to handle the IL2CPP'ed WinRT types to point the registration manifest to.

    Follow up clarification on where this activation factory implementation would live and how this gets wired up end-to-end:
    1. We would include the above source in the IL2CppOutputProject project (and make sure the DllCanUnloadNow & DllGetActivationFactory gets exported correctly, etc).
    2. And then update the package.appxmanifest/appxmanifest.xml to reference to `GameAssembly.dll` as the path of the in-proc COM server for those WinRT types.
    So the IL2CPP bug here is that it should have done #1 automatically but even if IL2CPP generates this correctly, apps will likely need to manually point the DLL to GameAssembly.dll from CLRHost.dll, correct?

    I'm going to give it a try tomorrow and will report back.

    I'll also get a bug open on this in a bit.
     
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    You can put the .cpp file directly into your Unity project and it will include it into Il2CppOutputProject automatically when you build from Unity.

    And yes, you need to change it from CLRHost.dll to GameAssembly.dll.
     
  5. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    @Tautvydas-Zilys, I'm still struggling to get the the activation factory gets loaded correctly by the winrt runtime.

    Two problems:
    • I tried using IL2CPP_EXPORT/__declspec(dllexport) but because of the __stdcall decoration, the function name is getting decorated (compared to the other IL2CPP exports):

    Code (CSharp):
    1.  
    2. Dump of file GameAssembly.dll
    3. File Type: DLL
    4. ...
    5.           1    0 02C20DA0 CloseZStream = _CloseZStream
    6.           2    1 02C20EB0 CreateZStream = _CreateZStream
    7.           4    2 02C21020 Flush = _Flush
    8.           5    3 02C21050 ReadZStream = _ReadZStream
    9.           6    4 02AC93F0 UnityPalGetLocalTimeZoneData = _UnityPalGetLocalTimeZoneData
    10.           7    5 02AC99B0 UnityPalGetTimeZoneDataForID = _UnityPalGetTimeZoneDataForID
    11.           8    6 02ACB480 UnityPalTimeZoneInfoGetTimeZoneIDs = _UnityPalTimeZoneInfoGetTimeZoneIDs
    12.           9    7 02ACB6E0 UseUnityPalForTimeZoneInformation = _UseUnityPalForTimeZoneInformation
    13.          10    8 02C211B0 WriteZStream = _WriteZStream
    14.           3    9 02C331F0 _DllGetActivationFactory@8 = _DllGetActivationFactory@8
    15.          11    A 02A72580 il2cpp_add_internal_call = _il2cpp_add_internal_call
    16.           ...
    17.  
    Not using STDAPI/__stdcall is easy enough for DllGetActivationFactory but doing this for DllCanUnloadNow cause combaseapi.h to complain about DllCanUnloadNow redefintion with different linkage:

    Code (CSharp):
    1. 1>...\src\Unity\WindowsStoreApp\Il2CppOutputProject\Source\CppPlugins\Il2CppActivationFactoryWorkaround.cpp(92): error C2375: 'DllCanUnloadNow': redefinition; different linkage
    2. 1>C:\Program Files (x86)\Windows Kits\10\Include\10.0.19512.0\um\combaseapi.h(1397): note: see declaration of 'DllCanUnloadNow'
    It looks like the correct way is exposing these via a DEF file but I don't see how that's done in the Il2CppOutputProject.

    It's also not clear to me if DllCanUnloadNow is actually required for this to function here considering this is WinRT (I'm not familiar enough with the nuance here).​
    • Unfortunately, the WinRT runtime, for some reasons, didn't seem to like the MyActivationFactory implementation returned. It simply barfed with:
    Code (CSharp):
    1. Exception thrown: read access violation.
    2. ppActivationFactory was nullptr.
    Callstack just showed this is deep in the runtime code:

    Code (CSharp):
    1. >    combase.dll!CClassCache::GetOrLoadWinRTInprocClass(IWinRTRuntimeClassInfo * pRuntimeClassInfo, IActivationFactory * * ppActivationFactory) Line 4942    C++
    2.      [Inline Frame] combase.dll!CCGetOrLoadWinRTInprocClass(IWinRTRuntimeClassInfo *) Line 8123    C++
    3.      [Inline Frame] combase.dll!WinRTGetActivationFactoryOfInprocClass(IWinRTRuntimeClassInfo * pRuntimeClassInfo, const _GUID &) Line 2339    C++
    4.      combase.dll!_RoGetActivationFactory(HSTRING__ * activatableClassId, unsigned __int64 userContext, const _GUID & iid, void * * factory) Line 931    C++
    5.      combase.dll!RoGetActivationFactory(HSTRING__ * activatableClassId, const _GUID & iid, void * * factory) Line 1030    C++
    6.      [Inline Frame] vccorlib140_app.dll!Windows::Foundation::GetActivationFactory(HSTRING__ *) Line 236    C++
    7.      vccorlib140_app.dll!__getActivationFactoryByHSTRING(HSTRING__ * str, Platform::Guid & riid, void * * ppActivationFactory) Line 70    C++
    8.      vccorlib140_app.dll!FactoryCache::GetFactory(const wchar_t * acid, Platform::Guid & iid, void * * pFactory) Line 343    C++
    9.      vccorlib140_app.dll!GetActivationFactoryByPCWSTR(void * str, Platform::Guid & riid, void * * ppActivationFactory) Line 419    C++
    Stepping through the DllGetActivationFactory method shows that we're correctly instantiating MyActivationFactory and returning it, so its not clear what its unhappy about.
    Here's the full code for reference (I tried out a couple of things but its mostly some as what you originally posted):

    Code (CSharp):
    1. #include <activation.h>
    2. #include <hstring.h>
    3. #include <wrl.h>
    4.  
    5. #include "il2cpp-api.h"
    6. #include "vm\Assembly.h"
    7. #include "vm\Class.h"
    8. #include "vm\Image.h"
    9. #include "vm\Object.h"
    10. #include "vm\Runtime.h"
    11. #include "vm\CCW.h"
    12. #include "utils\StringUtils.h"
    13. #include "codegen\il2cpp-codegen.h"
    14.  
    15. INIT_ONCE g_InitOnce = INIT_ONCE_STATIC_INIT; // Static initialization
    16.  
    17. static BOOL CALLBACK InitializeIL2CPP(PINIT_ONCE InitOnce, PVOID Parameter, PVOID* lpContext)
    18. {
    19.     // Assumption:
    20.     // Getting the ActivationFactory to instantiate a managed WinRT object is only expected to be invoked
    21.     // by the non-Unity process. In those cases, we'll need to make sure that IL2CPP is initialized.
    22.  
    23.     // This needs to match the game name. It can probably be computed but is hardcoded for now.
    24.     const wchar_t* kExecutablePath[] = { L"mygame.exe" };
    25.     il2cpp_set_commandline_arguments_utf16(1, kExecutablePath, nullptr);
    26.     il2cpp_init("Background Process Domain");
    27.     il2cpp_set_config_utf16(kExecutablePath[0]);
    28.  
    29.     return TRUE;
    30. }
    31.  
    32. class MyActivationFactory : public Microsoft::WRL::RuntimeClass<
    33.     ::Microsoft::WRL::RuntimeClassFlags<Microsoft::WRL::WinRtClassicComMix>,
    34.     ::IActivationFactory,
    35.     ::Microsoft::WRL::FtmBase>
    36. {
    37. public:
    38.     HRESULT RuntimeClassInitialize(HSTRING className)
    39.     {
    40.         HRESULT hr = m_ClassName.Set(className);
    41.         return hr;
    42.     }
    43.  
    44.     //
    45.     // Start Boilerplate IUnknown/Inspectable code
    46.     //  This is usually implemented by the InspectableClass macro; but it does not expect ActivationFactory implementations so we're duplicating that here
    47.     //
    48.  
    49.     // IUnknown
    50.     STDMETHOD(QueryInterface)(REFIID riid, void **ppvObject)
    51.     {
    52.         return RuntimeClassT::QueryInterface(riid, ppvObject);
    53.     }
    54.  
    55.     STDMETHOD_(ULONG, Release)()
    56.     {
    57.         return RuntimeClassT::Release();
    58.     }
    59.    
    60.     STDMETHOD_(ULONG, AddRef)()
    61.     {
    62.         return RuntimeClassT::AddRef();
    63.     }
    64.  
    65.     // Inspectable
    66.     STDMETHOD(GetIids)(ULONG *iidCount, IID **iids)
    67.     {
    68.         return RuntimeClassT::GetIids(iidCount, iids);
    69.     }
    70.  
    71.     STDMETHOD(GetRuntimeClassName)(HSTRING* runtimeName)
    72.     {
    73.         *runtimeName = nullptr;
    74.         return ::WindowsCreateString(L"MyActivationFactory", static_cast<UINT32>(::wcslen(L"MyActivationFactory")), runtimeName);
    75.     }
    76.  
    77.     STDMETHOD(GetTrustLevel)(TrustLevel* trustLvl)
    78.     {
    79.         *trustLvl = TrustLevel::BaseTrust;
    80.         return S_OK;
    81.     }
    82.  
    83.     //
    84.     // End Boilerplate IUnknown/Inspectable code
    85.     //
    86.  
    87.     // IActivationFactory
    88.     STDMETHOD(ActivateInstance)(IInspectable** instance)
    89.     {
    90.         InitOnceExecuteOnce(&g_InitOnce, InitializeIL2CPP, nullptr, nullptr);          // Receives pointer to event object stored in g_InitOnce
    91.  
    92.         unsigned length;
    93.         auto buffer = m_ClassName.GetRawBuffer(&length);
    94.         auto classNameUTF8 = il2cpp::utils::StringUtils::Utf16ToUtf8(buffer, length);
    95.  
    96.         auto lastDot = classNameUTF8.find_last_of('.');
    97.         if (lastDot == std::string::npos)
    98.             return REGDB_E_CLASSNOTREG;
    99.  
    100.         auto namespaze = classNameUTF8.substr(0, lastDot);
    101.         auto name = classNameUTF8.substr(lastDot + 1);
    102.  
    103.         for (auto assembly : *il2cpp::vm::Assembly::GetAllAssemblies())
    104.         {
    105.             auto image = il2cpp::vm::Assembly::GetImage(assembly);
    106.             auto klass = il2cpp::vm::Image::ClassFromName(image, namespaze.c_str(), name.c_str());
    107.             if (klass != nullptr)
    108.             {
    109.                 auto constructor = il2cpp::vm::Class::GetMethodFromName(klass, ".ctor", 0);
    110.                 if (constructor == nullptr)
    111.                     return E_FAIL; // class has no constructor
    112.  
    113.                 auto instantiatedObject = il2cpp::vm::Object::New(klass);
    114.  
    115.                 Il2CppException* exception;
    116.                 il2cpp::vm::Runtime::Invoke(constructor, instantiatedObject, nullptr, &exception);
    117.                 if (exception != nullptr)
    118.                     return E_FAIL; // constructor threw an exception
    119.  
    120.                 *instance = reinterpret_cast<IInspectable*>(il2cpp::vm::CCW::GetOrCreate(instantiatedObject, Il2CppIInspectable::IID));
    121.                 return S_OK;
    122.             }
    123.         }
    124.  
    125.         return REGDB_E_CLASSNOTREG;
    126.     }
    127.  
    128. private:
    129.     Microsoft::WRL::Wrappers::HString m_ClassName;
    130. };
    131.  
    132.  
    133. extern "C"
    134. {
    135.     STDAPI DllCanUnloadNow()
    136.     {
    137.         // Always return S_FALSE to prevent the library from getting unloaded
    138.         return S_FALSE;
    139.     }
    140.  
    141.     __declspec(dllexport) HRESULT DllGetActivationFactory(HSTRING activatableClassId, IActivationFactory** ppActivationFactory)
    142.     {
    143.         Microsoft::WRL::ComPtr<IActivationFactory> activationFactory;
    144.         HRESULT hr = Microsoft::WRL::MakeAndInitialize<MyActivationFactory>(&activationFactory, activatableClassId);
    145.         if (FAILED(hr)) return hr;
    146.  
    147.         *ppActivationFactory = activationFactory.Detach();
    148.         return S_OK;
    149.     }
    150. }


    Any insights here?
     
  6. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    I figured out the issue and it comes back to not correctly exporting the DllGetActivationFactory, specifically, the __stdcall calling convention that's part of STDAPI is important.

    Looking at other examples here this is working correctly, the exports uses an alias that COM/WinRT is looking for:
    Code (CSharp):
    1. DllGetActivationFactory = _DllGetActivationFactory@8
    Stripping away the __stdcall convention like I did above made it:
    Code (CSharp):
    1. DllGetActivationFactory = _DllGetActivationFactory
    Because of the difference in calling convention, the stack/return value got corrupted resulting in the error.

    It looks like these really need to get exported via a DEF file (instead of __declspec(dllexport) to get the alias'ing support required. MSDN claims that
    Code (CSharp):
    1. #pragma comment(linker, "/export:alias=decorated_name")
    should also work but it looks like this isn't supported by IL2CPP.exe.

    Any suggestions how to get IL2CPP.exe to generate this correctly?
     
  7. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    I was wrong -- #pragma comment(linker, "/export:alias=decorated_name") actually works if I do a full rebuild of the project.

    So far, I have this:

    Code (CSharp):
    1.     // TODO: This needs to be
    2.     // #pragma comment(linker, "/export:DllCanUnloadNow=DllCanUnloadNow")
    3.     // on ARM
    4.     #pragma comment(linker, "/export:DllCanUnloadNow=_DllCanUnloadNow@0")
    5.     STDAPI DllCanUnloadNow()
    6.     {
    7.         // Always return S_FALSE to prevent the library from getting unloaded
    8.         return S_FALSE;
    9.     }
    10.  
    11.     // TODO: This needs to be
    12.     // #pragma comment(linker, "/export:DllGetActivationFactory=DllGetActivationFactory")
    13.     // on ARM
    14.     #pragma comment(linker, "/export:DllGetActivationFactory=_DllGetActivationFactory@8")
    15.     STDAPI DllGetActivationFactory(HSTRING activatableClassId, IActivationFactory** ppActivationFactory)
    16.     {
    17.         Microsoft::WRL::ComPtr<IActivationFactory> activationFactory;
    18.         HRESULT hr = Microsoft::WRL::MakeAndInitialize<MyActivationFactory>(&activationFactory, activatableClassId);
    19.         if (FAILED(hr)) return hr;
    20.         *ppActivationFactory = activationFactory.Detach();
    21.         return S_OK;
    22.     }
    There are still issues with MyActivationFactory implementation that I'm working through but now at least the WinRT is correctly receiving the IActivationFactory type.

    I'll post another update once I get something working end-to-end.
     
  8. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Oh, I didn't realize you were targeting x86 (calling conventions don't matter on other CPU architectures).

    I was gonna say just put the .def file in source directory and it should start working.. but then I checked and realized that this functionality is only enabled for tests :(. However, the alias seems to work for me:

    Code (csharp):
    1. #if _M_IX86
    2. #pragma comment(linker, "/export:DllGetActivationFactory=_DllGetActivationFactory@8")
    3. #pragma comment(linker, "/export:DllCanUnloadNow=_DllCanUnloadNow@0")
    4. #else
    5. #pragma comment(linker, "/export:DllGetActivationFactory")
    6. #pragma comment(linker, "/export:DllCanUnloadNow")
    7. #endif
    8.  
    9. STDAPI DllCanUnloadNow()
    10. {
    11.     return S_FALSE;
    12. }
    13.  
    14. STDAPI DllGetActivationFactory(HSTRING activatableClassId, IActivationFactory** factory)
    15. {
    16.     *factory = Microsoft::WRL::Make<MyActivationFactory>(activatableClassId).Detach();
    17.     return S_OK;
    18. }

    Regarding this: "// This is usually implemented by the InspectableClass macro; but it does not expect ActivationFactory implementations so we're duplicating that here" - you don't need to do that. RuntimeClass base class automatically implements those for you. The only thing it does not implement is "GetRuntimeClassName", but that is illegal to call on ActivationFactory anyway. Also, I remembered that you can inherit from "Microsoft::WRL::ActivationFactory<>" (just like that, without generic arguments), which saves you a bit of typing with the whole runtime class flags.

    EDIT: Oops, looks like you beat me to it :).
     
    heffendy likes this.
  9. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    @Tautvydas-Zilys, yeah, this is a UWP, HoloLens application and we're iterating over this on the Emulator.

    "I remembered that you can inherit from "Microsoft::WRL::ActivationFactory<>"

    Yup, I tried that too and its essentially equivalent it looks like.


    A follow up question. I hit into another runtime issue where the VCLib runtime is QI'ing the ActivationFactory with the IID of the C# WinRT type IID and it does not support it:

    Code (CSharp):
    1.      GameAssembly.dll!MyActivationFactory::QueryInterface(const _GUID & riid, void * * ppvObject) Line 121    C++
    2.      [Inline Frame] vccorlib140_app.dll!Microsoft::WRL::ComPtr<IUnknown>::CopyTo(const _GUID &) Line 409    C++
    3. >    vccorlib140_app.dll!FactoryCache::GetFactory(const wchar_t * acid, Platform::Guid & iid, void * * pFactory) Line 355    C++
    4.      vccorlib140_app.dll!GetActivationFactoryByPCWSTR(void * str, Platform::Guid & riid, void * * ppActivationFactory) Line 419    C++
    5.  
    From C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Tools\MSVC\14.16.27023\crt\src\vccorlib\vcwinrt.cpp:


    Code (CSharp):
    1.  
    2.     HRESULT GetFactory(LPCWSTR acid, Platform::Guid& iid, void** pFactory)
    3.     {
    4.         ...
    5.         ComPtr<IUnknown> factory;
    6.         Platform::Guid riidUnknown(__uuidof(IUnknown));
    7.         hr = __getActivationFactoryByHSTRING(className, riidUnknown, &factory);
    8.  
    9.         ::WindowsDeleteString(className);
    10.  
    11.         if (FAILED(hr))
    12.         {
    13.             return hr;
    14.         }
    15.  
    16.         if (apartmentCache != nullptr && addToCache)
    17.         {
    18.             apartmentCache->AddFactory(acid, factory.Get());
    19.         }
    20.  
    21.         return factory.CopyTo(iid, pFactory); <--- HERE where iid is the UUID of my managed WinRT type
    22.     }
    Obviously, MyActivationFactory is not aware of this without an explicit ActivatableClassWithFactory mapping. I see that IL2CPP actually has all of these in Il2CppInteropDataTable.

    Is there anyway to get access to that so that we can check to make sure that either matches to the class name (or its statics) to be activated?
     
  10. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    So this happens when C++/CX tries to call a construct that has more than 0 parameters or call a static method. IActivationFactory only supports constructing objects that have parameterless constructors:

    Unfortunately, there isn't really a good way to work around this on your side until we fix it because to construct an object using a non-default constructor you have to QI the factory for an interface that has "Create" that takes the desired parameters and then call it. Of course, it's not impossible, but you'd basically have to walk custom attributes of the class you want to instantiate to find the "factory with create method" type and then get the GUID of it in order to respond to QI. I do not, however, recommend doing this.

    How many classes do you have in your C# .winmd file that get used from C++? Perhaps you can change static methods to be instance methods on objects without parameterless constructors, and change constructors with parameters to a pair of methods (constructor without parameters + Initialize method with those parameters)?

    I understand it's ugly but I think that's the best way forward for you right now.
     
  11. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    Yeah, this particular invocation is a static method. I wasn't aware the workaround doesn't support this. Thanks for the clarification.

    We have 2-3 of these managed WinRT components and fortunately their surface is pretty small so re-writing them might be feasible -- I'll go down that approach.

    I have also filed the bug report referencing this forum thread.
     
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Yeah, I saw the bug report. Thanks for reporting it!
     
  13. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    Hi @Tautvydas-Zilys , I'm able to make some progress but unfortunately, hit into new roadblocks.

    After converting our usage of the IL2CPP'ed from statics to class with default ctor, we're finally making it to the IActivationFactory.ActivateInstance implementation.

    Two issues with the original code that I attempted to address:
    • The original InitializeIL2CPP implementation was missing setting of the data directory. The fix seems trivial enough:
    Code (CSharp):
    1. static BOOL CALLBACK InitializeIL2CPP(PINIT_ONCE InitOnce, PVOID Parameter, PVOID* lpContext)
    2. {
    3.     // Assumption:
    4.     // Getting the ActivationFactory to instantiate a managed WinRT object is only expected to be invoked
    5.     // by the non-Unity process. In those cases, we'll need to make sure that IL2CPP is initialized.
    6.  
    7.     // This needs to match the game name. It can probably be computed but is hardcoded for now.
    8.     const wchar_t* kExecutablePath[] = { L"mygame.exe" };
    9.     il2cpp_set_commandline_arguments_utf16(1, kExecutablePath, nullptr);
    10.     il2cpp_set_data_dir("Data\\il2cpp_data");
    11.     if (!il2cpp_init("Background Process Domain")) { return FALSE; }
    12.     il2cpp_set_config_utf16(kExecutablePath[0]);
    13.  
    14.     return TRUE;
    15. }
    • Attempting to create the CCW for the IL2CPP'ed WinRT type subsequently didn't work because the CCW implementation assumes if its a WinRT type, it'll be a native/platform implementation and would use its corresponding RCW (which of course it does not have):

      Code (CSharp):
      1.  
      2.     Il2CppIUnknown* GarbageCollector::GetOrCreateCCW(Il2CppObject* obj, const Il2CppGuid& iid)
      3.     {
      4.         if (obj == NULL)
      5.             return NULL;
      6.  
      7.         // check for rcw object. COM interface can be extracted from it and there's no need to create ccw
      8.         if (obj->klass->is_import_or_windows_runtime)
      9.         {
      10.             Il2CppIUnknown* result = RCW::QueryInterfaceNoAddRef<true>(static_cast<Il2CppComObject*>(obj), iid);
      11.             result->AddRef();
      12.             return result;
      13.         }
      14.         ...
      15.     }
      I worked around this by simply clearing the is_import_or_windows_runtime flag...

      Code (CSharp):
      1.     // IActivationFactory
      2.     STDMETHOD(ActivateInstance)(IInspectable** instance)
      3.     {
      4.         InitOnceExecuteOnce(&g_InitOnce, InitializeIL2CPP, nullptr, nullptr);          // Receives pointer to event object stored in g_InitOnce
      5.  
      6.         unsigned length;
      7.         auto buffer = m_ClassName.GetRawBuffer(&length);
      8.         auto classNameUTF8 = il2cpp::utils::StringUtils::Utf16ToUtf8(buffer, length);
      9.  
      10.         auto lastDot = classNameUTF8.find_last_of('.');
      11.         if (lastDot == std::string::npos)
      12.             return REGDB_E_CLASSNOTREG;
      13.  
      14.         auto namespaze = classNameUTF8.substr(0, lastDot);
      15.         auto name = classNameUTF8.substr(lastDot + 1);
      16.  
      17.         for (auto assembly : *il2cpp::vm::Assembly::GetAllAssemblies())
      18.         {
      19.             auto image = il2cpp::vm::Assembly::GetImage(assembly);
      20.             auto klass = il2cpp::vm::Image::ClassFromName(image, namespaze.c_str(), name.c_str());
      21.             if (klass != nullptr)
      22.             {
      23.                 auto constructor = il2cpp::vm::Class::GetMethodFromName(klass, ".ctor", 0);
      24.                 if (constructor == nullptr)
      25.                     return E_FAIL; // class has no constructor
      26.  
      27.                 auto instantiatedObject = il2cpp::vm::Object::New(klass);
      28.  
      29.                 Il2CppException* exception;
      30.                 il2cpp::vm::Runtime::Invoke(constructor, instantiatedObject, nullptr, &exception);
      31.                 if (exception != nullptr)
      32.                     return E_FAIL; // constructor threw an exception
      33.  
      34.                 // Il2Cpp's CCW implementation assumes that a WinRT type is *always* a platform type (and not a IL2CPP'ed managed WinRT type)
      35.                 // and will attempt to use its corresponding RCW which won't be applicable.
      36.                 // We'll simply lie by clearing its WinRT flag.
      37.                 // IL2CPP_TODO: Will this have any dire consequences?
      38.                 instantiatedObject->klass->is_import_or_windows_runtime = 0;
      39.  
      40.                 *instance = reinterpret_cast<IInspectable*>(il2cpp::vm::CCW::GetOrCreate(instantiatedObject, Il2CppIInspectable::IID));
      41.                 return S_OK;
      42.             }
      43.         }
      44.  
      45.         return REGDB_E_CLASSNOTREG;
      46.     }

    Explicitly clearly the is_import_or_windows_runtime is sketchy but it does uncover the next issue...the CCW does not support the interface of the interface at the ABI layer, e.g.
    MyManagedComponentNamespace.IMyWinRTClass
    as expected:

    This results in the attempt to ref new the type in C++/CX getting a null pointer.

    Stepping through the code, it looks like the root cause is due to interopData being null:

    Code (CSharp):
    1.     Il2CppIUnknown* CCW::CreateCCW(Il2CppObject* obj)
    2.     {
    3.         // check for ccw create function, which is implemented by objects that implement COM or Windows Runtime interfaces
    4.         const Il2CppInteropData* interopData = obj->klass->interopData;
    5.         if (interopData != NULL)
    6.         {
    7.             const CreateCCWFunc createCcw = interopData->createCCWFunction;
    8.  
    9.             if (createCcw != NULL)
    10.                 return createCcw(obj);
    11.         }
    12.  
    13.         // otherwise create generic ccw object that "only" implements IUnknown, IMarshal, IInspectable, IAgileObject and IManagedObjectHolder interfaces
    14.         void* memory = utils::Memory::Malloc(sizeof(ManagedObject)); <--- We're going down this code path
    15.        if (memory == NULL)
    16.            Exception::RaiseOutOfMemoryException();
    17.        return static_cast<Il2CppIManagedObjectHolder*>(new(memory) ManagedObject(obj));
    18.    }
    Any ideas how to have IL2CPP correctly generate this interopData?

    FWIW, I have explicitly specified the
    windowsruntime="true"
    attribute for my assembly in a link.xml file:


    Code (CSharp):
    1. <?xml version="1.0" encoding="utf-8" ?>
    2. <linker>
    3.   <assembly fullname="System.Core">
    4.     <!-- LightLambda must be included to use Newtonsoft.Json in UWP builds.  This forces it to be included when Unity compiles/builds -->
    5.     <type fullname="System.Linq.Expressions.Interpreter.LightLambda" preserve="all" />
    6.   </assembly>
    7.  
    8.   <!--
    9.     Ensure all code gets generated because some of these are consumed by our background process and
    10.     Unity wouldn't know about those and would accidentally strip them out
    11.    -->
    12.  <assembly fullname="MyManagedAssembly" preserve="all" />
    13.  <assembly fullname="MyManagedWinRTAssembly" windowsruntime="true" preserve="all" />
    14. </linker>
     
  14. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    I just checked and it is indeed like you say - IL2CPP for some reason doesn't think that classes implements interfaces needed by its CCW. I don't know why that is - I'll investigate. Hang tight!
     
  15. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    @Tautvydas-Zilys , do you have any updates?

    Just splunking through the code, I notice there's this giant
    Il2CppInteropData
    table that looks something like this:

    Code (CSharp):
    1. Il2CppInteropData g_Il2CppInteropData[15735] =
    2. {
    3.     { NULL, NULL, NULL, NULL, CreateComCallableWrapperFor_RegistryKey_t29D81BFF6D6710C7AF7557F80446D514B0AB7574, NULL, &RegistryKey_t29D81BFF6D6710C7AF7557F80446D514B0AB7574_0_0_0 } /* Microsoft.Win32.RegistryKey */,
    4. ...
    5.     { NULL, NULL, NULL, NULL, NULL, &IBackgroundAuthenticationManagerPropertyNamesRT_Il2CppWorkaroundClass_tCB801826DA5709F69FC6C330131908F7011D14A6::IID, &IBackgroundAuthenticationManagerPropertyNamesRT_Il2CppWorkaroundClass_tCB801826DA5709F69FC6C330131908F7011D14A6_0_0_0 }, <--- this is the type/interface that I'm trying to activate
    6. ...
    7. }
    8.  
    Perhaps we can leverage the above in some form? Although its strange I don't see an entry for my concrete type; just its interface, so there isn't a
    Il2CppInteropData.createCCWFunction
    entry unfortunately.

    We're also hitting this very early in the app launch sequence and would be open to any short terms workaround, if you have any. We're hoping to be able to get pass this roadblock so that we can start being able to exercise and validate the rest of the application.

    Thanks!
     
  16. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Sorry for the delay. Yeah, it seems like IL2CPP misunderstands managed types in .winmd files and don't make them implement those visible interfaces :(. To be honest, at this point I have to apologize to you for wasting your time with this workaround and tell you to wait for me to fix it properly in IL2CPP.

    Which Unity version are you on? I'll have to backport the fix there.
     
  17. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    Hi @Tautvydas-Zilys,

    I appreciate the support!

    We're currently on Unity LTS, 2018.4.X, so it'll be great eventually gets back-ported to target that.

    However, in the short term, if the fix will go to 2019.2.X first, then we're also open to testing it out on that directly for development purposes.

    Can you share a timeline on when we would be able to test out a fix?
     
  18. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    I'll start working on it this week. I don't know the scope of changes needed for this, unfortunately. So I cannot promise any dates.
     
  19. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Hey,

    sorry for lack of updates from my side. The fix was quite hairy and I wanted to make sure it worked well in all scenarios so it took longer than expected (the holiday season didn't help...). I've fixed this and it's making its way to a release (including to 2018.4 LTS).
     
  20. heffendy

    heffendy

    Joined:
    Aug 3, 2018
    Posts:
    14
    Thanks for the update @Tautvydas-Zilys !

    I assume that the fix is currently not in the latest 2018 LTS (as of 1/30/2020), 2018.4.16f1. I looked through the release notes and don't see this specifically mentioned.

    Can I assume that it'll be the next LTS release, 2018.4.17 or it might be past that?
     
  21. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    Part of the fix has already landed to 2018.4.17 (which fixes CCWs for managed types not implementing needed interfaces). The other part of the fix (generating activation factories) is in a pull request for review for 2018.4 but should land shortly. I can't promise it will hit 2018.4.17 but it's likely it will. I will let you know for sure when it lands.
     
  22. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,680
    The fix landed to 2018.4.18. It missed 2018.4.17 because it had accidentally broken build on Mac, so I had to go and fix it which delayed it.