Search Unity

UWP assemblies not working when using IL2CPP but work on .NET scripting backend

Discussion in 'Windows' started by jpaiva-msft, May 28, 2018.

  1. jpaiva-msft

    jpaiva-msft

    Joined:
    Oct 17, 2017
    Posts:
    7
    I am trying to port a project to Unity 2018, using IL2CPP since .NET is being deprecated.

    In this project I use MSAL which works perfectly fine with .NET scripting backend, however when using IL2CPP the UWP app crashes because of improper IL2CPP conversion.

    I found IL2CPP generated code tries to load DLLs because some of the P/Invokes required by MSAL are not found. Examples are: kernel32.dll!GetNativeSystemInfo and advapi32.dll!RegOpenKeyEx. Worth mentioning that MSAL internally takes a dependency on mscorlib.dll.

    Eventually the code calls to LoadLibrary and it fails because UWP app does not have permissions to load libraries outside of app container boundaries.

    Is this a bug or just an unsupported scenario in the current release?

    I've found this problem with other assemblies too. If .NET is being deprecated, this seems a critical issue to fix.
     
  2. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Hi, is this a rumor or is this stated somewhere official?
    Vungle currently doesn't support IL2CPP and that maybe could be a reason to convince them ...
     
  3. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,040
    This sounds like a bug in IL2CPP. The .NET class libraries should not p/invoke into Win32 APIs on UWP. Can you submit a bug report?
     
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    This is a bug. As Josh mentioned, please report a bug.

    It's not a rumor. If you open 2018.2 beta and set your scripting backend to .NET, you'll see the deprecation warning.
     
  5. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Hi,
    thanks for pointing that out! We cannot use 2018 yet due to some extensions (which are using defines such as 5_6 or 2017, but not 2018).
     
  6. StephenHodgson-Valorem

    StephenHodgson-Valorem

    Joined:
    Mar 8, 2017
    Posts:
    148
    I've seen this issue too, where the same method that works with the .NET Native Backend doesn't in IL2CPP and throws unauthorized access exceptions.
     
  7. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    ^ we'd like bug reports on all those cases. IL2CPP shouldn't have any less functionality compared to .NET where possible.
     
  8. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Are you also talking about third party sdks like Vungle? Or is it their responsibility to get it running?
     
  9. jpaiva-msft

    jpaiva-msft

    Joined:
    Oct 17, 2017
    Posts:
    7
    I created a bug report for this issue last week: 1043171
     
  10. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    Thanks, we'll take a look.
     
  11. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    @Tautvydas-Zilys I created a report regarding the Vungle issue: case 1045847.

    Please note that when testing, it is better to use Unity 2017, as I do not think Vungle already works with Unity 2018: They use defines like UNITY_5_6 or UNITY_2017, but havent seen any UNITY_2018 ...
     
  12. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    I took a quick look at this issue, and it seems there are multiple issues with it:

    1. Microsoft.Identity.Client.dll (which is part of the project you submitted) P/Invokes directly into kernel32.dll to call GetNativeSystemInfo. The fact that this works on .NET scripting backend is a coincidence, and would fail WACK anyway as kernel32.dll exists only on PC and not other UWP devices. This needs to be fixed within that DLL (it's nothing IL2CPP can do here). If you are able to modify this DLL, I suggest using "__Internal" instead of "kernel32.dll", which will resolve this API at link time.

    2. There are failures P/Invoking into ETW logging functions down the road from within our class libraries. This is on us to fix.
     
  13. jpaiva-msft

    jpaiva-msft

    Joined:
    Oct 17, 2017
    Posts:
    7
    According to the documentation, GetNativeSystemInfo should be available in UWP. In fact, while I was debugging this issue, I changed the IL2CPP generated code to link with GetNativeSystemInfo and I was able to fix that crash. Shouldn't this issue be fixed in IL2CPP?

    The next problem I encountered was with advapi32.dll!RegOpenKeyEx. This I was not able to fix because this API is not available for UWPs. Did you also see an issue with this API? Do you have a suggestion on how to fix this one?

    Thanks for your investigation Tautvydas-Zilys!
     
  14. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    Well, first of all, while GetNativeSystemInfo is available on UWP, it's not in kernel32.dll. Secondly, UWP apps are not allowed to dynamically lookup functions from system libraries, so they must be bound at link time. We have that mechanism, and it's using "[DllImport("__Internal")]" syntax.

    TimeZone APIs do call into the registry, but those exceptions are caught and a fallback code path is taken. There's no harm in this.
     
  15. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    Thanks for the report! I've identified the issue to be in IL2CPP. We will fix it.
     
  16. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Great news, thanks!
     
  17. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    @Tautvydas-Zilys Do you know when and if this fix will be in 2017 LTS? We recently integrated AssetBundles with our 5.6 project, and are running into some memory problems which didn't occur with 2017 IL2CPP. Thanks!
     
  18. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    Several weeks probably. Unfortunately backporting process is lengthy as we have to make sure nothing breaks when you do it.
     
  19. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    With backporting you mean into the 2017 branch? Will the fix be sooner in 2018? Or asked differently: How can I get my hands on a fix the earliest way? I am also willing to test a patch release ...
    Thanks!
     
  20. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    The fixes are being backported simultaneously to both 2018.1 and 2017.4. Unfortunately it's hard to tell which release it will reach first due to the fact that they aren't released together at the same time.
     
  21. jpaiva-msft

    jpaiva-msft

    Joined:
    Oct 17, 2017
    Posts:
    7
    @Tautvydas-Zilys does it include the fix to allow P/Invoke to kernel32.dll!GetNativeSystemInfo in WinRT? Or is that coming separately?
     
  22. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    That's a separate fix but is also coming to both of those releases.
     
  23. RobJellinghaus

    RobJellinghaus

    Joined:
    Jul 10, 2017
    Posts:
    16
    It would be super helpful if there could be a blog post about this. Right now the latest blog post on the .NET scripting runtime work says:

    It definitely doesn't say or even hint that the .NET backend is being deprecated. Why is this being done? Some background and public visibility on the roadmap here would be greatly appreciated. Thanks.
     
  24. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    Yeah, we actually have been working on a post for the last few weeks, but it got delayed by the hackweek and my personal vacation. I'll post it today.
     
    RobJellinghaus likes this.
  25. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Hi,
    I have also written to support:
    Unfortunately the issue isn't resolved for us ( 1045847 ).
    While the project exports now, runtime doesnt work.
    If I use .NET, everything works fine.
    Can you look at that again?
    Thanks!
     
  26. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    QA said to create a new issue, so here it is (nice demo included :)): 1066188
     
    RobJellinghaus likes this.
  27. ashikns

    ashikns

    Joined:
    Nov 23, 2016
    Posts:
    7
    Do you have any update on when the ETW related issue will be fixed in IL2CPP?
     
  28. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    It’s already fixed!
     
  29. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Is there an update for my prev issue aswell?
    1066188
    Havent heard back from QA so far and we're kinda locked down with updating our app ...

    Thanks!
     
  30. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    I just pinged QA about it. Sorry about the delay.
     
  31. ashikns

    ashikns

    Joined:
    Nov 23, 2016
    Posts:
    7
    I still get the errors with "IL2CPP does not support marshalling..." with ETW in the callstack when I use DataContractJsonSerializer.
     
  32. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    What exact error and where do you get? The error that was fixed was one where ETW functions failed to be loaded.
     
  33. ashikns

    ashikns

    Joined:
    Nov 23, 2016
    Posts:
    7
    Not sure if I'll have time to repro it again. Basically anytime you used Deserialize using DataContractJsonSerializer it would fail in IL2CPP. I fixed the problem by switching to Json .Net.
     
  34. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
    Oh my, time flies.
    We are locked to Unity 5.6 for Windows Store because of this issue ( 1066188 ).
    On September 11th, support says:
    We have been able to reproduce this bug and have sent it for resolution with our developers.

    What's the status of this issue? In some other thread you are asking "what is preventing us to switch to IL2CPP". Seriously, with Vungle not working on Windows Store, you cannot monetize via ads on Windows Store (there's no real alternative).

    And performance when loading Asset Bundles SUCKS REAL REAL HARD with 2017.x and .NET.
    It is OK with IL2CPP (compared to performance of 5.6).
     
  35. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    The bug was picked up recently by another dev on my team. I'll talk with him about it - it will probably be fixed very soon.
     
  36. mr_zog

    mr_zog

    Joined:
    Jan 21, 2014
    Posts:
    165
  37. StephenHodgson-Valorem

    StephenHodgson-Valorem

    Joined:
    Mar 8, 2017
    Posts:
    148
    Not sure if this is a regression but I noticed using `DateTimeOffset` causes this issue as well.

    There was an issue in the tracker, but this call causes an exception cause it tried to reach into the registry to get the time.

    FYI I was calling this in a managed class from the UI Thread. Not sure if that makes a difference.
     
    Last edited: Nov 13, 2018
  38. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    What's the callstack for the exception? Is it actually leaked or caught inside DateTimeOffset code?
     
  39. StephenHodgson-Valorem

    StephenHodgson-Valorem

    Joined:
    Mar 8, 2017
    Posts:
    148
    upload_2018-11-14_8-19-29.png

    I tried to get a text stack, but couldn't figure out how to copy them without doing each line by line, lol.

    How come exceptions in the output window don't print the stack as well?
    Is that something that could be added to the il2cpp logging output?
     
  40. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
  41. chrisoje

    chrisoje

    Joined:
    Apr 18, 2018
    Posts:
    1
    What version of Unity has these fixes? We are on 2018.1.9f2.
     
  42. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    Which fixes? There are were a lot of different topics discussed in this thread.
     
  43. LSPredator

    LSPredator

    Joined:
    Jun 23, 2014
    Posts:
    8
    What is current state of Vungle and IL2CPP?
    When can we use Vungle at Windows Store with IL2CPP scripting backend?
     
  44. snowconesolid

    snowconesolid

    Joined:
    Dec 9, 2011
    Posts:
    868
    Any updates on this? I can build with IL2CPP using Unity 2018.3.3f1,
    but when opening up the .sln file in visual studio that unity creates, it says

    (Unavailable)

    And underneath it says:
    The project file was not loaded

    What gives? I can build just fine with .Net as the scripting backend. IL2CPP is broken, even in the newest version of Unity. Does anyone have a fix?
     
  45. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    6,545
    This is not related to above issues. I'm not sure why VS doesn't load it. Did you try right clicking on it in solution explorer and pressing reload project?
     
  46. snowconesolid

    snowconesolid

    Joined:
    Dec 9, 2011
    Posts:
    868
    Yes I did. I sent you a message with the error i'm getting.
    I thought this might be a similar issue to what others are experiencing here that's why replied here.