Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

ARKit Remote and .NET 4.6 Experimetal

Discussion in 'AR' started by JaviTheGreat, Feb 9, 2018.

  1. JaviTheGreat

    JaviTheGreat

    Joined:
    Sep 28, 2015
    Posts:
    6
    Hello community,

    I have a AR project that uses .NET 4.6.
    Now I want to integrate ARKit to have all the amazing features.
    All the ARKit examples (with remote) work perfectly on .NET 3.5. But as soon as I integrate to my project and use .NET 4.6, the ARKit Remote tool can't connect to the device. Since this is kind of new, I haven't found any answer on Internet yet. Please help.

    The error is related to Serialization of an object in the remote scene (See log bellow).

    Environment:
    + MacOs High Sierra 10.13.1
    + Unity 2017.2.0p4
    + ARKit
    + ARKit Plugin
    + ARKit Remote
    + In-house libraries (that need .NET 4.6)

    Repro steps:
    1. In Player Settings, change "Scripting Runtime Version" from .NET 3.5 to .NET 4.6 Experimental
    2. Follow ARKit Remote Demo instructions to build, run and connect: https://forum.unity3d.com/threads/a...nity-arkit-plugin.474385/page-13#post-3153415

    Log...

    SerializationException: Unable to find assembly 'Assembly-CSharp-firstpass, Version=1.0.0.0, Culture=, PublicKeyToken=null'.
    System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly () (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binarycommonclasses.cs:408)
    System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType (System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo assemblyInfo, System.String name) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryobjectreader.cs:1489)
    System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor (System.String objectName, System.String[] memberNames, System.Runtime.Serialization.Formatters.Binary.BinaryTypeEnum[] binaryTypeEnumA, System.Object[] typeInformationA, System.Int32[] memberAssemIds, System.Runtime.Serialization.Formatters.Binary.ObjectReader objectReader, System.Int32 objectId, System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo assemblyInfo, System.Runtime.Serialization.Formatters.Binary.SizedArray assemIdToAssemblyTable) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binarycommonclasses.cs:2153)
    System.Runtime.Serialization.Formatters.Binary.ObjectMap.Create (System.String name, System.String[] memberNames, System.Runtime.Serialization.Formatters.Binary.BinaryTypeEnum[] binaryTypeEnumA, System.Object[] typeInformationA, System.Int32[] memberAssemIds, System.Runtime.Serialization.Formatters.Binary.ObjectReader objectReader, System.Int32 objectId, System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo assemblyInfo, System.Runtime.Serialization.Formatters.Binary.SizedArray assemIdToAssemblyTable) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binarycommonclasses.cs:2204)
    System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped (System.Runtime.Serialization.Formatters.Binary.BinaryObjectWithMapTyped record) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryparser.cs:683)
    System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped (System.Runtime.Serialization.Formatters.Binary.BinaryHeaderEnum binaryHeaderEnum) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryparser.cs:656)
    System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run () (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryparser.cs:146)
    System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize (System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Runtime.Serialization.Formatters.Binary.__BinaryParser serParser, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryobjectreader.cs:174)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Boolean isCrossAppDomain, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryformatter.cs:197)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck, System.Runtime.Remoting.Messaging.IMethodCallMessage methodCallMessage) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryformatter.cs:173)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler, System.Boolean fCheck) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryformatter.cs:118)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream, System.Runtime.Remoting.Messaging.HeaderHandler handler) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryformatter.cs:149)
    System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize (System.IO.Stream serializationStream) (at /Users/builduser/buildslave/mono/build/mcs/class/referencesource/mscorlib/system/runtime/serialization/formatters/binary/binaryformatter.cs:111)
    Utils.ObjectSerializationExtension.Deserialize[T] (System.Byte[] byteArray) (at Assets/Plugins/UnityARKitPlugin/ARKitRemote/ObjectSerializationExtension.cs:39)
    UnityEngine.XR.iOS.ARKitRemoteConnection.UpdateCameraFrame (UnityEngine.Networking.PlayerConnection.MessageEventArgs mea) (at Assets/Plugins/UnityARKitPlugin/ARKitRemote/ARKitRemoteConnection.cs:123)
    UnityEngine.Events.InvokableCall`1[T1].Invoke (T1 args0) (at /Users/builduser/buildslave/unity/build/Runtime/Export/UnityEvent.cs:206)
    UnityEngine.Events.UnityEvent`1[T0].Invoke (T0 arg0) (at /Users/builduser/buildslave/unity/build/Runtime/Export/UnityEvent_1.cs:58)
    UnityEngine.Networking.PlayerConnection.PlayerEditorConnectionEvents.InvokeMessageIdSubscribers (System.Guid messageId, System.Byte[] data, System.Int32 playerId) (at /Users/builduser/buildslave/unity/build/Runtime/Export/Networking/PlayerConnection/PlayerEditorConnectionEvents.cs:66)
    UnityEditor.Networking.PlayerConnection.EditorConnection.MessageCallbackInternal (System.IntPtr data, System.UInt64 size, System.UInt64 guid, System.String messageId) (at /Users/builduser/buildslave/unity/build/Editor/Mono/Networking/PlayerConnection/EditorConnection.cs:143)
     
    Last edited: Feb 9, 2018
  2. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    This is expected - you have to have the same version of .NET on the iOSPlayer as well as the Editor, otherwise the serialization will be off.
     
  3. XantoViorama

    XantoViorama

    Joined:
    Sep 22, 2017
    Posts:
    1
    Thank you for the quick response jimmya . I am working on this project with JaviTheGreat.

    I was under the impression though that we are using the same .NET on the compiled iOS project and the Editor when the steps we follow are:

    1. Change Player Settings to 4.6 on the iOS player settings (which is the selected platform).
    2. Build the app (with the scene mentioned in the instructions (UnityARKitRemote.scene) as it comes from the plugin plus the ARKitRemoteConnection prefab and nothing else).

    Doesn't the Editor change to 4.6 as well when we select it in the active player settings?

    If not, where is the .NET version of the Editor determined and changed? I failed to find an answer through Googling about this.

    Would be awesome if you could answer these. Thanks a lot!

    EDIT:

    Looking at the .csproj and Editor.csproj of the project, both have
    TargetFrameworkVersion v4.6
    LangVersion 6

    To be extra clear as I realise the original post might have been misleading, this issue is happening by using the most basic setup, just add the ARKit Plugin to a new project and build the UnityARKitRemote.scene with the ARKitRemoteConnection prefab as per the instructions, without using anything unrelated to the plugin.
     
    Last edited: Feb 14, 2018
  4. max_viorama

    max_viorama

    Joined:
    Apr 29, 2015
    Posts:
    5
    So apparently the issue is .Net's BinaryFormatter [1] which is not supposed to be used across different net implementations. This apparently works for IL2CPP + Editor in version 3.5 but creates issues with both sides in version 4.6.

    We had a closer look at the binary data, and it seems that they only differ in the message header.
    We tried using a different formatter, which failed due to reflection not being available in IL2CPP runtime.

    We will be investigating the following two solutions on monday:
    1. patch the header of the serialized data stream, probably less elegant but a quick fix for our issue
    2. use a different serialization library that does not rely on reflection. Do you have any idea which one would be ideal considering the performance and licensing constraints?


    [1] https://msdn.microsoft.com/en-us/li...matters.binary.binaryformatter(v=vs.110).aspx
     
  5. max_viorama

    max_viorama

    Joined:
    Apr 29, 2015
    Posts:
    5
    I continued to investigate the issue. Option 1 failed, so I did a quick hack using JsonUtility as an alternative serializer: https://github.com/mschneider/arkit...n/ARKitRemote/ObjectSerializationExtension.cs

    Now the only error I get is a problem to deserialize one of the plugin scripts on iOS .net 4.6.

    Code (csharp):
    1. A script behaviour (probably UnityEngine.XR.iOS.ARKitRemoteConnection?) has a different serialization layout when loading. (Read 32 bytes but expected 56 bytes)
    2. Did you #ifdef UNITY_EDITOR a section of your serialized properties in any of your scripts?
    3.  
    See attached the full log (editor):

    Code (csharp):
    1. [proxy] Start proxy: fc1a3ae2522d87e2425e57b8c54cc5591df78198 10000->55000
    2. Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
    3. UnityEditorInternal.ProfilerDriver:DirectURLConnect(String)
    4. UnityEditor.AttachProfilerUI:DirectURLConnect(String) (at /Users/builduser/buildslave/unity/build/Modules/Profiler/Editor/ProfilerWindow/AttachProfilerUI.cs:63)
    5. UnityEditor.<AddDeviceProfilers>c__AnonStorey2:<>m__1() (at /Users/builduser/buildslave/unity/build/Modules/Profiler/Editor/ProfilerWindow/AttachProfilerUI.cs:142)
    6. UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32) (at /Users/builduser/buildslave/unity/build/Modules/Profiler/Editor/ProfilerWindow/AttachProfilerUI.cs:36)
    7.  
    8. [/Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp line 116]
    9. (Filename: /Users/builduser/buildslave/unity/build/Modules/Profiler/Editor/ProfilerWindow/AttachProfilerUI.cs Line: 63)
    10.  
    11. [proxy] Connected to device
    12. Serialize path=/Users/max/Library/Application Support/Unity Technologies/Unity ARKit Plugin type=Utils.serializableFromEditorMessage
    13. UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
    14. UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
    15. UnityEngine.Logger:LogFormat(LogType, String, Object[])
    16. UnityEngine.Debug:LogFormat(String, Object[])
    17. Utils.ObjectSerializationExtension:SerializeToByteArray(Object) (at Assets/UnityARKitPlugin/ARKitRemote/ObjectSerializationExtension.cs:27)
    18. UnityEngine.XR.iOS.ARKitRemoteConnection:SendToPlayer(Guid, Object) (at Assets/UnityARKitPlugin/ARKitRemote/ARKitRemoteConnection.cs:203)
    19. UnityEngine.XR.iOS.ARKitRemoteConnection:SendInitToPlayer() (at Assets/UnityARKitPlugin/ARKitRemote/ARKitRemoteConnection.cs:193)
    20. UnityEngine.XR.iOS.ARKitRemoteConnection:OnGUI() (at Assets/UnityARKitPlugin/ARKitRemote/ARKitRemoteConnection.cs:69)
    21. (Filename: Assets/UnityARKitPlugin/ARKitRemote/ObjectSerializationExtension.cs Line: 27)
    22.  
    23. <i>Autoconnected Player</i> The referenced script on this Behaviour is missing!
    24. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Mono/MonoBehaviour.cpp Line: 1510)
    25.  
    26.  
    27. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    28.  
    29. <i>Autoconnected Player</i> The referenced script on this Behaviour (Game Object 'ARKitRemoteConnection') is missing!
    30. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Mono/MonoBehaviour.cpp Line: 1753)
    31.  
    32.  
    33. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    34.  
    35. <i>Autoconnected Player</i> A script behaviour (probably UnityEngine.XR.iOS.ARKitRemoteConnection?) has a different serialization layout when loading. (Read 32 bytes but expected 56 bytes)
    36. Did you #ifdef UNITY_EDITOR a section of your serialized properties in any of your scripts?
    37. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Serialize/SerializedFile.cpp Line: 1894)
    38.  
    39.  
    40. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    41.  
    42. <i>Autoconnected Player</i> Setting up 1 worker threads for Enlighten.
    43.  
    44. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    45.  
    46. <i>Autoconnected Player</i>   Thread -> id: 16ee37000 -> priority: 1
    47.  
    48. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    49.  
    50. <i>Autoconnected Player</i> UnloadTime: 3.364041 ms
    51.  
    52. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    53.  
    54. <i>Autoconnected Player</i> STARTING ConnectToEditor
    55. UnityEngine.XR.iOS.ConnectToEditor:Start()
    56. (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
    57.  
    58.  
    59. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    60.  
    61. <i>Autoconnected Player</i> connected
    62. UnityEngine.XR.iOS.ConnectToEditor:EditorConnected(Int32)
    63. UnityEngine.Events.UnityEvent`1:Invoke(T0)
    64. UnityEngine.PlayerConnectionInternal:UnityEngine.IPlayerEditorConnectionNative.RegisterInternal(Guid)
    65. UnityEngine.Networking.PlayerConnection.PlayerConnection:Register(Guid, UnityAction`1)
    66. UnityEngine.XR.iOS.ConnectToEditor:Start()
    67. (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
    68.  
    69.  
    70. (Filename: /Users/builduser/buildslave/unity/build/Runtime/Network/PlayerCommunicator/EditorConnection.cpp Line: 102)
    71.  
    72. Refresh: detecting if any assets need to be imported or removed ... Refresh: elapses 0.020424 seconds (Nothing changed)
    73.  
    Could you have a look at the repository? It is a minimal example project of the remote. Both editor and iOSPlayer are correctly configured to 4.6. The file ObjectSerializationExtension.cs contains all the modifications needed to circumvent the issues of the original binary serializer. What do you think about the error related to UnityEngine.XR.iOS.ARKitRemoteConnection ?
     
    Last edited: Feb 17, 2018