Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Visual Scripting vs Managed Code Stripping: findings and workarounds facing "Deserialization error"

Discussion in 'Visual Scripting' started by huulong, Dec 20, 2021.

  1. huulong

    huulong

    Joined:
    Jul 1, 2013
    Posts:
    224
    So, I had many errors when playing my IL2CPP builds (and also stripped Mono builds) and the error messages were not trivial to understand, as the reason lied not on the Visual Scripting side but on the stripping system.

    I'll sum-up the error messages I have encountered (for easy search), the circumstances in which stripping will make your Visual Script Graphs fail, and how to prevent stripping custom unit (node) classes.

    Really, once you understand how stripping works, the Visual Scripting errors have nothing special and are trivial to fix, but having myself searched a lot about this on the Visual Scripting forum and found nothing, I want at least to open a thread just to people like me have a beacon to then keep searching more on the stripping side.

    Error/warning messages and symptoms

    Watch out for the following behavior and errors/warnings in your builds (I recommend a Development Build to see errors appear live in the console, but remember that non-error log messages will not show up, you'll have to find them yourself in the last Player log when you suspect something is wrong):

    - A Visual Script graph does not execute, or executes partially. It may stop right before the first custom unit (node).
    - Error log: "System.Runtime.Serialization.SerializationException: Deserialization into 'Unity.VisualScripting.ScriptGraphAsset' failed. ---> System.InvalidOperationException: Internal Deserialization Error - Object definition has not been encountered for object with id=[NODE_ID]; have you reordered or modified the serialized data" where NODE_ID is the ID of the failing node in the JSON (debugged right above this message in the Player log).
    - Log (silent as not considered an error): "Failed to add element to graph during deserialization: [UNIT_NAME]#[ID]... System.NullReferenceException: Object reference not set to an instance of an object" where UNIT_NAME is the name of the failing unit (probably a custom unit).

    Possible causes

    - The Visual Script graph is using a custom Unit class that is stripped from the build, considering current settings
    - The Visual Script graph is using a standard Unit class but even standard units are stripped from the build, considering current settings
    - The Visual Script graph is using a standard Unit class but it references custom classes or members (including methods) via reflection, and those custom symbols are stripped from the build, considering current settings

    Stripping levels and preventing stripping

    The whole trick is to choose a stripping level that's good enough to reduce build size without losing too much information needed for Visual Scripting / Reflection. On one side, we must not lost the Units (esp. Custom Units) themselves during build. On the other side, we must not lose classes and members that may only be accessed via Visual Scripting (e.g. methods called via InvokeMember).

    So, the overall idea is to study the symptoms and error messages / warnings mentioned above, check the units/classes pointed by the ID and messages, and then make sure that those units/classes are not stripped.

    To do this, I followed the full documentation on stripping at: https://docs.unity3d.com/2022.1/Documentation/Manual/ManagedCodeStripping.html

    and those threads/pages:
    https://forum.unity.com/threads/while-custom-package-need-a-link-xml.727460/
    https://forum.unity.com/threads/the-current-state-of-link-xml-in-packages.995848/#post-6545491
    https://github.com/jilleJr/Newtonsoft.Json-for-Unity/wiki/Embed-link.xml-in-UPM-package

    To make you gain some time, I'll sum up the effect of stripping levels for Visual Scripting specifically so you can identify your case:

    First, IL2CPP needs at least Managed Stripping Level: Low

    Second, let's split Custom Unit definition "scopes" in 3 levels:
    - Native (provided by the VisualScripting package and nodes directly using the Unity API)
    - Local (Game Assembly and secondary Assemblies located under Assets/)
    - External (extra Assemblies located under Packages/)

    Then the different stripping levels preserve the following scopes (when not using extra tricks to prevent stripping):

    Stripping Level - Preserved scopes
    Disabled - All
    Low - Native + Local
    Medium - None
    High - None

    In addition, Medium and High levels seem to strip even native functions like Mathf.Abs, preventing its usage in Visual Script graphs (as the Abs node actually accesses it via reflection).

    This means that:
    - You'll want to set your Stripping Level at Low if you want to reduce the build size while still be able to use most common methods and your Custom Units defined in your own assemblies
    - The Custom Units defined in packages are not available by default. You need to prevent stripping manually, or the package provider must provide you with help to do so. However, if you do not rely on Custom Units (or use a package that handles stripping prevention), then you're good to go at level Low!
    - Medium and High level are almost unusable, although heavy stripping prevention may allow you to use them (untested)

    So, assuming you got the errors at the top, first switch to Low stripping level. If you still have them, focus on Custom Units in external Packages. Then apply the strip prevention techniques mentioned in the links above.

    Here is a sum-up of a few workarounds I've used myself, depending on the situation:

    a. You own the package and you don't care always preserving all the code there in all projects using the package: Perfect, just put `[assembly: AlwaysLinkAssembly, Preserve]` in one of the package files, in the outside scope, and you're good to go.

    b. You just need a few custom units and you don't want a complicated process: add dummy code to construct an instance of that custom unit, like this:

    Code (CSharp):
    1. public class AntiStrip
    2. {
    3.     private CustomUnit ref1;
    4.  
    5.     private void Define()
    6.     {
    7.         ref1 = new CustomUnit(); // new is important to prevent constructor stripping
    8.     }
    9. }
    This should force assembly linking and preserve the class and its constructor. Note that I haven't tested it on more complex Units which may need more than just the new, to prevent stripping extra methods. I did that at first as a trick, and after reading the full doc I wouldn't recommend relying on it too much.

    c. You want a clean way to preserve specific assemblies/symbols in your project: use a link.xml file as described in the doc. This must be on the project side and re-added for each project, but the advantage is that it can be adapted to each project to just pick the parts you need (this applies to any Package and symbol accessed via reflection, not just custom units for Visual Scripting). The drawback is that you must keep link.xml up-to-date with the latest symbol usages (and you don't have issues until you build, so you may miss a few during development).

    Finally, as I mentioned on my post in one of the threads linked above (https://forum.unity.com/threads/the-current-state-of-link-xml-in-packages.995848/#post-7723114), some packages like https://github.com/RealityStop/Bolt.Addons.Community may be self-contained as they have a link.xml template that they copy into your project on pre-build. You can use this as an inspiration for your own package too.

    Going further

    I don't know if it's possible to make Medium and High stripping levels work. In theory, by adding the native Unity API to link.xml you could preserve even the most simple methods like Mathf.Abs. Medium-stripped build wasn't that smaller than Low-stripped for me, so I didn't bother. But maybe it's different for some projects, and maybe High stripping level can really be beneficial.

    So if somebody tried Medium-High stripping + preserving required Unity assemblies, they could relate their experience in this thread.
     
    dave-mona and JayadevHaddadi like this.
  2. huulong

    huulong

    Joined:
    Jul 1, 2013
    Posts:
    224
    Interestingly, Visual Scripting itself generates a link.xml in Assets/Unity.VisualScripting.Generated/VisualScripting.Core during build to preserve their own symbols:

    Code (CSharp):
    1. <linker>
    2.   <assembly fullname="Unity.VisualScripting.Core" preserve="all" />
    3.   <assembly fullname="Unity.VisualScripting.State" preserve="all" />
    4.   <assembly fullname="Unity.VisualScripting.Flow" preserve="all" />
    5. </linker>
    This is removed after the build.

    And yet, I now get another error, on IL2CPP build (Development mode) only, stripping level Low:

    Not sure why StaticFieldAccessor is not preserved, and why this only showed now (I changed my Visual Script Graphs a bit but not much... the biggest change was the use of a ForEach loop).
     
    Last edited: Dec 24, 2021
  3. huulong

    huulong

    Joined:
    Jul 1, 2013
    Posts:
    224
  4. huulong

    huulong

    Joined:
    Jul 1, 2013
    Posts:
    224
    Hm, the IL2CPP build suddenly worked again, even without the link.xml... Not sure what happened.