Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

4.5 - Serialization Depth

Discussion in 'Editor & General Support' started by LightStriker, May 27, 2014.

  1. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579
    No. Script behaviour may change drastically as none of the data will be serialized.

    Without the attribute, up to 8 levels of property nesting will be serialized. This is by-value serialization, such that if you have a variable that is a of a serializable type then it is always serialized, even if it's null. This is where the problems come from, particularly with recursive structures; because the serialization is by-value then a) it always follows the recursion as far as possible, i.e. to a depth of 8, and b) in many cases it actually doesn't do what you'd expect because object identity isn't preserved.

    With the attribute, nothing is serialized.
     
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579

    Well, you've got a few options:

    a) Use the new ISerializationCallbackReceiver interface to pack/unpack your nested structure into some other serializable format - for example, using regular .NET MemoryStream/BinaryWriter to serialize your object graph to an array of bytes, then hand that array of bytes to Unity to serialize.

    b) Have each EditableList be a separate ScriptableObject (potentially within a single asset file). Because they'd derive from UnityEngine.Object, references to them get serialized by-reference (with support for nulls, identity, etc) instead of by-value.

    c) Flatten your structure, like I demonstrated earlier in the thread - instead of having nested elements, have a single list of elements with fixed indices, and then refer to those elements by index instead of embedding them directly.
     
    Steamroller and dusthand like this.
  3. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Right best go back and edit.
     
  4. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    I suspect that it is, but I don't think that means it's broken such that it needs to be "fixed"...

    While I haven't done it for quite some time, I used to be able to edit and reload scripts at runtime in Unity. As far as I understand, that means the scripting runtime needs to be able to grab all the data out of the running objects before the reload and poke it back in afterwards. It wouldn't be much use if it couldn't do that to all of your fields, otherwise the internals of every object would get borked every time it happened (which is what typically happens to me now when I try, but I use a lot of stuff that I know isn't serialisation friendly anyway).

    So, I think it's a requirement of being able to recompile code while your game is running in the Editor, and I suspect that's a deliberate feature since otherwise I'd expect recompiles to be delayed until you exit play mode.
     
  5. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    But the whole reason we're supposed to put "Serializable" on private fields is because the private fields weren't supposed to be serialized. If they are being serialized anyway, then why make us add the redundant attribute in order to keep the state?
     
  6. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    Good question.

    My guess - and that's all it is - is that there's internally a difference between short term persistence (within an Editor session, where the intention is hot-loading of compiled scripts) and long term persistence (via disk, where the intention is to re-load data we explicitly wanted saved to disk).

    Perhaps it's not documented because it's meant to be something the Editor does transparently for us?

    In particular...
    1. There's a difference between "weren't supposed to" and "aren't". As I already mentioned, there are actually good reasons that Unity might internally need to store that data.
    2. We're supposed to put Serializable on private fields based on use case, not based on implementation. In other words, we're marking stuff we want saved permanently, we're not telling the Editor how to do its job.

    Basically what I'm saying is that I think there are actually two jobs going on here that happen to have a lot of overlap in implementation. I don't think it's a bug that a bunch of other data also gets stored, I think it's deliberately done to facilitate other functionality, but this new error message is exposing it and confusing matters because of the functional overlap with things we do on our side of the fence.


    To be completely honest, while I'm not aware of why they've chosen this particular approach and there could be great reasons I simply haven't thought of, I think this error is a bit... silly... for other reasons anyway. Partly because eight layers isn't very deep, and partly because I don't see this as an effective solution against cycles - depth alone isn't a good indicator, and I'd have thought it's better dealt with at write time rather than at read time... I mean, if I serialise something and the data is junk because it's got dodgy cycles or because cycles have been flattened, then it's far more useful to know about that at the time instead of later on when I go to re-load it.
     
    Last edited: Jun 19, 2014
  7. ldaughtry

    ldaughtry

    Joined:
    Oct 30, 2012
    Posts:
    38
    For the record, the AudioController class in the Audio Toolkit by Clockstone ( http://unity.clockstone.com/assetStore-audioToolkit.html ) definitely exhibits this behavior. I've contacted them about it. Will keep everyone posted.

    UPDATE - So the previous message was sent after I'd updated to the latest version of the plugin (6.7) and the plugin still exhibited this behavior. HOWEVER, I determined it was this by disabling the AudioController class instance which caused the errors to no longer display.

    Once I enabled the instance of the AudioController class again, the errors no longer showed.

    Not sure if that helps anyone, but that's what's happened on this end.
     
    Last edited: Jun 20, 2014
  8. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    The sudden inclusion of this new warning error with the explanation that a class referencing itself is a "huge performance hit" despite no one else ever experiencing that in the last 4 and a half versions of Unity, with no followup answers from Unity, seems like maybe the dev who included this new warning is not on the same page as the rest of the dev team, and that Unity themselves aren't totally sure how their serialization works right now. My guess is they're going to leave this and almost all other Mono issues unresolved with the hope that they can get il2cpp out the door, which will likely be changing serialization anyway.
     
  9. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    Where does the idea that this is in any way related to IL2CPP or versions of Mono come from? I also don't understand the implication that they "aren't totally sure how their serialization works" comes from.

    Unity's serialisation is... odd, I'll give you that, but since I've got little experience creating serialisation systems I can't really pass meaningful judgement.
     
  10. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    I mean since they're going to have a huge change to how Mono works plus an upgrade to a new Mono once they implement IL2CPP, they're probably not going to invest a whole lot of time trying to fix this version. My statement that they aren't sure how it works comes from the conflicting explanations in this and the "All about serialization" thread.
     
  11. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579
    The serialization system isn't part of Mono. I don't think it'll be affected by the version upgrade.
     
  12. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    They use their serialization system to marshal everything between the Mono runtime and the underlying C++ engine. With il2cpp getting rid of the runtime, they will either skip serialization there or maybe have some new weird version to go between the il2cpp "sort of a runtime" and the engine. But yes, this is speculation. Still, I think if they were ever going to answer any of the questions in this thread they probably would have replied by now.
     
  13. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579
    No they don't. This isn't cross-AppDomain communications; serialisation is not involved in managed/native boundary crossings at all. They use serialisation to get objects from disk into memory and back again - a task that does not go away with IL2CPP - and they use it to Instantiate objects - again, something unaffected by IL2CPP.

    Also, IL2CPP is not 'getting rid of' the runtime. There's still a runtime, it's just much smaller and lighter-weight than Mono (i.e. it only does stuff like garbage collection, instead of the entire JIT circus).
     
  14. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    In this thread: http://forum.unity3d.com/threads/serialization-best-practices-megapost.155352/

    They say when you press play, this happens:
    • Pull all the serializable data out of managed land, creating an internal representation of the data on the C++ side of Unity.
    • Destroy all memory / information associated with the managed side of Unity, and reload the assemblies.
    • Reserialize the data that was saved in C++ back into managed land.

    And I meant IL2CPP is getting rid of the Mono runtime specifically; i did mention they will have their own sort-of runtime.
     
  15. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579
    Oh, AppDomain reload. OK, yes, they use it there too. (Though that's hardly using it to 'marshal everything,' it's just one situation that exists only in the editor).

    But none of this is relevant to your point: IL2CPP and the impending Mono upgrade has no impact on the serializer. The new runtime - whether it's an upgraded Mono in the editor, or the IL2CPP runtime - will still need to have data loaded and saved, will still need to support Instantiate, and will still need to support AppDomain reloading. There's no reason to assume that the existing SerializedObject/SerializedProperty system will be changed in the process.
     
  16. Bovine

    Bovine

    Joined:
    Oct 13, 2010
    Posts:
    147
    So if I have a class as mentioned like so

    Code (CSharp):
    1. <code>
    2. [Serialize]
    3. public class Node
    4. {
    5.    public Node Child;
    6. }
    7. </code>
    I don't necessarily care that Child serializes and then its child serializes and so on, if for instance I have something like:

    Code (CSharp):
    1. [Serialize]
    2. public class NodeList
    3. {
    4.    public List<Node> Nodes;
    5.    public Node RootNode;
    6. }
    So long as each Node in Nodes gets serialized (but their Node field isn't serialized recursively), BUT I need Node.Child to maintain the reference to the Node referred to in Node.Child.

    I guess what I am saying is we could use a new attribute:

    Code (CSharp):
    1. <code>
    2. [Serialize]
    3. public class Node
    4. {
    5.    [SerialzieAsReference]
    6.    public Node Child;
    7. }
    8. </code>
    So the reference is maintained, but the Value of Child is assumed to be serialized elsewhere.

    Otherwise I'm going to have to re-write everything to use some Node.Id and use those everywhere and find actual Nodes from a master list.

    Is this the case?
     
  17. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    If Node is a MonoBehaviour or ScriptableObject then serialisation by reference works fine, so that's another option (though there are other drawbacks).
     
  18. smilecow

    smilecow

    Joined:
    Jul 4, 2014
    Posts:
    2
    I am surffering for this problem, too.

    I love unity engine.
    But I can't understand why serialization system was implemented this way.

    I made my own serialize system in unity environment.
    This support null, polymorphism (without scriptableobject), and circular reference.
    There is no performance penalty.

    Until now, I implemented serialization three times.
    So, I think I know it is difficult to implement useful serialization system.
    But, I really don't think it is not reachable goal.

    Is there things i missed?
    Or this function cause too massive change to currently unity implementation?

    (I'm not native english speaker. sorry for my poor writing)
     
  19. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    There probably is, but for your use cases it probably doesn't matter. Unity uses serialisation for a huge number of things and, while I'm not experienced enough in the area to be able to judge whether or not it's a great solution, it makes sense and lines up with what other game developers working on similar systems have told me in the past.
     
  20. Waz

    Waz

    Joined:
    May 1, 2010
    Posts:
    279
    Okay, I can handle adding System.NonSerialized attributes to my private fields for whatever reason it's needed (despite the documentation saying the opposite). Could Unity at least give a backtrace of the fields when this error occurs to make it easier to find the "cause" of the problem?
     
  21. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579
  22. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    I remember this thread from forever ago and we never really did get an answer: does anyone at UT know if the serialization of private variables that are not marked serializable is a bug or a feature? And if it's a feature, what is the point of marking things Serializable, and will there ever be a change to the documentation since it is all incorrect?
     
  23. Waz

    Waz

    Joined:
    May 1, 2010
    Posts:
    279
    Does this emulate the same choices of serializing private variables that Unity itself does (which no-one here seems to understand nor can we seem to get a straight answer other than "sometimes Unity does it")?
     
  24. Lucas-Meijer

    Lucas-Meijer

    Unity Technologies

    Joined:
    Nov 26, 2012
    Posts:
    175
    When we serialize your monobehaviours with the purpose of reloading the app domain (when you hit play), then we serialize the private fields on purpose.

    It was made like this on purpose back in the day, but today, we are strongly considering not serializing private variables in this (or any other) scenario, because it can be confusing, and for game code does not solve many real world problems (imo). The reason it's not changed yet is we need to figure out how to do it in a way that we do not break everyone's editor windows (including our own), as they are likely to rely on the fact that their private variables get serialized for domain reloads.

    Bye, Lucas
     
  25. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    Thanks, I'm glad we finally got an answer. I'd agree that private variables should not be serialized, even if it breaks some editor windows, because currently the documentation on how to deal with serialization in editor windows is all wrong anyway, and people shouldn't be relying on an undocumented, sort of accidental preservation of their variables. It also makes much more sense to ask people to put [SerializeField] on private variables they want to keep, as the docs say to, then changing the docs now to ask everyone to put [Nonserialized] on all the private variables they don't want to keep; something that I'm pretty sure nobody at all did until this thread appeared.
     
  26. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    Actually, that whole thing might benefit from a fresh look. [SerializeField]'s use cases would change dramatically if we had to use it to preserve private data across domain reloads, because currently it's used among other things for displaying private data in the Inspector, ie: separating the designer interface from the coder interface. There already exists [HideInInspector] to make something that gets serialized but does not get displayed, so we're functionally covered, but is that a well designed workflow? I think if this stuff has to change anyway we might be better off if the designers consider the use cases rather than what's happening under the hood - [ExposeInInspector], [PreserveForDomainReload] and [SerializeWithScene] or similar.

    Having said all of that, as long as we know when and why privates are serialized, what's the problem? It makes sense to me to preserve the entire state for domain reloads, because as a user I expect that when I play and unplay everything is exactly as I left it, including internal state.
     
  27. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    To me, it's the opposite; I expect that private variables that are not settable from the inspector have their default values whenever entering or exiting play mode. Especially since that's what the documentation more or less says, and since there are instructions on how to preserve data within the editor for various reasons (ScriptableObjects, SerializeField, etc). I can't think of any situations where I would particularly want the current behavior, where some private variables are preserved in certain odd situations like exiting play mode but not in others like recompiling or opening/closing an editor window or clicking on a different inspector. Especially since this current, mostly unwanted behavior apparently leads to "huge performance hits" as mentioned earlier.
     
  28. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    15,290
    I'd probably agree if I'd run into the "certain odd situations" you mention. To be clear, though, I'm talking about stuff in the scene, not editor windows and such. In any case, if/where there are inconsistencies they should certainly be cleared up.
     
  29. KalebGraceTwistory

    KalebGraceTwistory

    Joined:
    Mar 6, 2014
    Posts:
    3
    I got this error when I used an auto-implemented property, even though it shouldn't be serialized. I moved some code around and it seems to be fine now.

    BEFORE (throws error):
    Code (CSharp):
    1.  
    2.   [System.Serializable]
    3.   public class AudioInfo {
    4.   public List<AudioClipGroup> m_audioClipGroups = new List<AudioClipGroup>();
    5.   //...
    6.   }
    7.  
    8.   [System.Serializable]
    9.   public class AudioClipGroup {
    10.   public List<AudioClip> m_audioClips = new List<AudioClip>();
    11.   public AudioInfo info { get; set; }
    12.   //...
    13.   }
    14.  
    AFTER (no error):
    Code (CSharp):
    1.  
    2.   [System.Serializable]
    3.   public class AudioInfo {
    4.   public List<AudioClipGroup> m_audioClipGroups = new List<AudioClipGroup>();
    5.   //...
    6.   }
    7.  
    8.   [System.Serializable]
    9.   public class AudioClipGroup {
    10.   public List<AudioClip> m_audioClips = new List<AudioClip>();
    11.   [System.NonSerialized]
    12.   private AudioInfo m_info;
    13.   public AudioInfo info { get { return m_info; } set { m_info = value; } }
    14.   //...
    15.   }
    16.  
     
  30. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Weird, report a bug.
     
  31. crafTDev

    crafTDev

    Joined:
    Nov 5, 2008
    Posts:
    1,816
    I started getting this error after I upgraded to 4.6.2 yesterday... :/

    I am not clear on how to fix.

    Nvm, I finally understood the error and found the cycle in my code and fixed it. Dunno why I coded it like that in the first place o_O
     
    Last edited: Feb 5, 2015
  32. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    323
    If it helps anyone, I just had a recursion depth issue, that was a bit trickier to solve, as the problematic class was never actually identified.

    The issue was caused by an API class that I had on my GameSaveData class. So it was structured like so:

    Code (CSharp):
    1.  
    2. [Serializable]
    3. GameSaveData {
    4.    public GameSaveApi api;
    5.  
    6.    public GameSaveData(){
    7.           api = new GameSaveAPI(this);
    8.    }
    9. }
    10.  
    11. [Serializable]
    12. GameSaveApi {
    13.       //Internal references to parent instances
    14.       private GameSaveData data;
    15.       private LevelProgress progress;
    16.  
    17.       public GameSaveApi(GameSaveData data){
    18.             this.data = data;
    19.             progress = data.progress;
    20.       }
    21.       //various api calls...
    22.       public void SetLevelProgress(){}
    23.       public void CollectStar(){}
    24.       public void CollectItem(){}
    25.       //etc
    26.  
    27. }
    Having a child object, with a private reference to a parent object, causes this type of recursion.

    I should never have been serializing GameSaveApi anyways, since it really has no state, simply removing [Serializable] fixed all issues.

    It would have been nice though, if the error message would've been able to more accurately point to the issue. The error never once mentioned GameSaveAPI the source of the recursion, instead complaining about GameSaveData, LevelProgress as having recursion issues, when both of those classes were actually fine and exhibited no obvious signs of recursion.

    It probably would have been more obvious, had I been aware that private fields were actually serialized behind the scenes, but since all of GameSaveApi's fields were private, I just assumed it was not an issue.
     
    Last edited: Feb 11, 2015
  33. perlohmann

    perlohmann

    Joined:
    Feb 12, 2009
    Posts:
    221
    Is this something that will be worked on in Unity 5?
    Since the code is working I would like to know if we should spend a week optimizing this or if unity 5 will bring some changes to this which will improve the performance.
     
  34. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    There haven't been any major serialization changes in the Unity 5 beta so far, so I don't think they have any short term plans to fix any of this.
     
  35. Coredumping

    Coredumping

    Joined:
    Dec 17, 2014
    Posts:
    49
    Just updated to 4.6.3 and this happened to me. I have a tree structure, which obviously has a recursive reference. I serialize it using the BinaryFormatter, so not having it serialized is just not an option for me. The editor doesn't even touch this class ever.

    This is getting pretty annoying, why are all the bug reports on this marked as resolved, when it is clearly not resolved?
     
  36. ccklokwerks

    ccklokwerks

    Joined:
    Oct 21, 2014
    Posts:
    62
    Well, from Unity's POV this is working as intended. :-/

    I'm in the same boat as you. Developers prior to me used Serializable/SerializeField to tag non-scene data objects for serialization in a tree type structure, and now that I've updated from 4.5.4 to 4.6.3 (despite it NOT complaining during 4.5, which seems weird after reading this thread) I get all the errors.

    So, currently my plan is to switch over to using actual XML tags. It's unclear to me why Serializable/SerializeField actually work with XMLSerializer. It seems like a bit of a "trap".

    ( http://wiki.unity3d.com/index.php?title=Saving_and_Loading_Data:_XmlSerializer )

    edit: on further testing, it seems the developers before me put the SerializeField tags in for no reason whatsoever, and that the XMLSerializer was just doing its own analysis.
     
    Last edited: Mar 17, 2015
  37. Deleted User

    Deleted User

    Guest

    So, I understand the useful nature of this message for people just beginning to create new projects to be aware of the recursive limitations in Unity's serialization system, particularly if a large portion of performance based reports stem from this limitation. This is a good thing.

    What I have a hard time understanding is the service failure. There are thousands of projects prior to this release that use recursive systems quite cautiously, so rendering them near useless because they could possibly be taking a performance hit and not allowing them to accept responsibility is just not logical.

    I read somewhere in the past that custom inspector scripts are ignored in the compilation build of the end project. Is this true? If it is, is there a reason possible performance hits matter in custom inspectors? They shouldn't impact the game at all, ever.

    Lucas wrote a blog about this in the summer of 2014. In that blog he states:
    We actually messed up the warning implementation in such a way that it gives you so many warnings, you have no other option but to fix them right away. We’ll soon ship a fix for this in a patch release, the warning is not gone, but you will only get one per “entering playmode”, so you don’t get spammed crazy.

    I just now updated my project to the very latest version of unity 4. Can we get an ETA on this patch release? I can't imagine it legitimately took a year to implement or figure out, so was it just forgotten?

    Edit - 4.6.1 Does not throw the error message.
    Edit II - For the record, "fixing" this is relatively easy. It's just not always necessary. My only gripe is just the 'my way' approach.

    Regards,
     
    Last edited by a moderator: Apr 7, 2015
  38. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,346
    One option that might be nice to see is allowing a build setting like "Don't ever serialize my private variables". As I understand it, the main reason Unity does it is so that you can live-edit code while a scene is running in play mode and keep going, but that feature has never actually worked for me in any build of Unity. In this case, is there a need to serialize private variables or references that will disappear when exiting play mode?
     
  39. Pangamini

    Pangamini

    Joined:
    Aug 8, 2012
    Posts:
    53
    How can i track which class / field is causing this error? I'd gladly put the [NonSerialized] there, but the thing is i have no clue... the error log gives me nothing but the deepest class name, which is some random not serializable class. Is there any way how to track the root of this serialization tree? Where do i put the attribute? It's driving me and the whole team nuts, we had to update unity and this bug is slowing us down
     
  40. Yuanfeng

    Yuanfeng

    Joined:
    Feb 14, 2014
    Posts:
    2
    Serialization depth limit exceeded at 'GameDraw::part'. There may be an object composition cycle in one or more of your serialized classes.
    UnityEditor.HostView:OnGUI()
    I get hundreds of this error too after each build and just confuse about this.
     
  41. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    Well I'd love cyclic weak references for members that aren't serialized as well. I'm not sure how to do it without setting unity off on one.

    But regarding it never working for you to update live, this is because you have to AFAIK repair all references, ie all scripts need to setup again using OnEnable or such. Been a few years since I last bothered.
     
  42. FuriousEX

    FuriousEX

    Joined:
    Mar 13, 2014
    Posts:
    49
    We recently updated to 5.1 from 4.6 and are now getting the Serialization depth limit error even though our code is the same. The class that it is reporting does not have composition cycles - our project is very data driven and our data hierarchy legitimately goes 6 or 7 levels deep in the serialization with no recursive references.

    It is a hassle to see a few hundred errors for this (we don't seem to be losing any data from it either). What I would like to see is an editor option to define an acceptable serialization depth (that is assuming you can't add real cycle detection) - this hard coded magic number approach is sloppy at best.
     
  43. Steamroller

    Steamroller

    Joined:
    Jan 2, 2013
    Posts:
    71
    We hate to necro this thread but since this seem to be the first thread that pops up when doing a search for "Unity serialization depth limit exceeded", we figured we would post our solution here to give back to the community that we have leeched off of for so long :p

    Quick disclaimers:
    • Thanks to superpig for his suggestions which are the basis of this solution.
    • This is probably one of many ways to solve this problem
    • This code is lightly tested, but will be testing and updating this code, as we put it to use in our current game
    • We're totally open for suggesting on how to make this code better
    • We're using Full Inspector since its was the easiest way to test this code. There a couple bits of code that should be able to be remove/replace, as noted, if Full Inspector not part of your pipeline.
    • We hope this isn't something super obvious and is helpful for the community.
    Without further ado here is our solution.

    First of all we've created a nestable interface that you can implement in your own classes:

    INestable.cs
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3.  
    4. public interface INestable : IEnumerable
    5. {
    6. // Make sure this is [NonSerialized] in the implementation
    7. INestable parent
    8. {
    9. get;
    10. set;
    11. }
    12.  
    13. // Make sure this is [NonSerialized] in the implementation
    14. List<INestable> children
    15. {
    16. get;
    17. set;
    18. }
    19.  
    20. List<int> indices
    21. {
    22. get;
    23. set;
    24. }
    25.  
    26. void Initialize();
    27. }
    28.  
    Here is an example of an implementation that you can use as a base class if that works for your data structure or copy and paste this into your own class:

    NestedNode.cs
    Code (CSharp):
    1. using FullInspector;
    2. using System;
    3. using System.Collections;
    4. using System.Collections.Generic;
    5. using UnityEngine;
    6.  
    7. [Serializable]
    8. public class NestedNode : INestable
    9. {
    10.     [NonSerialized]
    11.     private INestable _parent;
    12.     public INestable parent
    13.     {
    14.         get
    15.         {
    16.             return _parent;
    17.         }
    18.         set
    19.         {
    20.             _parent = value;
    21.         }
    22.     }
    23.  
    24.     [NonSerialized]
    25.     private List<INestable> _children = new List<INestable>();
    26.     [ShowInInspector] // NOTE: Using Full Inspector to show list
    27.     public List<INestable> children
    28.     {
    29.         get
    30.         {
    31.             return _children;
    32.         }
    33.         set
    34.         {
    35.             _children = value;
    36.         }
    37.     }
    38.  
    39.     // The indices are the only things serialized on this class and they are hidden from the user
    40.     [HideInInspector]
    41.     public List<int> indices
    42.     {
    43.         get;
    44.         set;
    45.     }
    46.  
    47.     public void Initialize()
    48.     {
    49.         if ( children == null )
    50.         {
    51.             children = new List<INestable>();
    52.         }
    53.  
    54.         if ( indices == null )
    55.         {
    56.             indices = new List<int>();
    57.         }
    58.     }
    59.  
    60.     public IEnumerator GetEnumerator()
    61.     {
    62.         return children.GetEnumerator();
    63.     }
    64. }
    65.  
    And here is the root level structure that does all the book-keeping:

    NestedList.py
    Code (CSharp):
    1. using System;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5.  
    6. [Serializable]
    7. public class NestedList<T> : ISerializationCallbackReceiver where T : INestable
    8. {
    9.     [SerializeField]
    10.     public List<T> children = new List<T>();
    11.  
    12.     [SerializeField]
    13.     public List<T> hierarchy = new List<T>();
    14.  
    15.     public void Initialize()
    16.     {
    17.         if ( children == null )
    18.         {
    19.             children = new List<T>();
    20.         }
    21.  
    22.         if ( hierarchy == null )
    23.         {
    24.             hierarchy = new List<T>();
    25.         }
    26.     }
    27.  
    28.     private int SerializeHierarchy( T node )
    29.     {
    30.         if ( node == null )
    31.         {
    32.             return -1;
    33.         }
    34.  
    35.         node.Initialize();
    36.  
    37.         hierarchy.Add( node );
    38.  
    39.         node.indices.Clear();
    40.  
    41.         if ( node.children != null )
    42.         {
    43.             foreach ( T _child in node.children )
    44.             {
    45.  
    46.                 node.indices.Add( SerializeHierarchy( _child ) );
    47.             }
    48.         }
    49.  
    50.         return hierarchy.IndexOf( node );
    51.     }
    52.  
    53.     public void OnBeforeSerialize()
    54.     {
    55.         Initialize();
    56.  
    57.         hierarchy.Clear();
    58.  
    59.         foreach ( T _child in children )
    60.         {
    61.             SerializeHierarchy( _child );
    62.         }
    63.     }
    64.  
    65.     private void DeserializeHierarchy( T node )
    66.     {
    67.         node.Initialize();
    68.  
    69.         node.children.Clear();
    70.  
    71.         foreach ( var _index in node.indices )
    72.         {
    73.             if ( _index == -1 )
    74.             {
    75.                 node.children.Add( null );
    76.             }
    77.             else
    78.             {
    79.                 var _child = hierarchy[ _index ];
    80.                 _child.parent = node;
    81.  
    82.                 node.children.Add( _child );
    83.             }
    84.         }
    85.     }
    86.  
    87.     public void OnAfterDeserialize()
    88.     {
    89.         Initialize();
    90.  
    91.         foreach ( var _child in children )
    92.         {
    93.             DeserializeHierarchy( _child );
    94.         }
    95.     }
    96.  
    97.     public IEnumerator GetEnumerator()
    98.     {
    99.         return children.GetEnumerator();
    100.     }
    101. }
    102.  
    And finally here is an example usage:

    Foo.cs
    Code (CSharp):
    1. using FullInspector;
    2.  
    3. // NOTE: Replace with Monobehaviour if not using Full Inspector
    4. public class Foo : BaseBehavior
    5. {
    6.    // We are using our base class NestedNode here with the NestedList
    7.    // but you can use your own sub-class or your own implementation of the INestable
    8.     public NestedList<NestedNode> nodes = new NestedList<NestedNode>();
    9. }
    10.  
    We hope this is helpful for anyone running into this problem. Please let us know what you think.

    Cheers
     
    Last edited: Jul 24, 2015
    Aurigan and superpig like this.
  44. bjarkeck

    bjarkeck

    Joined:
    Oct 26, 2014
    Posts:
    301
    Here is how i got around the Error message, quite simple.

    This codes triggers the error:
    Code (CSharp):
    1. class MyComponent : MonoBehaviour
    2. {
    3.      private MyTree myTree;
    4. }
    5.  
    6. [Serializable]
    7. class MyTree
    8. {
    9.      public List<MyTree> children;
    10. }
    11.  
    And this code doesnt:
    Code (CSharp):
    1. class MyComponent : MonoBehaviour
    2. {
    3.      private MyTreeContainer myTree;
    4. }
    5.  
    6. class MyTreeContainer
    7. {
    8.      public MyTree MyTree;
    9. }
    10.  
    11. [Serializable]
    12. class MyTree
    13. {
    14.      public List<MyTree> Children;
    15. }
    16.  
    So just add a class that is not marked [Serializeable] as a container for it.
    Hope it helps someone.
     
  45. snlehton

    snlehton

    Joined:
    Jun 11, 2010
    Posts:
    99
    Sorry to return to an old thread but why in the earth do I need to do this? If the list of children is empty, why do I get this warning? Isn't it up to me to make sure that the depth just doesn't go too deep? What is the reasoning for generating the warning in this case? Default list is empty, and empty list stops the hierarchy.

    I find this child index book-keeping extremely unintuitive. It makes authoring hierarchical datastructures even more tedious. Speaking on which... is there a easy (asset store?) way of editing hierarchical things in same way that scene hierarchy or ReorderableList work? That'd be nice.
     
  46. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,579
    It's because of the way our serialisation system works. Once per MonoBehaviour/ScriptableObject class, we generate what is effectively a small program for the serialisation system. This allows us to process that type extremely quickly in the future - it means we don't need to do any reflection on it, etc - but it also means that the program we generate for it has to cover the 'worst case' for serialisation, which is to assume that lists are never empty.

    There are ways of fixing this, but none of them are small to implement, and it's extremely hard to do it without losing any performance in the non-hierarchical case.
     
  47. snlehton

    snlehton

    Joined:
    Jun 11, 2010
    Posts:
    99
    Thanks for taking time to answer this. It makes sense now, to be honest I was kind of assuming it's probably doing something like that in the background.

    The issue is that I've come across the same limitation more than once, and every time it feels like I'm need to jump some unnecessary hoops to get the data structure to work the way I want -- at least in the users perspective (editor side).
     
  48. Pangamini

    Pangamini

    Joined:
    Aug 8, 2012
    Posts:
    53
    It takes one extra minute just to print these errors in the console... is there seriously no way how to at least find the root / path of what exactly has the "composition cycle"? Because i am clueless.. i am now adding [NonSerialized] to any code i can find, but still get 30k of these errors and it's really slowing down the production of entire team!
     
  49. andyz

    andyz

    Joined:
    Jan 5, 2010
    Posts:
    2,054
    I get 'Serialization depth limit exceeded' on simple non-monobehaviour classes (though with nested children of same type) I create which I have no intention of having serialized, so have to add nonserialized to the affected variables just to get them to compile.
    This seems a bit silly but is it just a better alternative to having to add serialize to all classes you Want to serialize? (this is in js BTW (I know...) ). Can you not mark a whole class to serialize or not?
     
  50. Pangamini

    Pangamini

    Joined:
    Aug 8, 2012
    Posts:
    53
    My problem is that i don't know which class is doing this and i have no way how to tell