Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Enter the 2020.2 Beta Sweepstakes for a chance to win an Oculus Quest 2.
    Dismiss Notice

Elvis operator / Null-Coalescing operator

Discussion in 'Experimental Scripting Previews' started by jvlppm_pf, Sep 28, 2016.

  1. jvlppm_pf

    jvlppm_pf

    Joined:
    Apr 24, 2016
    Posts:
    6
    Null-conditional / Null-Coalescing operators ( '?.' and '??' ) cannot currently be used on serialized fields.

    When deserializing a MonoBehaviour, all unreferenced fields are not set to null, instead it references a object with a == operator overload, so it results true when testing for null (when in fact it is not a null reference).

    This leads to some unexpected behaviors like:

    Code (CSharp):
    1.  
    2. public class TestBehaviour : MonoBehaviour
    3. {
    4.     UnityEngine.GameObject reference;
    5.  
    6.     void Start() {
    7.         if (reference?.transform == null) {
    8.             Debug.Log("It works!"); // This line is printed as expected
    9.         }
    10.     }
    11. }
    12.  
    But changing the reference to public changes the execution path

    Code (CSharp):
    1.  
    2. public class TestBehaviour : MonoBehaviour
    3. {
    4.     public UnityEngine.GameObject reference;
    5.  
    6.     void Start() {
    7.         if (reference?.transform == null) { // throws an exception
    8.             Debug.Log("Not executed"); // This line is not reached
    9.         }
    10.     }
    11. }
    12.  
    Exception:
    UnassignedReferenceException: The variable reference of TestBehaviour has not been assigned.
    You probably need to assign the reference variable of the TestBehaviour script in the inspector.​


    This null reference behaviour was probably created to help developers when trying to access unassigned references, where it would print the attribute name inside the exception, but this 'helper' is now causing an UnassignedReferenceException where it should not exist.


    The ?? operator also does not work as expected on that scenario.

    Code (CSharp):
    1.  
    2. var validReference = optionalReference ?? referencedAtribute;
    3.  
    The code above would always store 'optionalReference' on 'validReference', regardless if 'optionalReference' is set or not.

    -

    My opinion:
    this pretty UnassignedReferenceException only works when trying to access a Serialized Object attribute, so the user is not safe from NullReferenceExceptions.

    - It tries to solve a 'problem' that is not Unity's fault (out of its domain)
    - It helps only on some specific scenarios
    - It creates other problems with it

    Basically not worth it.


    What should companies do to avoid such weird bugs? Recommend its developers to not use '?.' and '??' at all? (yes I've heard that).


    At least give me the option to read attributes as null, when they are not referenced on Unity editor.
     
  2. strich

    strich

    Joined:
    Aug 14, 2012
    Posts:
    288
    I think many would agree. This will become quite a lot worse when C# 6 is rolled out and ?. becomes available as well.
     
  3. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    Could you do a quick check for what the second version does in builds? I believe Unity does not replace the null with a bullshit "let me help you will your nullref" object in builds.

    If that's the case, maybe the difference between editor time and runtime is enough to get rid of it! It really only helps the people who are struggling through Roll-A-Ball, and is an ugly gotcha for everyone else.


    The ?? error has probably always been there, I just haven't run into it because ?? doesn't really work with the whole == override thing.
     
    laurentlavigne likes this.
  4. pvloon

    pvloon

    Joined:
    Oct 5, 2011
    Posts:
    587
    I'm wondering what unity is to do here. I agree that all the bogus null objects should be removed (for serialization and also for GetComponent returns) - we just want to write standard C# (and get rid of useless allocations while at it.

    However, you'd still have this problem for Destroy() and your own variables. The nullref overload there is well... it is convenient and at this point impossible to remove really

    Maybe now that they're in the .net steering group they could ask microsoft for some mechanism to have ? and ?? compiles to using the actual Equals or == operator instead of hardcoded compare. I guess that might impact performance so it would need to only compile to that for certain types.
     
  5. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    This was discussed in some length on the blog a couple of years ago.

    Basically, the custom operator is there to avoid the need to double up every single null check with an isAlive check or similar, since the object could have been destroyed on the C++ side of things. In my opinion, the custom operator should be removed and the implicit bool cast that is already in place should be used for that purpose instead.
     
  6. pvloon

    pvloon

    Joined:
    Oct 5, 2011
    Posts:
    587
    While I agree for if (blah == null) - I think with C# 6 the situation is different now. It means that we still couldn't write m_comp?.Var defeating the now common ?. operator.

    I really don't like the operator either but at the same time unity has always set the goal that you can just write standard C# . I think atm I agree, this is broken, but maybe with their influence in .net now they can do something about the underlying problem.
     
  7. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,683
    Thanks for bringing this up. We're having some internal discussions about how to best handle this now, but we don't have a consensus yet. With these new C# 6 operators, it certainly brings the underlying issue to the forefront. We're open to more discuss/suggestions about how to best deal with this.
     
    GarthSmith and pvloon like this.
  8. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    Are you talking about the special handling of null fields in the editor, or the == null override? Or both?
     
  9. JoshPeterson

    JoshPeterson

    Unity Technologies

    Joined:
    Jul 21, 2014
    Posts:
    4,683
    Really just the == null override, but I think everything is open to discussion at the moment.
     
  10. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    So the special handling of null fields is easier to handle - just remove it. It's causing inconsistencies between the editor and the runtime and it's causing the profiler to show memory allocations that's not there at runtime. The help it gives to new users who doesn't know what a nullref is doesn't seem to help much, from what I'm seeing in the scripting forums.

    The == null thing is harder. The problem is that it's really convenient that == null checks for destroyed objects. It has a downside where it's surprising for experienced programmers that's new to Unity and it breaks around interfaces (see discussion here), so I'm a bit back-and-forth on what's the good option.

    If the goal is that these statements should be equivalent:

    Code (csharp):
    1. var foo = bar != null ? bar : new Foo();
    2. //and
    3. var foo = bar ?? new Foo();
    and that these statements should be equivalent:

    Code (csharp):
    1. if(foo != null) {
    2.     foo.DoSomething();
    3. }
    4. //and
    5. foo?.DoSomething();
    There are some options:

    1: Remove the == override, and have all of the statements above break on references to destroyed objects. This is consistent with how C# works, but requires retraining all Unity developers, retraining all of Unity's developers, and breaks every single free and paid plugin that includes scripts.
    This also doesn't make either ? or ?? useful in Unity. It just makes things consistent over the board. It's not a very good solution.
    2: Don't change anything. This doesn't break anything, but since the elvis operator is a lot more useful than ??, I'm assuming that a lot more people will do this and despair:
    Code (csharp):
    1. transformThatHasBeenDestroyed?.Translate(direction); //MissingReferenceException
    Which means that no problem has been solved, and ? just can't be used in Unity (kinda like ?? really can't be used unless you're completely certain that the object will never be destroyed)

    3: Somehow convince Microsoft to have ? and ?? use the class' == operator instead of the default one to check for null. This is potential breakage in ALL .NET CODE OUTSIDE UNITY, so the chances are slim.

    4: Somehow convince Microsoft to make ? and ?? be overrideable on a per-class basis, and override it in UnityEngine.Object.
    This has the least impact on things, and solves all problems. On the other hand, even if it happens, it probably won't happen before C# 8, which is probably years off. Unless they're still taking suggestions for C# 7?

    5: Make Unity actually use UnityC#, which is a C# language that's equivalent to C# in every single way except that ? and ?? uses the class' == check. As an added bonus, the == check for object could be changed to do what UnityEngine.Object's == does.
    This option sounds really good, but is also completely off the walls insane, so idk.

    Is there any other option available?
     
    Mikael-H likes this.
  11. alexzzzz

    alexzzzz

    Joined:
    Nov 20, 2010
    Posts:
    1,439
    This is the only option I like. At first I thought it's kind of crazy, but then realized that non-nullable reference types are expected to be one of the main themes for C# 8. Since null-whatever-operators will have absolutely no sense for non-nullable types, the override might become quite handy even outside of Unity.
    Code (CSharp):
    1. public class Foo
    2. {
    3.     public static bool operator ?(Foo foo) => foo.IsValid;
    4. }
    PS
    Oops! It will never happen, because otherwise all the C# code all around the world will have to be recompiled. Not good.
     
    Last edited: Sep 29, 2016
  12. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    There could be another solution, Unity is quite keen on patching assemblies in general so it could potentially be possible to remove the custom null check on the language side and patch it in directly at the IL level.

    That should allow all the operators to function as expected, at the expense of having even more potentially unexpected behaviour.

    Still wouldn't help when accessing Unity objects through interfaces and the like though.
     
  13. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    Does anyone know how the compiler generates the code for the new ?. operator?

    For example, why doesn't this code work out of the box:
    Code (CSharp):
    1. transformThatHasBeenDestroyed?.Translate(direction); //MissingReferenceException
     
  14. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    Since transformThatHasBeenDestroyed isn't null, and ? probably uses the object's referenceEquals.

    The latest version of the C# specs I can find (here, not official, but the latest official ones are from 2012 and doesn't contain the operator), says this about the operator:

    Note that the mention is specifically "is non-null", not "== null resolves to false". So I'd guess the generated result is equivalent to:

    Code (csharp):
    1. if(!object.ReferenceEquals(transformThatHasBeendestroyed, null)) {
    2.     transformThatHasBeenDestroyed.Translate(Direction);
    3. }
    If this "worked" in Unity, ie. the line would be passed by if the object was destroyed, then Unity would somehow be breaking the C# specs.

    There is no solution to this that doesn't have major drawbacks. I think the UnityC# superset is the most appealing, but I've got no idea how much work that would be.
     
  15. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    I just ran this test and peeked at the generated code (using ildasm):
    Code (CSharp):
    1. .method private hidebysig instance void  Start() cil managed
    2. {
    3.   // Code size       31 (0x1f)
    4.   .maxstack  8
    5.   IL_0000:  nop
    6.   IL_0001:  ldarg.0
    7.   IL_0002:  ldfld      class [UnityEngine]UnityEngine.Transform TestScript::transformThatHasBeenDestroyed
    8.   [B]IL_0007:  brtrue.s   IL_000e[/B]
    9.   IL_0009:  br         IL_001e
    10.   IL_000e:  ldarg.0
    11.   IL_000f:  ldfld      class [UnityEngine]UnityEngine.Transform TestScript::transformThatHasBeenDestroyed
    12.   IL_0014:  call       valuetype [UnityEngine]UnityEngine.Vector3 [UnityEngine]UnityEngine.Vector3::get_zero()
    13.   IL_0019:  callvirt   instance void [UnityEngine]UnityEngine.Transform::Translate(valuetype [UnityEngine]UnityEngine.Vector3)
    14.   IL_001e:  ret
    15. } // end of method TestScript::Start
    16.  
    What happens (if i understand correctly) is that an IL instruction (brtrue.s) is used to determine if an object is not-null, and if so, jumps to the correct location to continue evaluation.
    This completely overrides Unity's (or in fact C# 's) operator evaluation and the overloading mechanism, so Unity's overloaded operator is not used in this case.
     
  16. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    BTW - this new operator will only work incorrectly in the cases where it is triggered against a destroyed object. How often do you think this behaviour is encountered by developers? for the most cases it should be fine, though i still think there's something that should be done about it.
     
  17. LaneFox

    LaneFox

    Joined:
    Jun 29, 2011
    Posts:
    6,673
    There was some chatter about this on Slack in the #code channel on Sep 27.

    My question was why the heck the 'Missing' even exists, because it should just be null. It was pointed out that the C++ side of the data can be destroyed but the C# side cant because with C# there is no way to find all references to an object and null them, hence the C# wrapper has to stay (missing/kinda dead) until you drop all of the references to it and GC hits it.

    Dunno if they're looking into a workaround for it or not.
     
  18. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    The vast majority of null-checks I write are actually destroyed-checks. Most of them are of the kind "handle that the object you were following/checking/managing got destroyed".

    I've got a hunch they are.
     
  19. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    IMHO, all proposed solutions are not very good ones (e.g: change the universe to make this work) or other hacks.

    it was said that it's not technically possible (or desired ? due to performance reasons), to scan for all references to a given object and invalidate them (make them null). If every native (C++) object holds a pointer to its managed (C#) counterpart, is it possible to nullify these (e.g: destroy the object) ? or is it the reference that should be nullified as well ?

    Another (ugly) suggestion - a UnityEngine.Object will have another reference (e.g: this.self or something similar). This will be monitored by the native side and nullified in case the object is destroyed.

    You would then check by something like:
    Code (CSharp):
    1. transformThatHasBeenDestroyed?.self?.Translate(Direction);
    Ugly as f** i know :)
     
  20. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,368
    The problem is not the reference that the C++ object holds to the C# object, or even the C# object itself - it is the references that _other_ C# objects hold to that C# object. Scanning every other C# object to look for potential references to the object being deleted... well, you're already familiar with what that looks like: it looks like the garbage collector. So how do you feel about running the GC every single time an object is deleted? :)

    That's an interesting idea - I'd probably want to call it something like "IfAlive()", so you'd do "transformThatHasBeenDestroyed?.IfAlive()?.Translate(Direction)." I think this is something you could already do with an extension method though.
     
  21. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    Yes, you can achieve that to work around the probem. It's just that this new operator was introduced for simplifying long (boring) code for null checking.. this kinda defeats the purposes (slightly), and also did i mention its pretty ugly? :)
     
  22. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    It's also side-stepping the problem a bit.

    Fundamentally what's going on here is the Unity uses some tricks to redefine the semantics of the language.

    In C#, the semantics of "a == null" is "a has not been assigned".
    In Unity, the semantics of "a == null" is "a has not been assigned OR the things a was assigned to has been destroyed"

    I believe that it would be better if Unity changed the semantics of "null", so all the parts of the C# language that works with the concept of null resolves that as "not assigned or destroyed".


    The only thing introducing an IsAlive() extension does is to push the responsibility of the semantic change from Unity to the user. It's not only ugly, it also essentially exactly the same as just removing the overload altogether and pushing the responsibility of handling destroyed objects to the user.
     
  23. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    I have to disagree :) a null reference is just that - a reference to nothing. Trying to expand that to other things only leads to confusion and other (negative) things.

    From my viewpoint - it won't be the end of the world if they dropped the overloaded equals (==) operator and asked me to check if an object is destroyed or null. Not that this will help this particular scenario anyway (as its not even using the overloaded operator).
    Sometimes you have to correct things that were done wrong in the past :)
     
  24. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,316
    I also was envisioning exactly liortal's solution. Add a new property to UnityEngine.Object that is actual null if the base is fake null. Something like:

    Code (csharp):
    1. Object Checked => { if (this == null) return null; return this; }
    Then you could do:
    Code (csharp):
    1. transformThatsBeenDestroyed?.Checked?.Translate()
    Granted we could do this with an extension method, but maybe there's some trickery Unity can do to make it faster, and you could also add a new compiler warning for anyone using ?? or ?. on a Component without following it with Checked. It's not a great solution but it's better than doing nothing.
     
  25. makeshiftwings

    makeshiftwings

    Joined:
    May 28, 2011
    Posts:
    3,316
    Well it's not EXACTLY the same as removing it; actually removing it would break every Unity project in existence. This is just adding a way for developers who know about "??" and "?." to have official support for those operators.
     
  26. cerebrate

    cerebrate

    Joined:
    Jan 8, 2010
    Posts:
    261
    This would be the best as it wouldn't require people to code any differently - old stuff would continue working as normal, and ?. and ?? would suddenly work as expected. However, this doesn't work if you're using external dlls. In that case, a Checked() extension method that returns self/real null works better.
     
  27. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    Why not? There's nothing stopping Unity from patching DLLs before loading their assemblies. I do wonder, does the automatic serialisation of UNET work if you compile to DLL and import? All that stuff is patched in.
     
  28. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,069
    If you're using non-Unity .dll's, that might break assumptions in those dlls. Like "If I do a null-check before putting an object into a list, I can safely assume that the list will never contain null"
     
  29. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    their code weaving is operated at the IL level from what i know. so it never modifies any "source code", only compiled code.

    In that case, they should have no issue patching DLLs that you throw at them, not sure if thats how it works though.
     
  30. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    Yes, but that is already the case if you are using anything deriving from UnityEngine.Object
    Yeah, as far as I'm aware it Cecils all that stuff in, on loading the assemblies in editor, and when building.
     
  31. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    Yeah, but that is already the case now with any UnityEngine.Object. Any injected null check modification would only be applied to when the operands derive from that base class. I stepped through the ILSpy code to see how it decompiles such expressions and all that information is available to it from the IL at the time of matching the expressions, so it's possible.
     
  32. Haapavuo

    Haapavuo

    Joined:
    Sep 19, 2015
    Posts:
    54
  33. joncham

    joncham

    Unity Technologies

    Joined:
    Dec 1, 2011
    Posts:
    256
  34. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    446
    I also feel that the time of the override == operator is over. It should have been over years ago when they were discussing it. The elvis/null coalescing op is too common outside of Unity. Overriding null is some cases and not others causes all sorts of unintended problems (performance and knowledge). I feel it is not worth the small convenience of having a IsAlive() or something similar that would be defined on the objects that need it. Obviously, this is a huge change and change is hard. Maybe have a Unity preference for enabling it in the player if people really need the legacy behavior. Going forward this is going to bring the scripting more inline with C# standards.
     
    Flavelius likes this.
  35. ldhongen1990

    ldhongen1990

    Joined:
    Dec 18, 2015
    Posts:
    61
    Oh dear, this is scary as hell, I just shot myself in the foot trying to do something like this in the new .NET 4.x api:

    Code (CSharp):
    1. shellPrefabRB = shellPrefabRB ?? (GameObject)Resources.Load(RES_FOLDER + shellPrefabRB_Res);
    where shellPrefabRB is actually a serialised field in the Monobehavior, and I have already written code like this all over a project (luckily not so much of them), now I have to go back and re-wire those code with the old null check format.
     
  36. hexagonius

    hexagonius

    Joined:
    Mar 26, 2013
    Posts:
    98
  37. Deleted User

    Deleted User

    Guest

    @joncham There is quite the discussion going on there. What do you think about it? Is this something that will be addressed or discussed in the future?
     
  38. ammirea

    ammirea

    Joined:
    Oct 22, 2018
    Posts:
    8
    Pardon me but any hackish improvisation deviating from standard is bound to backfire in ugly, devious ways at some point. And I'm talking about overriding "==" in the first place. Further prolonging the overload agony complicates things even more.
    Why call
    Let "null" mean "null", "==" mean "==" and "initialized with non-null yet non-usable value" be encompassed in something to mean just that, like "invalid" (as opposed to something working differently, hidden from the user).
    So instead of
    Code (CSharp):
    1. if(reference != null)
    let users do something like
    Code (CSharp):
    1. if(reference.isValid())
    or maybe
    Code (CSharp):
    1. if(reference != invalid)
     
    Qbit86 likes this.
  39. Flavelius

    Flavelius

    Joined:
    Jul 8, 2012
    Posts:
    778
    This behaviour, now with all the shiny runtime upgrades, still makes it hard to write code at times where you're for example have to work ontop of someone elses code and need to dive into whole inheritance hierarchies just to figure out if you're allowed to use standard language features. I hope Unity is still considering to rework this.
     
  40. liortal

    liortal

    Joined:
    Oct 17, 2012
    Posts:
    3,455
    Not sure what you mean by that comment....

    Why do u need to dig into someone else's code to know if u can safely use certain features?
     
  41. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    I assume he means checking if what you’re dealing with is derived from UnityEngine.Object, although I always found that a case of pressing F12 once.
     
    Last edited: Jun 22, 2019
    Flavelius likes this.
  42. K0ST4S

    K0ST4S

    Joined:
    Feb 2, 2017
    Posts:
    21
    What workarounds to use ?. for serialized gameobject fields are there?

    One I can think of: extension method for GameObject IsValid() which uses == null, and using it like this: obj.IsValid()?.Call(). However, it's not perfect.
     
    Last edited: Jul 2, 2020
  43. hexagonius

    hexagonius

    Joined:
    Mar 26, 2013
    Posts:
    98
    You can actually do
    Code (CSharp):
    1. if(gameobject)
    , which is overloaded and uses the overloaded == operator
     
    SirIntruder likes this.
  44. AlkisFortuneFish

    AlkisFortuneFish

    Joined:
    Apr 26, 2013
    Posts:
    847
    -ish, both call CompareBaseObjects(). It actually puzzles me a bit that operator bool is not just (object)o != null && IsNativeObjectAlive(o). Surely there's no need to go through all the conditionals in case the compiler doesn't just optimise it all away.
     
  45. Flavelius

    Flavelius

    Joined:
    Jul 8, 2012
    Posts:
    778
    Plus this doesn't help with the elvis operator.

    It's still something easily missed when working with external code (that sometimes you don't even have the source or license to decompile to trace an objects root type)
     
    goncalo-vasconcelos likes this.
  46. Prodigga

    Prodigga

    Joined:
    Apr 13, 2011
    Posts:
    910
    While not exactly the same issue as the new operators, I think it's worth bringing up the fact that interfaces suffer from the same problem. A null check against a reference to a Unity Object through an interface also bypasses the checks.
     
    Vivraan and goncalo-vasconcelos like this.
  47. darthbator

    darthbator

    Joined:
    Jan 21, 2012
    Posts:
    166
    I feel like there should probably be a documentation note about this at the top of the script ref or something. I just ran into this while implementing a property and it's pretty confusing. It's been a long time since I've been over the core docs but it would probably be very good to maintain some page of divergences from the current csharp standard within Unity.

    It might be even more ideal if there's some way to do a compile time detection and throw a warning within Unity. Something like "Hey you're trying to coalesce a serialized field and that won't work!". Currently it just results in a somewhat confusing runtime break.
     
  48. Deleted User

    Deleted User

    Guest

    Visual Studio really can help here. I am running 16.6.2 and
    Visual Studio 2019 Tools for Unity
    (4.6.1.0). Building the project in VS shows Unity specific issues in the output window, such as:
    info UNT0007: Unity objects should not use null coalescing
     
    Ghetaldus and triple_why like this.
unityunity