Resolved Casting ref structs with string properties fails if done in the first FixedUpdate call.

On the first frame of fixed update generic ref struct fields are not handled properly when cast between a generic and non generic ref struct that stores the generic ref structs state as a Span<byte>.

Casting fails if you attempt to access the not generic ref structs span value.

In the attached unity package you can start the game and see that the incorrect value is printed out for the first frame. You will also see that it will be incorrect for every frame if the non generic ref structs span is accessed.

The example is an empty scene so there is not much load on the renderer but in my experience anytime a FixedUpdate occurs without a corresponding update the values will be incorrect.

I suspect that any reference type as a property on the ref struct will cause this behavior, not just string.

Any solutions other than avoiding reference type keys on the first frame?

 

I won't be able to view your unity package until tomorrow, but ref structs cannot be fields. So, your description of the problem is inherently inaccurate.

https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/ref-struct

• A ref struct can't be a declared type of a field of a class or a non-ref struct.

 

Please post your code, not an entire package. I‘m sure this is just some false expectation or a bug in your scripts, probably due to casting.

 

Firstly, I wanted to apologize if my assertion that your description was inaccurate seemed harsh. I only meant to say that if the issue is not described accurately then nobody will be able to provide you a proper answer. We are programmers, we must do things 100% accurately or they will not work. We should also (try to) describe things 100% accurately if trying to get a useful answer from a forum. I myself make mistakes all the time, but if you're going to get these structs to do what you're attempting to do then you'll eventually need to remove any error, including from your understanding and descriptions. Sorry, if it sounded any other way. However, I think you will probably need to take a step back to think about what you're trying to do, and why are you trying to do it this way?

Code (CSharp):

1.     void RefStructTest(float value)
2.     {
3.         float originalValue = value;
4.         RefStruct<float> originalRefStruct = new RefStruct<float>(originalValue);
5.         RefStruct castedRefStruct = originalRefStruct;
6.         string inspectionResults = InspectRefSpan ? " bytes: " + ContentsToString(castedRefStruct.Span) : default;
7.         RefStruct<float> recastedRefStruct = castedRefStruct;
8.         float castedValue = recastedRefStruct;
9.         Debug.Log($"Frame {frame} RefStruct Expected value: {originalValue} Actual value: {castedValue} {inspectionResults}");
10.     }

These are not fields. These are local variables. That difference in terminology happens to be directly relevant when using any type of ref struct.

After inspecting the code you provided, there are many things I don't understand, but in my response I will try to ignore anything that doesn't pertain to what I think you are trying to achieve. I'm going to ignore the KeyedRefStruct structs completely because I don't know what their actual intended purpose is, or how they are different from the RefStruct sructs for the purposes of this example.

I'll start with RefStruct`1.cs. I would avoid having your generic and non-generic types share the exact same name. Then you don't have to have these strange file names. Something like RefStructOfType<T> is what I prefer.

Code (CSharp):

1. using System;
2. using System.ComponentModel;
3. using System.Runtime.InteropServices;
4.  
5. public readonly ref struct RefStruct<T> where T : unmanaged
6. {
7.     #region Unsupported
8.  
9.     [EditorBrowsable(EditorBrowsableState.Never)]
10.     public override bool Equals(object obj)
11.     {
12.         throw new NotSupportedException();
13.     }
14.  
15.     [EditorBrowsable(EditorBrowsableState.Never)]
16.     public override int GetHashCode()
17.     {
18.         throw new NotSupportedException();
19.     }
20.  
21.     #endregion
22.  
23.     public RefStruct(T state)
24.     {
25.         State = state;
26.     }
27.  
28.     public readonly T State;
29.  
30.     public static implicit operator T(RefStruct<T> snapshot)
31.     {
32.         return snapshot.State;
33.     }
34.  
35.     public static implicit operator RefStruct(RefStruct<T> snapshot)
36.     {
37.         var state = snapshot.State;
38.         Span<byte> span = MemoryMarshal.Cast<T, byte>(MemoryMarshal.CreateSpan(ref state, 1));
39.         return new RefStruct(span);
40.     }
41.  
42.     public static implicit operator RefStruct<T>(RefStruct snapshot)
43.     {
44.         var span = snapshot.Span;
45.         T state = MemoryMarshal.Cast<byte, T>(span)[default];
46.         return new RefStruct<T>(state);
47.     }
48. }

It seems this struct does only one thing, stores the value of an unmanaged type. Again, I'm not sure why you need this struct (like what is it actually going to be used for), or why you have overriden object.Equals(object) and GetHashCode() to throw exceptions. But, that's beside the point as far as I can tell. For the purposes of your test case we have a generic readonly ref struct that stores the value of an unmanaged type.

Now for RefStruct (non-generic).

Code (CSharp):

1. using System;
2. using System.ComponentModel;
3.  
4.  
5. public readonly ref struct RefStruct
6. {
7.     #region Unsupported
8.  
9.     [EditorBrowsable(EditorBrowsableState.Never)]
10.     public override bool Equals(object obj)
11.     {
12.         throw new NotSupportedException();
13.     }
14.  
15.     [EditorBrowsable(EditorBrowsableState.Never)]
16.     public override int GetHashCode()
17.     {
18.         throw new NotSupportedException();
19.     }
20.  
21.     #endregion
22.  
23.     public RefStruct(Span<byte> span)
24.     {
25.         Span = span;
26.     }
27.  
28.     public readonly Span<byte> Span;
29. }

This one is strange to me. It is a readonly ref struct that accepts another readonly ref struct, Span<byte>, and then holds onto that value. Why does this struct exist? Why wouldn't you just use the pre-existing Span<byte> struct anywhere that you would theoretically need RefStruct? It's just a concrete wrapper around the inflated generic Span<T>.

So, when implicitly converting from a RefStruct<T> to a RefStruct (which is really just Span<byte>), you do the following. I just added my comments inline.

Code (CSharp):

1. public static implicit operator RefStruct(RefStruct<T> snapshot)
2.     {
3.         // I prefer T state here to var state. I think it's more readable for you to know your own types and not leave it up to the compiler to determine for you.
4.         var state = snapshot.State;
5.         // MemoryMarshal.CreateSpan() is now creating a Span<float> for the local float value that is on the stack here within this implicit conversion operator.
6.         // I assume this is the crux of your whole issue. How can the local float here within this implicit operator, that will no longer exist once you leave be what you want?
7.         // Then you cast from Span<float> to Span<byte>, but both spans are still Spans of the local memory on the stack. How can this be what you want?
8.         Span<byte> span = MemoryMarshal.Cast<T, byte>(MemoryMarshal.CreateSpan(ref state, 1));
9.         // It seems like you're trying to use some kind of trickery to violate the rules of ref structs, namely that they cannot leave the stack.
10.         // Maybe you're close, in that it's not throwing exceptions, but it's just not allowing you to break the .NET laws.
11.         // Returning a Span<byte>, which is basically what we're doing here with RefStruct, that refers to a local variable on the stack here, is not valid.
12.         return new RefStruct(span);
13.     }

I just feel even more confused at this point. I feel like at least one of us really doesn't understand what you're trying to do here. I'm sorry I can't get more specific, but I feel like I've looked at this as long as I can.

 

If you're going to pass unmanaged value types, like float, by reference instead of passing by value, then you need to actually pass them with ref. Again, I'm not sure if these tidbits of information will really fix what you're trying to do here, but it seems like there are some things going at cross purposes. I don't have direct experience with using the MemoryMarshal class, but my instinct is that you are passing value types by value into an implicit conversion operator, then trying to make a Span over that memory and return it, which you normally wouldn't be allowed to do because the compiler would point out that you can't return a Span that refers to local memory on the stack that is going to be destroyed as soon as you return, and then you use the MemoryMarshal methods to create enough versions of that Span that it gets confused and doesn't know how to tell you that what you're trying to do is illegal. If you are trying to create a Span over the original float, it would have to be passed in by ref, not by value. However, I don't think that's the root issue. I think if you took a step back and described what it is that your're actually trying to do, maybe someone could suggest a way to do it that doesn't involve all of these steps you're attempting.

 

https://learn.microsoft.com/en-us/d...ervices.memorymarshal.createspan?view=net-7.0

Warning: Even though the ref is annotated as scoped, it will be stored into the returned span, and the lifetime of the returned span will not be validated for safety, even by span-aware languages.

 

Thank you. I'm glad you are taking the time to answer.

A ref struct's fields* is what I meant.

This isn't how I would typically name/organize a project. This was purely for example. I was trying to recreate the problem in as simple a manner as possible.

This is used to store snapshot data of a frame for network reconciliation purposes. The data is stored using a special collection type that only accepts unmanaged data but the unmanaged data itself can come in many forms. The classes responsible for transcribing the data to store work with the generic ref struct and the collection works with the nongeneric ref struct. The overrides are just so anybody using my library that edits the source code is aware that they can't make use of those methods in a ref struct. I accidentally left those in when I copy and pasted parts of the ref struct. Your summary of the generic ref struct is accurate.

The actual non generic ref struct in my program contains key data for storage and retrieval in a collection. I just omitted that in order to show the issue without adding in anything extra. I think you are maybe a bit too focused on the example code which isn't my actual implementation, its just there to be able to demonstrate the issue when you hit play.

This makes sense to me but then it begs the question: Why do RefStruct<T> to RefStruct conversions not have the issue? All the code you are referencing is of RefStruct and Refstruct<T> but these work perfectly every time.

The issue only occurs for the KeyedRefStruct conversions, specifically when you are attempting to do the operation during a FixedUpdate call that does not have a preceding update call.

If you do the conversions via a console app there is no issue.

If you do the conversions after the first update call when your update occurs as frequently or more frequently than fixedupdate there is no issue.

The issue only occurs on fixedupdates that were not preceded by an update when using KeyedRefStructs.

If you remove the FixedUpdateLoop from the PlayerLoop system and use this code while manually simulating physics there is also no issue.

I agree with you that the issue seems to be due my copying of a local variable during the implicit conversion but if that is the case I would think that the issue should be present for RefStruct and RefStruct<T> conversions.

 

Code (CSharp):

1.  
2. using System;
3. using System.ComponentModel;
4.  
5.  
6. public readonly ref struct RefStruct
7. {
8.     public RefStruct(Span<byte> span)
9.     {
10.         Span = span;
11.     }
12.  
13.     public readonly Span<byte> Span;
14. }
15.  

Code (CSharp):

1.  
2. using System;
3. using System.ComponentModel;
4. using System.Runtime.InteropServices;
5.  
6. public readonly ref struct RefStruct<T> where T : unmanaged
7. {
8.     public RefStruct(T state)
9.     {
10.         State = state;
11.     }
12.  
13.     public readonly T State;
14.  
15.     public static implicit operator T(RefStruct<T> snapshot)
16.     {
17.         return snapshot.State;
18.     }
19.  
20.     public static implicit operator RefStruct(RefStruct<T> snapshot)
21.     {
22.         var state = snapshot.State;
23.         Span<byte> span = MemoryMarshal.Cast<T, byte>(MemoryMarshal.CreateSpan(ref state, 1));
24.         return new RefStruct(span);
25.     }
26.  
27.     public static implicit operator RefStruct<T>(RefStruct snapshot)
28.     {
29.         var span = snapshot.Span;
30.         T state = MemoryMarshal.Cast<byte, T>(span)[default];
31.         return new RefStruct<T>(state);
32.     }
33. }
34.  

Code (CSharp):

1.  
2. using System;
3. using System.ComponentModel;
4.  
5. public readonly ref struct KeyedRefStruct
6. {
7.     public KeyedRefStruct((int, string) key, Span<byte> span)
8.     {
9.         Key = key;
10.         Span = span;
11.     }
12.  
13.     public readonly (int, string) Key;
14.     public readonly Span<byte> Span;
15. }
16.  

Code (CSharp):

1.  
2. using System;
3. using System.ComponentModel;
4. using System.Runtime.InteropServices;
5.  
6. public readonly ref struct KeyedRefStruct<T> where T : unmanaged
7. {
8.     public KeyedRefStruct((int, string) key, T state)
9.     {
10.         Key = key;
11.         State = state;
12.     }
13.  
14.     public readonly (int, string) Key;
15.     public readonly T State;
16.  
17.     public static implicit operator T(KeyedRefStruct<T> snapshot)
18.     {
19.         return snapshot.State;
20.     }
21.  
22.     public static implicit operator KeyedRefStruct(KeyedRefStruct<T> snapshot)
23.     {
24.         var state = snapshot.State;
25.         Span<byte> span = MemoryMarshal.Cast<T, byte>(MemoryMarshal.CreateSpan(ref state, 1));
26.         return new KeyedRefStruct(snapshot.Key, span);
27.     }
28.  
29.     public static implicit operator KeyedRefStruct<T>(KeyedRefStruct snapshot)
30.     {
31.         var span = snapshot.Span;
32.         T state = MemoryMarshal.Cast<byte, T>(span)[default];
33.         return new KeyedRefStruct<T>(snapshot.Key, state);
34.     }
35. }
36.  

Code (CSharp):

1.  
2. using System;
3. using UnityEngine;
4.  
5. public class RefStructConverter : MonoBehaviour
6. {
7.     public float testValue = 5f;
8.     public bool InspectRefSpan;
9.  
10.     int frame = default;
11.  
12.     private void FixedUpdate()
13.     {
14.         if (frame <= 2)
15.         {
16.             RefStructTest(testValue);
17.             KeyedRefStructTest(testValue);
18.         }
19.  
20.         frame++;
21.     }
22.  
23.     void RefStructTest(float value)
24.     {
25.         float originalValue = value;
26.         RefStruct<float> originalRefStruct = new RefStruct<float>(originalValue);
27.         RefStruct castedRefStruct = originalRefStruct;
28.         string inspectionResults = InspectRefSpan ? " bytes: " + ContentsToString(castedRefStruct.Span) : default;
29.         RefStruct<float> recastedRefStruct = castedRefStruct;
30.         float castedValue = recastedRefStruct;
31.         Debug.Log($"Frame {frame} RefStruct Expected value: {originalValue} Actual value: {castedValue} {inspectionResults}");
32.     }
33.  
34.     void KeyedRefStructTest(float value)
35.     {
36.         float originalValue = value;
37.         KeyedRefStruct<float> originalRefStruct = new KeyedRefStruct<float>(default, originalValue);
38.         KeyedRefStruct castedRefStruct = originalRefStruct;
39.         string inspectionResults = InspectRefSpan ? " bytes: " + ContentsToString(castedRefStruct.Span) : default;
40.         KeyedRefStruct<float> recastedRefStruct = castedRefStruct;
41.         float castedValue = recastedRefStruct;
42.         Debug.Log($"Frame {frame} KeyedRefStruct Expected value: {originalValue} Actual value: {castedValue} {inspectionResults}");
43.     }
44.  
45.     string ContentsToString<T>(Span<T> span)
46.     {
47.         string str = string.Empty;
48.  
49.         str += "[";
50.  
51.         for (int i = default; i < span.Length; i++)
52.         {
53.             str += $"{span[i]}";
54.  
55.             if (i < span.Length - 1)
56.             {
57.                 str += $", ";
58.             }
59.             else
60.             {
61.                 str += "]";
62.             }
63.         }
64.  
65.         return $"{span.Length} element(s): {str}";
66.     }
67. }
68.  
69.  

Output:
Frame 0 RefStruct Expected value: 5 Actual value: 5
Frame 0 KeyedRefStruct Expected value: 5 Actual value: 0
Frame 1 RefStruct Expected value: 5 Actual value: 5
Frame 1 KeyedRefStruct Expected value: 5 Actual value: 5
Frame 2 RefStruct Expected value: 5 Actual value: 5
Frame 2 KeyedRefStruct Expected value: 5 Actual value: 5

This is example code, not an actual implementation. It is intended to be the minimum amount of code needed to recreate the issue when you hit play.

I believe the poster above has made a correct point about the implicit conversion but I am still interested in why the error only occurs when using a KeyRefStruct during a FixedUpdate call that did not have a preceding Update call. Note: The error does not occur when I tested it in a console app in release mode.

 

I see. I was definitely getting caught up in the details. I think I understand the big picture better now. You're creating a collection that maps all kinds of unmanaged variables to keys by saving Spans over the memory of the original values that are already allocated on the stack. Is that right? Or is it just that you want the collection to handle any unmanaged type without boxing them as objects? In any case it sounds like an interesting problem.

I'm not sure if I will be able to look at it further unfortunately, so I'm sorry if I haven't been much help.

 

Yes. The collection itself has a preallocated array it uses to store the data by copying the span to the array, but the original source of the data is on the stack.

It's both. I want to avoid runtime heap allocations and prevent boxing.

I appreciate you taking the time to try.

Any ideas as to why the implicit conversions don't fail for the RefStruct and RefStruct<T> types? You're answer seems correct to me but the results don't really reflect that and I'm wondering if you and I both are misunderstanding what is going on there.

Any ideas who I could contact for more help with this?

 

You could also try StackOverflow. It's not a Unity issue.

 

Is it not? It only occurs for me within unity and under specific conditions related to update/fixedupdate timing.

 

Sorry. I'm typing on my phone so my replies aren't coming out right. I only meant to say that I don't think it's tied to Unity. That's just my gut feeling. I understand that it pops out under certain conditions within Unity.

 

CodeRonnie was correct it was due to the span<byte> being created based on a stack variable.

Here is my solution, it requires the use of a ref T in the ctor for the generic ref struct.

Code (CSharp):

1. using System;
2. using System.ComponentModel;
3. using System.Runtime.InteropServices;
4.  
5. public readonly ref struct KeyedRefStruct<T> where T : unmanaged
6. {
7.     public KeyedRefStruct((int, string) key, ref T state)
8.     {
9.         Key = key;
10.         Span = MemoryMarshal.Cast<T, byte>(MemoryMarshal.CreateSpan(ref state, 1));
11.     }
12.  
13.     public KeyedRefStruct((int, string) key, Span<byte> span)
14.     {
15.         Key = key;
16.         Span = span;
17.     }
18.  
19.     public readonly (int, string) Key;
20.     public readonly Span<byte> Span;
21.  
22.     readonly T State => MemoryMarshal.Cast<byte, T>(Span)[default];
23.  
24.     public static implicit operator T(KeyedRefStruct<T> snapshot) => snapshot.State;
25.     public static implicit operator KeyedRefStruct(KeyedRefStruct<T> snapshot) => new KeyedRefStruct(snapshot.Key, snapshot.Span);
26.     public static implicit operator KeyedRefStruct<T>(KeyedRefStruct snapshot) => new KeyedRefStruct<T>(snapshot.Key, snapshot.Span);
27. }

Code (CSharp):

1. using System;
2. using System.ComponentModel;
3.  
4. public readonly ref struct KeyedRefStruct
5. {
6.     public KeyedRefStruct((int, string) key, Span<byte> span)
7.     {
8.         Key = key;
9.         Span = span;
10.     }
11.  
12.     public readonly (int, string) Key;
13.     public readonly Span<byte> Span;
14. }

Frame 0 RefStruct Expected value: 5 Actual value: 5
Frame 0 KeyedRefStruct Expected value: 5 Actual value: 5
Frame 1 RefStruct Expected value: 5 Actual value: 5
Frame 1 KeyedRefStruct Expected value: 5 Actual value: 5
Frame 2 RefStruct Expected value: 5 Actual value: 5
Frame 2 KeyedRefStruct Expected value: 5 Actual value: 5