Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Feature Request Output data from PopEventForConnection to native array/slice instead of a DataStreamReader

Discussion in 'Unity Transport' started by PolarTron, Apr 4, 2022.

  1. PolarTron


    Jun 21, 2013
    I want to pop an event in a job and store a pointer to the internal sub array of that data so that I can later parse the data in other jobs.

    I cannot use the DataStreamReader with ReadBytes in this case because I am inside a job and cannot allocate temp memory (with ReadBytes) and return that as it's not allowed.
    Passing in a pre-allocated block of memory to the job increases code complexity and I really don't want to do that.

    I've made a tiny modification to NetworkDriver to work around this. It's probably unsafe.

    My ParsePacketsJob
    See how PopEventForConnection now outputs a NativeArray instead of DataStreamReader

    Code (CSharp):
    1. while ((cmd = Driver.PopEventForConnection(Connections[index], out NativeArray<byte> bytes)) != NetworkEvent.Type.Empty)
    2.                 {
    3.                     if (cmd == NetworkEvent.Type.Data)
    4.                     {
    5.                         var reader = new DataStreamReader(bytes);
    6.                         PacketType type = Packets.ReadPacketType(ref reader);
    7.                         reader.SeekSet(0);
    9.                         ReceivedMessages.Add((int) type, new PacketArrayWrapper()
    10.                         {
    11.                             Pointer = bytes.GetUnsafeReadOnlyPtr(),
    12.                             Length = bytes.Length,
    13.                             InternalId = Connections[index].InternalId
    14.                         });
    15.                     }
    16.                     else if (cmd == NetworkEvent.Type.Disconnect)
    17.                     {
    18.                         Debug.Log("Client disconnected from server");
    19.                         Connections[index] = default(NetworkConnection);
    20.                     }
    21.                 }
    Modification of NetworkDriver.cs

    Code (CSharp):
    1.             public NetworkEvent.Type PopEventForConnection(NetworkConnection connectionId, out NativeArray<byte> bytes)
    2.             {
    3.                 return PopEventForConnection(connectionId, out bytes, out var _);
    4.             }
    6.             public NetworkEvent.Type PopEventForConnection(NetworkConnection connectionId, out NativeArray<byte> bytes, out NetworkPipeline pipeline)
    7.             {
    8.                 pipeline = default;
    10.                 bytes = default;
    11.                 if (connectionId.m_NetworkId < 0 || connectionId.m_NetworkId >= m_ConnectionList.Length ||
    12.                     m_ConnectionList[connectionId.m_NetworkId].Version != connectionId.m_NetworkVersion)
    13.                     return (int)NetworkEvent.Type.Empty;
    15.                 var type = m_EventQueue.PopEventForConnection(connectionId.m_NetworkId, out var offset, out var size, out var pipelineId);
    16.                 pipeline = new NetworkPipeline { Id = pipelineId };
    18.                 if (type == NetworkEvent.Type.Disconnect && offset < 0)
    19.                     bytes = m_DisconnectReasons.GetSubArray(math.abs(offset), 1);
    20.                 else if (size > 0)
    21.                     bytes = ((NativeArray<byte>)m_DataStream).GetSubArray(offset, size);
    23.                 return type;
    24.             }
    Last edited: Apr 4, 2022
  2. simon-lemay-unity


    Unity Technologies

    Jul 19, 2021
    I understand your use case and will bring it up to the team, but unfortunately I'm not sure this is a change we'll be willing to make. The data stream API serves an important role for us in mediating access to our internal data structures.

    To process data in a separate job, I'd recommend just passing the
    itself to the other job. In your example code, that would mean modifying
    to store a
    instead of a pointer and length. All readers obtained when popping data events are valid until the next update job is scheduled.

    (Also small aside regarding the example code; I would advise against using
    . It's not a unique connection identifier, since the internal ID of a connection can be reused after it's closed. I'm not sure why it's part of our public API, honestly. It would be preferable to store
    structures directly. They're meant to be small and usable as value types.)
    PolarTron likes this.
  3. PolarTron


    Jun 21, 2013
    Aha. I now see where things went wrong. I assumed that DataStreamReader was only meant to be used in the current scope in a temporary way. Storing and passing DataStreamReader around feels weird but I now see that it's a more elegant and proper solution to what I'm trying to achieve. Thank you for the response :)
  4. PolarTron


    Jun 21, 2013
    One problem with passing in DataStreamReader to a job is that DataStreamReader.m_bufferPtr produces errors because I am passing an unsafe pointer to the job.

    I cannot use [NativeDisableUnsafePtrRestriction] because it has to be applied on the m_bufferPtr field itself inside of DataStream.cs

    This attribute will not do anything and it will still produce errors about using an unsafe pointer.

    Code (CSharp):
    1.     [BurstCompile]
    2.     public unsafe struct ParsePublicSnapshotJob : IJob
    3.     {
    4.         [NativeDisableUnsafePtrRestriction] public DataStreamReader Reader;
    Using Entities 0.50
  5. simon-lemay-unity


    Unity Technologies

    Jul 19, 2021
    Oh, good catch! I'll file a task on our end to fix this. Thanks for raising this!
    PolarTron likes this.