Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Question Example of AsyncReadManager.Read for multiple files in Burst Job?

Discussion in 'Burst' started by MNNoxMortem, Jul 18, 2020.

  1. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    723
    Hi!

    I am looking for examples regarding AsyncReadManager.Read to read multiple files from within a burst job.
    • What is the idea/concept/benefit of using multiple ReadCommands vs a single large read command in AsyncreadManager.Read? Was designed with having one large database file in mind and maintaining all indices/offsets within this file our own and then working on subranges within this file?
    • How to effiicently load multiple files with AsyncReadManager?
    • Is FixedString{Size} the correct new string struct to be used to pass the file path into a bursted job?
     
  2. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    When streaming entity binary files we want to stream directly into the chunks. Those might not have contiguous memory addresses. Thus we will read a file linearly but with multiple read commands.

    Larger read commands are always better, wherever that is possible.
     
    MNNoxMortem likes this.
  3. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    723
    Thank you for the explanation. That makes a lot of sense.

    Is there any plan for a variant of AsyncManager.Read that does support any of the string types as filename that is supported via Burst? Currently I am getting (expected as strings are not fully support per documentation)
    Code (CSharp):
    1.     [BurstCompile]
    2.     public struct ReadAsyncFileJob : IJobParallelFor
    3.     {
    4.         public NativeArray<ReadCommand> ReadCommands;
    5.  
    6.         [ReadOnly]
    7.         public NativeArray<FixedString512> Paths;
    8.  
    9.         [WriteOnly]
    10.         public NativeArray<ReadHandle> ReadHandles;
    11.  
    12.         public unsafe void Execute(int index)
    13.         {
    14.             ReadHandles[index] = AsyncReadManager.Read(Paths[index].ToString(), (ReadCommand*) ReadCommands.GetUnsafePtr() + sizeof(ReadCommand) * index, 1);
    15.         }
    16.     }
    17.  
    18. [Error] E:\repositories\visionary\Visionary\Assets\Plugins\AsyncFileJobs\AsyncFileJobs.cs(22,4): Burst error BC1016: The managed function `Unity.Collections.FixedString512.ToString(Unity.Collections.FixedString512* this)` is not supported
    19. at Plugins.AsyncFileJobs.ReadAsyncFileJob.Execute(Plugins.AsyncFileJobs.ReadAsyncFileJob* this, int index) (at E:\repositories\visionary\Visionary\Assets\Plugins\AsyncFileJobs\AsyncFileJobs.cs:22)
    20. at Unity.Jobs.IJobParallelForExtensions.ParallelForJobStruct`1<Plugins.AsyncFileJobs.ReadAsyncFileJob>.Execute(ref Plugins.AsyncFileJobs.ReadAsyncFileJob jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, ref Unity.Jobs.LowLevel.Unsafe.JobRanges ranges, int jobIndex)
    21. While compiling job: System.Void Unity.Jobs.IJobParallelForExtensions/ParallelForJobStruct`1<Plugins.AsyncFileJobs.ReadAsyncFileJob>::Execute(T&,System.IntPtr,System.IntPtr,Unity.Jobs.LowLevel.Unsafe.JobRanges&,System.Int32)
    22. at E:\repositories\visionary\Visionary\Assets\Plugins\AsyncFileJobs\AsyncFileJobs.cs:line 22
     
  4. MNNoxMortem

    MNNoxMortem

    Joined:
    Sep 11, 2016
    Posts:
    723
    @Joachim_Ante Any update reading blob files from BurstedJobs?