NativeStream.Writer.Allocate Allocation size is too large Exception

Hello Everyone,

im trying to allocate memory using NativeStream.Writer.Allocate function,

Code:

Code (CSharp):

1.  
2.              
3.                 BatchesStreamWriter.BeginForEachIndex(index);
4.                 var matricesArray = matricesList.AsArray();
5.                 var matricesArrayByteCount = matricesArray.Length * UnsafeUtility.SizeOf<float4x4>();
6.  
7.  
8.                 var batch = new BatchData
9.                 {
10.                     BatchCount = matricesList.Length,
11.                     RenderMeshSharedIndex = ChunksRenderMeshIndices[ChunksSortedIndices[ChunksStartIndex]],
12.                     MatricesPtr = BatchesStreamWriter.Allocate(matricesArrayByteCount)
13.                 };
14.  
15.                 // copy Data to the Pointer
16.                 UnsafeUtility.MemCpy(batch.MatricesPtr, matricesArray.GetUnsafeReadOnlyPtr(), matricesArrayByteCount);
17.                
18.                 BatchesStreamWriter.Write(batch);
19.                 BatchesStreamWriter.EndForEachIndex();
20.                
21.                 matricesArray.Dispose();

Error:

Code (CSharp):

1. ArgumentException: Allocation size is too large
2. Unity.Collections.NativeStream+Writer.AllocateChecks (System.Int32 size) (at Library/PackageCache/com.unity.collections@0.5.1-preview.11/Unity.Collections/NativeStream.cs:450)
3. Unity.Collections.NativeStream+Writer.Allocate (System.Int32 size) (at Library/PackageCache/com.unity.collections@0.5.1-preview.11/Unity.Collections/NativeStream.cs:360)
4. Unity.Rendering.CustomHybredRendererSystem+FrustumCullingJobV2.Execute (System.Int32 index) (at Assets/Scripts/CustomHybredRendererSystem.cs:222)
5. Unity.Jobs.IJobParallelForExtensions+ParallelForJobStruct`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at <f38c71c86aa64e299d4cea9fb7c715e1>:0)
6. Unity.Jobs.JobHandle:ScheduleBatchedJobsAndComplete(JobHandle&)
7. Unity.Jobs.JobHandle:Complete()
8. Unity.Rendering.CustomHybredRendererSystem:OnUpdate(JobHandle) (at Assets/Scripts/CustomHybredRendererSystem.cs:334)
9. Unity.Entities.JobComponentSystem:Update() (at Library/PackageCache/com.unity.entities@0.5.1-preview.11/Unity.Entities/JobComponentSystem.cs:120)
10. Unity.Entities.ComponentSystemGroup:UpdateAllSystems() (at Library/PackageCache/com.unity.entities@0.5.1-preview.11/Unity.Entities/ComponentSystemGroup.cs:182)
11. Unity.Entities.ComponentSystemGroup:OnUpdate() (at Library/PackageCache/com.unity.entities@0.5.1-preview.11/Unity.Entities/ComponentSystemGroup.cs:169)
12. Unity.Entities.ComponentSystem:Update() (at Library/PackageCache/com.unity.entities@0.5.1-preview.11/Unity.Entities/ComponentSystem.cs:107)
13. Unity.Entities.DummyDelegateWrapper:TriggerUpdate() (at Library/PackageCache/com.unity.entities@0.5.1-preview.11/Unity.Entities/ScriptBehaviourUpdateOrder.cs:152)

 

Did you look into the code? Max allocation size seems to be set at around 4k. Perhaps the stream isnt ideal for what you are doing here?

Code (CSharp):

1. internal unsafe struct UnsafeStreamBlockData
2. {
3.     internal const int AllocationSize = 4 * 1024;

 

:/ that's bad!
it seems like there is no other way to allocate memory inside bursted jobs.
thanks!

 

Come up with a way where you can pre-allocate the memory you need up front and pass that buffer into the job? I'm doing that in plenty of cases.

 

in my case i dont really know the memory i have to pre-allocate, im using it for FrustumCullingJob.
if i have to allocated for the worst case which is all entities with a possible visibility (100000) it can be really bad in term of performance. knowing it have to run flawlessly for low-end devices.

 

Can't you split this into a few jobs? Like first jobs find and counts what you need to use and returns a list of indexes. Then you allocate the buffer based on the number of returned indexes and second job then copies the relevant entities into the buffer.

 

i got it working with UnsafeUtility.Malloc!
thanks for the help!