Search Unity

Bug CreateBlobAssetReference Allocator

Discussion in 'Entity Component System' started by Soaryn, Apr 30, 2021.

  1. Soaryn

    Soaryn

    Joined:
    Apr 17, 2015
    Posts:
    328
    It would seem that the method `CreateBlobAssetReference` in `BlobBuilder` takes an allocator, but only uses that allocator in one spot. The Native arrays `offsets`, `sortedAllocs`, and `sortedPatches` force an Allocation.Temp. Is there a reason for this as this prevents this from being done in a Task outside of the job system or main thread.

    At a quick glance, using the allocator passed in for the first two native arrays seems to work, but I was unsure as to what all would be affected by this change. Alternatively, would it be possible in the future to allow Allocator.Temp to work in threads created by the user that are not necessarily in the job system.


    Usecase: I have a thread receiving json data from a 3rd party websocket api. I'd like to parse it and create blob reference on that thread before passing it onto the job system to utilize. However, if I attempt to call `var builder = new BlobBuilder(Allocator.TempJob)`, I am met with "Could not allocate native memory. If this allocation was made from a managed thread outside of a job, you must use Allocator.Persistent or Allocator.TempJob."
     
    Last edited: Apr 30, 2021
  2. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,356
    Unity does internal Temp allocations like this in a number of places. What would you have them do here? Complicate their api with say an additional allocator?. Maybe there is a good option here but it's not obvious to me atm what that would be. The actual allocation message means the restriction itself is by design. They probably have a good reason for that with Temp, like not wanting to setup state for threads they don't create.

    I would just pass the json ref to a Unity job via a static ConcurrentQueue if you just have this one scenario. Probably no practical gain in complicating it much beyond that.
     
  3. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,262
    Not of lot of people know that HeapString exists, but that's a good way to get a really long string into a job. Once the string is in a job, you can disable Burst and work with string types directly if you need to.
     
  4. Soaryn

    Soaryn

    Joined:
    Apr 17, 2015
    Posts:
    328
    Note, I already have a solution working to do this in main thread / job system as a psuedo byte array for passing around and form back into a string in a non-bursted job (I receive it as a byte array anyway), but I was trying to determine why this particular use was restricted to jobs only as it is for creating data. While it is appreciated, work arounds to this aren't really helpful to the question.

    As a simple change, would be to use the passed in allocator from the existing method `CreateBlobAssetReference<T>(Allocator allocator)` to the NativeArrays (GRANTED I have no idea how this affects the rest of the blob building, although it did work upon testing. I'd like Unity to identify if this would be detrimental at all).


    I may look into these a little more, but my solution currently works as needed.