Resolved Ring buffer while minimizing sync points

Hello,

Goal:

I'm trying to create a ring buffer of entities in Unity DOTS while minimizing sync points, e.g. while using an entity command buffer (ECB).

Failed plan:

My plan was to use an ECB to append entities (wrapped in buffer elements) to a dynamic buffer up until the ring buffer limit, and then start assigning instead of appending, starting with index 0. However, I don't think I can use an ECB to assign elements to a dynamic buffer. And, I can't immediately assign a deferred entity, since it will have Entity Index -1.

Project details:

I need to create entities of a particular archetype and keep track of the order in which I create them. Once I create enough entities, I need to delete the oldest one to make room for a new one. This is one reason why I need to keep track of the order, but there are other project-specific reasons to know the order, too.

New plan:

- Job 0: Defer entity creation. Also add `NewlyCreatedTag` and `DynamicBufferIndex`
- Playback ECB from Job 0.
- Job 1: Iterate through `NewlyCreatedTag` entities, wrapping each in a dynamic buffer element and assigning it to the dynamic buffer according to its `DynamicBufferIndex` component. Remove `NewlyCreatedTag`.

Elaboration of new plan:

I need a reference to the oldest entity so I can destroy it, to destroy its PhysicsCollider and other no-longer-needed data in the world.

I will assign the new entities to the dynamic buffer after the playback, where the entity references will be available, while still aggregating all the deferred entity destructions, entity creations, and component additions into one sync point.

However, since I still don't explicitly have the new entity references from the playback, I will add a `NewlyCreatedTag` and a `DynamicBufferIndex` component to the new entities (recorded in Job 0), so I can recover the new entity references after the playback.

I invite thoughts and advice.

 

Is there any reason you need a ring buffer for your particular use case instead of just a queue which you could do with just ECBs AppendToBuffer and a RemoveAt(0) on the buffer

 

In this project, any entity from this sequence of entities can be selected via a random raycast, and then either neighboring entity needs to be referenced. To enable this, I've stored each entity's dynamic-buffer-index in a component on the entity, where I can then add one or subtract one. Using a queue, these indices would change.

To enable the queue approach, I could store the index offset in a singleton component and add it when I add or subtract one. This would prevent me from having to update 20,000 index components each time I append an entity. A queue seems like it will work. Thank you.

 

How about fixed list instead of dynamic buffer?

 

How about this:

• You have your DynamicBuffer<EntityElement> ringbuffer with a pre-initialized size
• You have a separate DynamicBuffer<RingBufferAddCommand> that lives alongside the ringbuffer
• RingBufferAddCommand just has an Entity field

The idea is that jobs will use an ECB to append entities to the DynamicBuffer<RingBufferAddCommand> instead of appending directly to the ring buffer. A separate job right after ECB playback will then handle transferring the Entities from the RingBufferAddCommands to the ringbuffer

Spoiler: Details

So more specifically, right after the ECB that creates entities & appends them to DynamicBuffer<RingBufferAddCommand> is processed, a single-threaded job does the following:

• goes through all RingBufferAddCommands and handles adding them correctly to the ringbuffer (assign the entity to the correct index).
• Also handles setting the added index in a component on the added entities
• Use an ECB to handle the destruction of entities that got replaced in the buffer. Depending on your needs, that ECB could be processed in an already-existing CommandBufferSystem, so that it doesn't need to create an additional sync point.
  • Or, this whole job could just be done on the main thread and do all destruction instantly, since we're already right after the unavoidable sync point of entity instantiation anyway
• Clears DynamicBuffer<RingBufferAddCommands> once everything has been processed

 

I like this approach because it uses a second dynamic buffer to defer assignments instead of using structural changes to do so (`NewlyCreatedTag`). This approach also avoids having to shift N (e.g. 20,000) integer-pairs (Index:Version) to the left each time a new entity is created. In a way, it makes its own unofficial ECB to add the assignment functionality that I was after. Thanks, Phil.

Spoiler: Footnotes

I'll have the RingBufferAddCommand processor only assign the newly created entity. I can destroy the old entity and set the dynamic buffer index component via the original ECB. This will let me process the RingBufferAddCommands in parallel. There will be an invalid reference to a destroyed entity in the dynamic buffer for a moment, but it will be replaced with the new entity reference before anything uses it. I might store the dynamic buffer index as part of the RingBufferAddCommand so I don't need to use GetComponentDataFromEntity<DynamicBufferIndex>(true); the info will just be available in the command.