EntityCommandBuffer and Dynamic Buffers

Is it possible to interact with dynamic buffers through an EntityCommandBuffer?

I'm hoping to do things like add/remove IBufferElementData's at specific positions in the buffer.

Thanks for any advice.

 

You don't need to use ECB in order to interact with the buffer as you want. There are methods to do what you want like

Code (CSharp):

1. myDynamicBuffer.RemoveAt(index)

It's advised to get the buffer on the entity with IJobProcessComponentDataWithEntity and add the field for your buffer to the Job struct.

Code (CSharp):

1. [NativeDisableParallelForRestriction]
2. public BufferFromEntity<MyBuffer> buffers;

Code (CSharp):

1. inputDeps = new MyJob{
2.             buffers = GetBufferFromEntity<TargetKnowledge>(),
3.         }.Schedule(this, inputDeps);

 

You probably have seen it, but just in case
[Example] IJobProcessComponentDataWithEntity and IBufferElementData

 

@riskparitygawd & @Antypodish, thank you very much for the leads.

From what I've read, GetBufferFromEntity<T>() doesn't play nice with IJobParallelFor, since you can't guarantee that parallel jobs wouldn't be appending (or otherwise writing to) a dynamic buffer at the same time.

I'm sure you know this, but EntityCommandBuffer avoids this issue by queuing entity operations, and deferring them until later.

It would be nice to have a similar way to queue and defer dynamic buffer operations, so that they happen deterministically, back on the main thread.

Am I correct in thinking this isn't currently possible? Most of the components in my project are IBufferElementData's, fwiw. So I'm potentially looking at never/rarely being able to use IJobPerallelFor. Please correct me if I'm wrong.

It feels a bit incongruent to work one way for IComponentDatas and another way with IBufferElementDatas. It would be nice to use the same logical approach for both types of components.

Thanks!

 

If you can guarantee you won't be writing to different buffers from different threads you just use

[NativeDisableParallelForRestriction]

This should work fine 99% of cases because you usually only access a buffer of the entity that it belongs.

 

Thanks! I must admit I am a bit hesitant to go that route. @Joachim_Ante wrote a good 4 sentences of caution before discussing it back in August.

I understand what he's saying here, and if you can absolutely guarantee nothing is writing to the Dynamic Buffer, it should be fine.

I may just need to live with the idea for a bit, and rule out other options, and then eventually become comfortable and use it.

 

What I'd really like to do is defer the work until later. For example, I'd like to run 40 jobs in parallel, which each give a deferred command to append an IBufferElementData to the Dynamic Buffer. Then later, in a BarrierSystem, those commands would be carried out in a deterministic way on the main thread. Same model as with IComponentData work.

...This is a complete shot in the dark, but maybe with the source available for EntityCommandBuffer, one could write their own version that works with DynamicBuffers. Something like a "DynamicBufferCommandBuffer", or just extension methods for EntityCommandBuffer. I'm not at a computer where I can take a look, so maybe this is crazy talk.

 

*shrug*

I have 23 uses of NativeDisableParallelForRestriction in my project. I don't consider it risky at all as it's very obvious when you won't have race conditions (though I am quite confident writing parallel code so that may play into it.)

If you're using IJobChunk or IJobProcessComponentData it'd be pretty rare for it not to be safe.

I'm not sure what type of algorithm you'd be using with IJobParallelFor because it's so rarely used now except for specialized cases that aren't usually 1 to 1 entity related in which case it probably isn't safe.

 

I'll look into this. Thanks.

Really, IJobParallelFor is rarely used now? I didn't know this- so much of what I've read on the forums seemed to imply it was the future, whenever possible.

 

There's no real reason to use IJobParallelFor anymore because it's replaced for most use cases by IJobChunk or IJobProcessComponentData

-edit-

was curious about my own project uses of IJobParallelFor

IJobParallelFor: 5%
IJobChunk: 8%
IJobProcessComponentData: 87%

 

My project I am working on, I use both IJobParallelFor (if I got just large arrays) and IJobProcessComponentDataWithEntity.
Also I use plenty of buffers with NativeDisableParallelForRestriction.

 

Thank you. It is hard to keep track sometimes, not having been tuned in for those changes when they happened.

- - -

Something I left out before, fwiw: I want to run many jobs in parallel which would each write to the same dynamic buffer. I specifically want these jobs to add things to it, so I imagine [NativeDisableParallelForRestriction] can't apply.

That might lead someone to say "Ok, well then your jobs shouldn't be running in parallel.". Point taken, but with something like EntityCommandBuffer, you *can* do this type of thing with IComponentDatas. I imagine that using the same deferred command approach would solve this problem for writing to DynamicBuffers as well. Please correct me if I'm wrong.

 

What's your use case for this? I'm struggling to think of a good one.

 

Here's a quick, dirty version. I hope it's detailed enough. Thank you for considering it:

- Entity A represents an Ai Agent.
- Entities B-D represent that agent's brains.
- System E reads state information about the brains, and add components to the agent, representing its active behaviors.
- Systems F-J read those behavior components, and carry out the behaviors.

It's valid for an agent to be performing more than one instance of the same behavior. That means multiple instances of the same behavior component need to coexist on the same entity. So those behaviors are defined as IBufferElementDatas, rather than IComponentDatas.

System E should ideally process all of the brains, for every agent, in parallel. So, there could be multiple jobs scheduled by System E which would be adding new behavior components to the same dynamic buffer of a single agent entity.

- - -

If that's just not possible, then I could have System E not run its jobs in parallel. That would work, but it would be slower. Or, I could reorganize the System so that it runs a separate parallel job for each agent, etc.

Or I could just restrict the design, and say agents can only perform one instance of the behavior at a time. That's also an option, but I'd like to avoid it if possible.

 

Ah the infamous AI system.
I've stayed out of this discussion because I don't like the solution you've gone with; it's far from what I've done and would advocate and others have been extremely helpful.
To me, it seems to run into way way too many issues like this and feels like your constantly battling ECS rather than working with it.

 

Let’s talk more in PM. I respect your opinion and would love to know your thoughts, if you’re up for it.

 

I assume it was
Porting a Generic Behavior System to ECS
Somehow was for me looking complicated, so I also didn't intervened, or I maybe was tired / busy.

 

I recommend instead of trying to buffer writes directly to component changes, create intermediate Entities that represent changes. This is what I plan to do and has the following benefits:

• Multiple systems can read from the state and use EntityCommandBuffers to write out the desired changes with basically no overlap.
• Separate pass can resolve conflicting changes, although if system processing queries are built correctly this should never happen and can be reasonably eliminated.
• Can run various effects based on the pending change entities before they're applied and deleted, such as spawning / despawning stuff, playing a particle system, cutting to certain animations, etc.

You can have a setup like so:

...other game logic... -> AI Process -> PendingAIChangeBarrier -> Scan changes and generate effect events -> ApplyAIChangeBarrier -> ...other game logic...

 

Yes, that was a start. And it looks too complicated to me too. You can tell by the post dates in that thread that the design was changing quickly day to day. That didn’t stop - I just stopped posting.

What I’m implementing now is a different animal. It has changed in large part due to the excellent advice on these forums. Nearly every time someone has told me I was “fighting ECS”, that resulted in a refactor based on their advice.

But of course it could still be bad. I have sincerely benefited from the critiques. You guys have been awesome.

 

That's fine.

 

That’s an interesting idea, recursive. I can’t see a real downside (besides maybe creating and destroying many entities each frame). Thank you.

 

Wait. @recursive: What approach are you taking to create these entities from inside of jobs? Are you using EntityCommandBuffers?

 

Yes. I'm currently trying to port/reimplement some of the debug rendering from FPSSample to the LWRP and then I'm going to work on the design I've sketched out. I've used it before in some prototyping code but my plan is to build a frameowork to do the following:

Depending on player state, evaluate a simple set of Gameplay logic state datas. Some states may have sub-states that form a Hierarchical state machine, for example:

Airborne contains: Jumping, Falling, Launched, Bounced, Air Attacking
Ground Sticking contains: Standing, Idling, Moving, Ground Attacking, ect
Moving contains: Walking, Running, Dashing.

And so on and so forth. But the nice thing is I can use groups, chunk component type existance queries, etc to write individual systems that just take certain combinations of data into account and define what part of the state machine we're in. It's the approach Described in the Data-Oriented Design Book here: https://smile.amazon.com/dp/1916478700/. (there's an online text dump here but it doesn't contain all the chapters and sections, including the FSM chapter).

Each state has a unique list of "Input Combination Patterns" with control over delays for things like fighting game inputs, as well as platformer concepts, that that state allows. This prevents the issues older design I had in non-ECS, where I had a hard-coded input logic check generate events and post them as notifications, and states the may or may not actually support the events being generated.

Most of what I'm going to implement first is platformer gameplay state machines first, but then start working on AI state machines after I have the initial issues worked out, as the basic principles are the same.

More complex Enemy AIs will be built to just generate similar input buffers to the player and use different versions of the state machine archetypes. Simpler one's that are more "hazard-ish" will just have bespoke/common states, like classic platformer enemies.

 

That was a very cool read. Thanks for sharing your plans.

I'm sure you know this, but it's a fun topic: Ai in ECS is an interesting frontier. The internet is full of people trying to figure it out, since many of the established patterns don't seem to intuitively translate ...Certainly makes things interesting. What you've described is probably the most successful attempt I've seen to replicate a FSM pattern in ECS.

The most successful ai ECS approaches I've seen share a common principle: They use a designated group of "planning" systems as a switchboard for other, later, "behavior" systems. In other words: SystemA alters an entity's state, which then determines if it's considered by Systems B, C, and D (which actually do some work). Your design also seems to have that in common.

Good luck, I hope you post more about it when you have time to implement it.