Generically Applying Networked Component Data Diffs?

Anyone have some thoughts on how to generically send entity component data across the network? My non-generic solution involves a network message that looks like...

Code (JavaScript):

1. {
2.     PlayerHistories: [{
3.         Translation: //...
4.         Rotation: //...
5.         Metadata: //...
6.     }],
7. }

The client picks up that message, writes each PlayerHistory to the appropriate entity's PlayerHistory buffer. Works the same way for every kind of history, and every kind of history contains different components. Each entity archetype has its own kind of history. However, when it comes time to write the historical data from the buffer to the entity itself, I don't see a way around explicitly specifying each type that needs added to the Entity; I have to SetComponentData(x, componentType) for each unique component type I define in each history. This also means I have to create and specify a different history type for each archetype of entity I have.

I'd prefer to be able to specify an arbitrary list of components and have the appropriate Entity sync with them. If an Entity with a particular GUID is a player one server tick, but then is an projectile on the next tick, that should work. In the above example, that would involve having BOTH a PlayerHistory and a ProjectileHistory buffer on the Entity and consuming the correct one based on the time/frame. That can't be right. It seems like there should be a way to generically add / remove / modify the appropriate components based on the network message.

I thought about storing one historical component type per buffer and sending a network message more like this:

Code (JavaScript):

1. {
2.     Translations: [],
3.     Rotations: [],
4.     Metadata: [],
5.     // ... every unique possible component type
6. ]

Where each index in each component array is specific to an Entity. Then, the client (using a map of Entity -> Index) can iterate through every Entity, check the index, then add / remove / set each component as needed. This keeps the concept of archetypes out of the network message, but potentially involves some very sparse arrays, since a certain component might only be used twice in a scene with 1000 entities. It also means that the consuming code has to do a `ComponentDataFromEntity<T>.Exists(e)` for every single kind of component type for every single Entity, which feels bad. Nevermind the fact that you have to explicitly specify every CompnentDataFromEntity because you can't create them from a runtime type, AND have to explicitly call each one since you can't just toss them in a collection and iterate them. This feels bad too, since you are manually repeating the same .Exists?/.Add/.Set logic over and over.

To summarize, I'm looking for a way to apply arbitrary components received over the network to the appropriate entities without explicitly writing new code for every archetype. Let me know if you have any ideas. Thanks for reading!

 

I've opted for a model where the replicated components of an entity can't change during runtime, you can create any combination of components when you spawn the entity but when it's picked up by the network system the actual replicated ones for a specific entity can not change.

Supporting the case of "add/remove" of a replicated component on an entity is very memory consuming and bandwidth consuming, and becomes very complicated when dealing with rollbacks, etc.

This simplifies the replication of entities a whole lot. Now my target is an FPS type of game, and the case where I need to add/remove replicated components is very rare/non existant, so YMMV.

 

About generic serialization, this is also possible but requires a lot of fiddling with the lower levels of the ECS API, getting raw void* pointers to buffers, calculating field offsets, etc.

 

Oh, huh, that model makes a lot of sense actually. Luckily I don't think I care about add/remove component for rewinds, since my rewound state just needs the component data itself from that time, rather than requiring the whole Entity be reversed to that exact state for consumption.

I would be interested in more on ECS built in serialization if you have any resources; up until now, my understanding was serialization only happens at the World level.

I actually don't think add/remove component needs to be a huge pain, either. As long as I don't have to shuffle them around to leverage rewinds, Add/Remove component accepts typeof(X), so I can just check if the last frame did/didnt have a given component if I have the right information.

I'm actually kind of thinking about making my network messages slightly ridiculous, like...

Code (JavaScript):

1. {
2.     Entities: [{
3.         Components: [{
4.             type: typeof(Translation),
5.             data: object
6.         }, {
7.             type: typeof(Rotation),
8.             data: object
9.         }]
10.     }]
11. }

With this kind of data, as long as you can check the last message, seems like it'd be pretty quick to figure out if a component was removed or not, and you have the type on hand without specifying a field, so you can just loop through the components and call into Add/RemoveComponent(entity, type) as needed based on the history. The trickier part is the generic-ness of SetComponentData - you can't call it without a concrete compile-time type. I got this working trivially with a dynamic method, but I have no idea what the actual cost of that will be in practice. I suppose the other alternative is listing all possible networked components at startup, then using reflection to create some cached delegates that can handle SetComponent for the various types. There is also probably an equivalent to this doable via code generation, which I imagine is faster in practice.

Thanks for the reply; I was feeling stuck and this got me thinking about it in different ways, which helps a lot.

Edit: Of course, "dynamic" does not play nice with burst since it boxes the value. Could preprocess the message before the job? Ehh...