Question How does training multiple agents simultaneously work?

Hi, I've used the TrainingAreaReplicator to have multiple agents training in a single scene and I'm wondering how this works under the hood?

Are there multiple networks which get merged periodically?
Is there a single network that tracks every agent and their individual weights and histories somehow?
Or is every call to learn and decide completely independant and decoupled (no historical data)?