Agent communication & Self-pay

My environment it is just a matrix made of stationary cube agents, that change their color (state). I am trying to obtain a matrix where the cubes of the same color get together, but without getting all the matrix of the same color. For this, I have been trying lots of things, but right now I think that the only way to do this its to let the agents of the same colour communicate between each other. How can I do that?

Also, what is the self-play value, can I use it to train all the cubes as one team, or that does not make sense?