Managing a high amount of entities

Greetings !

I got started prototyping a game. The game is a strategy game with quite a big scale (in terms of space on the map and amount of units at one time), which means I have to plan from the get-go how I am going to handle entities in a way that supports having at least multiple thousands of them (perhaps not in a "loaded" state, I'll explain later).

The problem : Unity's integrated GameObject system is no good for very high amounts of entities because of the overhead involved with managing extra GameObjects, the necessity not to use the integrated physics system (at least not for all entities), and the necessity not to render all entities all the time.

What I want to achieve : Good performances when the game is managing thousands of entities, most of them being VERY simple entities like harvester drones, ressource collection structures, projectiles... . In my opinion, a good reference game for this is Planetary Annihilation, in which matches can easily go over one thousand entities at the same time, all of them being loaded and somewhat simulated properly on physics (projectiles hit whatever they happen to hit, not always their intended targets, and units can't go through one another).

All that in a game that is played mostly in third person view of a special unit called the Brood Commander, with the possibility to switch to a traditionnal RTS view in some cases.

Solution so far : None of it is tested or even implemented, but it's what I feel would perhaps work out, but that is where I need your input : are there any other ways which I haven't thought about ? How can my current solution(s) be improved ?

One solution involves using GameObject to render and simulate physics for entities that "matter" : mostly medium to big moving entities or big non-moving entities that are close to the player or one of the opponent AIs. For these entities, we want to make absolute certain physics and hitboxes are somewhat accurate. The rest of the entities are simulated through a vastly simplified physics system and are either not rendered at all or rendered through a Graphics.DrawMesh call. Furthermore, entity interactions are optimized by using a "Cell" system on the map : entities are kind of "sorted" in memory depending on their location. This allows, for example, not checking distance with every other entity in the game when looking for a target or checking a bullet's physics. Really big entities could be present on multiple cells at once.

Pros :

- Simpler to use Unity's integrated systems for entities that really matter to the immediate gameplay experience of the player (such as rendering and physics).

Cons :

- Need to make a good loading / unloading system for entities. This one seems mandatory in any case.

- Need to take into account the many differences between using the custom made entity / physics system for unloaded and unimportant entities and using a GameObject for the other entities which will have to, somehow, interact with the entity in order to "tell it" when something hit it for example.

- Might not be enough : what if we end up with a big concentration of entities that are considered "important" in one place ? It is a common strategy to "mass" units together before launching an attack.

------

The other solution involves not using GameObjects at all, but rather render ALL entities that are close enough using Graphics.DrawMesh and simulate physics for ALL entities using a rather simple, custom made system.
Entity interactions are optimized in the same way as the other solution.

Pros :

- Everything being "custom made", it is easier to account for differences between unloaded / loaded and important / unimportant entities.

- Might be more optimized than the first solution for high concentrations of important entities which could happen during battles.

Cons :

- Much harder to do overall. Unless all models have somewhat primitive hitboxes, need to re-invent the wheel in some places (such as multi-collider hitboxes).

- No idea how to handle animations using only Graphics.DrawMesh.

- Might not be precise enough for proper immediate gameplay experience (especially hitboxes). If it is not, will need to do much of what Unity is already doing in a very probably more optimized way, which would mean I wasted quite a lot of time for little gains, which is not suitable for a prototype.

There you go ! As said before, if you have any other solutions to propose with its pros and cons, I'm all ears. Same with feedback about my current solutions and what I think are their pros and cons.

In the meantime, I think I will choose the first solution, though I have to work on world generation first so it might be some time until I start working on entities.

 

Your description would perfectly fit the new Unity ECS approach, great for rendering and managing tens of thousands of objects on screen.

https://unity3d.com/unity/features/job-system-ECS

 

Oh my, how did I miss that ? I'll take a look ! Thanks a lot !

... But I'm still open to feedback on my solutions I also do this to learn. Maybe someday I will move out of the Unity nest and will need to fly on my own ;(