Large Scale RTS Battles

Hey that's quite good and it's using the in built navigation system nice, didn't think it was up to such a massive number of agents! Impressed.

 

Yes, I noticed that build in system is very fast also, as I do not need to use transform.Translate() anywhere. There are also no colliders (only agent Obstacle Avoidances). I also noticed that build in navigation is very stable, do not give any errors, and all units always stay on navmesh (what I wouldn't be able to say about some other popular navigation systems).

To check where is most computing power used you can compare runs enabling/disabling MeshRenderer component on units prefabs (this checks how much resources are used for rendering) + enabling/disabling Obstacle Avoidance (select none to disable).

However, there is also dark side of build-in navigation: it's not very dynamical (avoidances does not change actual paths - they just prevent units from moving along the path), problems on very large terrains, terrains can't be easily modified when NavMesh is baked (i.e. digging holes, smoothing surface during the gameplay), as it would require rebaking NavMesh, which takes several minutes.

Rendering is another issue. I think the real problem in Unity is not so big with polycounts as with number of draw calls. If you using models with 600 vertices, total largest vertices count for 10000 units can be 600x10000 = 6 million. However, I have scenes, where multiple meshes are merged onto single mesh to reduce number of meshes in the scene, i.e. this one:
https://dl.dropboxusercontent.com/u/248943005/TreeGen/TreeGenDemo.html
It reaches up to ~8 - 10 million vertices count in some crowded forest areas, but as draw call is about 100 - 150, I can run it with 50 - 60 FPS. The hope here is that some day Unity will find out a way how to increase draw call numbers much more to solve this old problem that you can't place many thousands of smaller meshes on the scene.

 

There is a feature I think of DirectX 9+ called instancing (see video below) but the new DirectX 12 and Mantle are very good at dealing with lots of drawcalls especially when they are for repeated meshes/materials.

And skinned instancing in DirectX 10

Or what Mantle or DirectX 12 should be able to do.

 

Great video on DirectX 12 and check out the performance boost they get in a demo about 54mins

 

Look interesting, but I guess in Unity it may take a while when support of such features will get implemented.

 

The have already stated that they will be bringing out a DirectX 12 compatible version of Unity in 5.x and they aim to have it ready in time for the Windows 10 release!

http://blogs.unity3d.com/2015/01/22/staying-ahead-with-directx-12/

 

It could be, but there might be that Unity draw calls system will not use full of DirectX features for a while. However, lets keep in mind this and keep monitoring rendering and draw calls if they will actually change.

P.S. I don't use Windows myself, so I might be looking also for some other ways how to bring larger amount of object into the scene.

 

Check out OpenGL. Their new version is aimed to match DirectX 12 / Mantle for low level access to the power of modern GPU's. This years GDC should be interesting!

 

As far as i know instancing is possible in openGL since 3.1. supporting few ealier versions is possible too thanks to one EXT but forgot full name

 

Going to revive this thread with a crowd rendering system I've been working on the past few months.

 

Nice! Is that in Unity?

 

This is really amazing. Please please please tell us that you have a demo available somewhere. I would love to try it.

 

Nice work. I can see from the demo that you are using LOD'ing so are you using Imposter sprites?

I presume you are using Instancing and some kind of GPU animation system, any details?

If you are combining Animated Instancing and LOD'ed Imposters, hat off that's good work, assuming you did it yourself and are not using an asset store solution.

Would love to know more details?

 

That's really, really impressive @angrypenguin . Any details how you achieved that?

 

That video isn't mine, it's @brilliantgames's.

I've done something similar, however - much higher unit counts, but a very different visual presentation. To achieve that, much like @brilliantgames (if you look at comments on the Youtube page) you need to avoid the overhead of GameObjects/Transforms per object. Separate the data parts of the work and the visualisation parts of the work and, depending on your project's specific needs, you can probably find really performant ways to approach each of those components.

 

There actually isn't any GPU instancing. I found this actually didn't speed things up enough to be worth it. I am simply avoiding transforms and game objects and resorting to my own physics solutions by using my own vectors and drawing the characters using 'Graphics.DrawMesh'. I made my own skinned mesh baker which automatically bakes all the skinned animations into solid meshes, as well automatically combines meshes using the same materials. When you get to this stage, you really start to realize just how expensive the combinations of skinned meshes with hundreds of bones(each counting as a transform) can be. Also, at this stage I'm not even sure the amount of polys makes much difference on solid character meshes. The characters are using LODs but it has a small impact on performance compared to when I use full detail. Using this method, it's all about reducing the number of meshes and materials being rendered. My character renderer automatically figures this all out, optimizes it at renders as few passes as possible(Each material and model has to call 'DrawMesh'.

 

Good work, brilliantgames. It is exciting work looking for ways to scale up the number of units in a scene.

 

Good point. I am also baking into static meshes and using DrawMesh. I also checked new instancing system and it additionally increases performance (I also have a video where I review some switching between Standard and Instanced shader (between 35:00 - 40:00), as well as other features). But I was facing issues with getting rid of GameObjects and Transforms completely. The point here is that usually together with large number of units comes navigation and pathfinding. Unity's build in navigation uses NavMeshAgent component, which I couldn't find how to replace without using gameObject per agent (anybody has any experience here?). I know A* Pathfinding Project allows to define non-component agents and calculate their velocities through scripts but if you using Unity's navigation, it seems to be a problem.

In the Unite 2016 keynote (starting at 8:08) they announced that they are updating Transform component. But I guess we will need to wait for version 5.6

 

Unity's built-in system is in no way designed for large-scale pathfinding. It's a generic solution, designed to be easy to drag-and-drop integrate with common usage scenarios. If you're doing something large scale you may well have to make something custom designed around your specific use case.

 

My units actually have full A* pathfinding at almost no expense to performance. The reason it has very little effect on performance is because they 'share targets', so a search algorithm is only used once considering the army usually has one broad goal. I have no shown off the pathfinding yet cause it's not ready to show. You may also notice they avoid eachother. How is this possible without using Unitys physics or navmesh? They operate on a grid. When they enter a new grid space they add themselves to that spot letting everyone know they are there. Because of this, all units are aware of their immediate surrounding enabling very fast character avoidance as well as choosing enemies.

The reason why I believe instancing is not making enough difference is because I believe all the performance loss I experience is on the CPU side. Using instancing reduces draw calls on the GPU side but the CPU is still working just as hard rendering the massive amount of individual models. Though the draw calls drop for me when using instancing, the FPS does not change because the CPU is still bottle necking. Does Unity not use a seperate CPU thread for rendering?

 

Hey guys. I uploaded a demo for a friend earlier, figured I would let you guys try it. 10,000 characters in this funny scenario of Romans Vs Orcs.

Once it's loaded you need to wait a few seconds for the characters to bake their animations.

-Press C to make them cheer
-Press K to start the battle.
-Press 9 to disable AI.
-Press 8 to disable animations.

Windows download: https://drive.google.com/file/d/0B7-D7l54O0QMc0M4bnQ2dUE4d1E/view?usp=sharing

 

Remade the battle for Minas Tirith. 15,000 characters in this one. New mounted units which can run over smaller units. Units have 'weight' now. Which means the more weight, the more the heavy characters will push away the smaller. Also in this video you will notice giant trolls which take up a great deal of space. Other units are able to avoid the large trolls successfully.

 

Impressive work and nice demo !!
If I get you well you combine meshes based on their material ? I like the idea a lot but how do you deal with vertices limit ? Unity Meshes in Unity cannot have more then 65536 vertices, because indices are 16 bit numbers internally.

 

So my crowd tech is now a game! And it's going pretty viral.

 

So this is unity right? And it will be on asset store RIGHT?

 

Yes it's Unity with my lighting and shader engine. I don't have plans to release the crowd system on the asset store any time soon as it is made for a game which has a bright future. Further down the road when the game has died down I may though. But part of my hesitation is supporting such a complex and massive code base.

 

Well if I understood correctly:
1a - you made all your calculus outside of unity, aka transform, animation then vertex position (dll?) and use drawmesh
1b - you made it into c# and use drawmesh
2 - you have your own pathfinding system

 

I'm loving the penguin one, for some reason...

 

I've done it finally. Pre computed crossfading between animations. Made everything look far more smooth. Also a few weeks ago I implemented CPU threading, which dramatically increased performance and numbers.

 

What are the specs of the computer running this? How does it scale to smaller computer?

Also character are basically duplicated, would it work with variation? (ie paperdoll part swap)
Would it work with arbitrary meshes?

 

that a very weird thinking from someone who seems experienced...
If you don't know how to optimize something doesn't mean engine can't do it lol.

 

Prove me wrong, or wait for the 2017 version?

 

??? This thread is full of proof, i don't get you.

 

Don't take my word for it!

@brilliantgames do you use Unity's out of the box terrain and physics system to move your Units?

 

yeah yeah i realise you specifically tried to do that with physics, you're right, blabla yada yada

 

@Arowx All the physics are made from scratch. It would be suicide to use Unitys. Plus, you can't run any Unity functions in another thread.

 

Dude, that's amazing - well done!

 

How big is the baked animation resolution? This demo ate almost 6 GB of RAM, which is huge for simple demo (simple scene-complexity-wise, not technology-wise) and it kind of surprised me.
Also, if that's not a problem to answer - how many tris/vert are Romans and Orcs models?
Thanks and keep up the good work!

 

Great work. Especially like how the bodies stay on the field and create a mound/wall.

Not sure the soldiers would be able to walk over it so easily as it would be slippery, rubbery and be very easy to get a leg or foot trapped in a gap (not to mention standing on your former friend).

I presume this is the Spartan 300 battle, surely the pass was narrower and there were fewer enemies?

 

I'm glad you brought this up cause I had forgotten about that demo. I removed it because of the growing popularity of this game and that could be a dangerous leak. So please don't share that.

That demo is very old. The system has been optimized a lot since then. That new massive Spartan battle I posted takes 3 gigs of ram. But this will be max settings. There will be a setting called 'animation quality' which will scale down the animation sampling, the lowest will be less than a quarter of the highest and the animations will appear choppy.

I'm not completely sure about tris/vert(They are fairly low at distant LOD), but I've learned the poly count has far less affect on performance than you would think. What affects performance the most is multiple materials. DrawMesh needs to be called for each material. A few characters(from the asset store) which I've used for this game have up to 4-5 materials which is brutal(Like rendering 4-5 characters). So I came up with a quick solution for this. Distant characters store colors in their vertices which match the colors of the textures used in their previous materials in low res, my shaders decode this and make them share one material.

 

Would you mind sharing some materials that you used for writing you own physics that can run on such huge scale?

Also, in current project's stage - do you use sprites for such large crowds or are all of those full 3D models (LODed of course)?

 

There's an asset (24h sale currently) that seems to be aimed at this kind of tasks:
https://www.assetstore.unity3d.com/en/#!/content/26009

I don't own it, so I can't comment on how it works.

 

This is awesome! Now I have to watch the whole Lord Of The Rings movie over again xD. It gave me goosebumps ffs.
Anyway, I am also working on a RTS project with a Game Artist friend of mine and we want to create something very original.
My project does not need that much Units as you have on your game. I really, really am interested in AI but I didn't get that much to learn about AI from my school, and their advises were always like "You should read a book about AI". I have did that,d I bought and read books like "Programming Game AI by Example" and "Game Coding Complete(Which covers a little bit about AI)". But they never explained me about an AI for a RTS. I am also trying to research this on the internet of course.

Is there anything you think of like. "Hey this really helped me understand to create an AI for a RTS game". And could you share that with me?

---
An example of some problems which I can't find a solution for:
-In our game the units walk in groups. You have captains and followers, like LOTR Battle for middle earth 2.
-Whenever you move the units the captain will move first and the followers will follow the captain.

My struggle is:
-How do I let the captain and the followers know which group units of the enemy to attack first(Which you can see on the picture).
-How do I make the units at the back wait till it is their turn to fight.
---

 

Of course I really thought this out and I got some solutions which might cause lots of performance issues.

-Check for every unit if it is near an enemy, something with sqrmagnitude .(Not every frame and also, what if I did have like 2000 units in the scene. That would drop the performance drastically.)
-Put for every unit a collision sphere. (From reading this thread, you actually don't want the Rigidbody getting involved with the RTS game. Since most part of the physics system would be unnecessary and it will drop the performance of the game. But same with the first solution. what if I did have more than 2000 units in the scene. This would also drop the performance drastically)

I have been already working on the prototype to figure out what does work and what does not. If you want to I could post a video to show how the games works for now.

 

Please Share Your result

 

Hey guys! I developed a unity tool that might interest you! Crowd Skinner was designed to render and animate large crowds. It uses GPU Skinning and GPU Instancing, along with the new Unity Entity Component System, to achieve incredible performance! On Full Mode, you can display 100 000+ characters at 60fps.

It's available now on the Asset Store : https://assetstore.unity.com/packages/tools/animation/crowd-skinner-129524

I would be happy to answer your questions,

Thanks

 

Can the animation be desync? It would look unnatural in most case.

 

Sure! Here all animations are launched at spawn time for demonstration purposes. But you still have full control over each character. In the video, you can see characters in the background that are desync, because spawned after.

 

On what hardware, at what resolution, etc.?

 

The video was captured on an i5-4690k quad core and a GTX 1070, at a 1920x1080 resolution. You can see it hits the 200 000 characters bar. That's why I think you can hit 100 000+ characters on a smaller config or a higher resolution.