GPU instancing count limit

Hey, I've created a billboard lod shader for trees and grass and I have quite a lot of them in a scene and they are procedurally generated, I have to start looking at Graphics.DrawMeshInstanced, Graphics.DrawMeshInstancedIndirect or similar to draw them but I'm not quite sure about the instancing count limit as to what the limit is referring to. Does the instancing count limit affect how many different materials I can instance or does it affect how many different instances of the same material there are... As the billboard shader has a vertex modifier to face the camera so I have to instance an id for every vertex modification and does this affect the instancing count?

Or is there possibly another way to draw a lot of trees and grass billboards that are facing the camera without it killing all the performance?

 

When rendering instanced with DrawMeshInstanced there is a limit of 1023 instances per drawcall. This is the max amount of Matrix4x4 you can have in the list/array.
one call is for one submesh with a single material, max 1023 instances.

With DrawMeshInstancedIndirect there is no limit.

Just remember that you need to handle frustum culling and LOD selection when drawing like this.

 

It's a limit of 1023 instances per DrawMeshInstanced call. An instance is one copy of the entire mesh.

So, if you need to draw 5000 copies of the same mesh with the same material, you'd need to call DrawMeshInstanced 5 times. Vertex count or shader complexity doesn't factor into the limit.

There is still a limit, it's just much higher.

 

Ah, thanks, I really did understand the limit wrong, I though it was more inline that there can be this many materials instanced at any given time.

 

Sorry to necro this thread, but I had a question which is basically identical but related to what you mentioned here bgolus - what exactly is that "much higher" limit to DrawMeshInstancedIndirect?

I've noticed that in my scene if I have too many instances being rendered using DrawMeshInstancedIndirect (across multiple objects, each with their own script and args buffer), some of my instances start vanishing, but only in my build on Mobile.

I'm using DrawMeshInstancedIndirect basically as a vertex / geo shader on steroids, so each instance is six triangles and corresponds to one triangle of the original mesh I started with. Once I have more than perhaps 10,000 or so triangles in my scene (i.e.10,000 instances of 6 verts + 6 tris each), I start losing tris. It also looks like those tris are being truncated in sequence which is why I think I'm running up against some undocumented limit.

I added a screenshot (through the lens on an Oculus Quest) to clarify...

(P.S. feel free to scold me if I've been a naughty boy reviving this thread - I'll make a new one)

 

This actually isn't the same problem.

Different platforms have different limits, but those limits are per-draw call, not total. The fact you're missing instances after some threshold from multiple draws confuses me a bit since they should all be independent of each other.

At the same time I'm having a strong sense of deja vu with this... unfortunately I can't remember the specifics, or even if they found a cause.

 

I'm on Oculus Quest.

 

On 2019.1.14f1 - yeah I noticed yesterday when doing some more testing that sometimes (not deterministic) a few of my instanced verts get moved wayyy far away rather than just being missing. So that might fit into the category of 'glitchy' mentioned by that issue...

 

OK I've run a more useful test today, here's the method, as well as a little more detail about what I'm actually doing with DrawMeshInstancedIndirect:

Overall method:
Take in a mesh, and create four ComputeBuffers:

• One that basically just has a mirror of the entire mesh (each struct has vertex position, normal, etc)
• One that contains a list of triangles (each struct has the three ints that make up the tri in the original mesh, plus some additional per-triangle data)
• One that contains a list of output vertices - I create six verts per original triangle, so the length of this buffer = 6 x the second compute buffer count

Then each frame I run one compute shader on the triangle buffer, using the indices to manipulate data in all three buffefrs, then another compute shader on the output vertex buffer, doing some more per-vertex manipulation.

Then I basically use that output vertex buffer in addition with an args buffer to do the instanced indirect drawing, where each 6-vertex 'chunk' of my buffer corresponds to one instance.

Test method:

• Created a prefab object that has an icosphere mesh with 960 verts and 320 tris. This creates an instance buffer with 320 items and an output vertex buffer with 320 x 6 = 1,920 items.
• Slowly added prefabs to the scene one by one, counting the total number of instances / instanced verts+tris as I go
• Watch for when I start losing instances.

Results:
I have no instance loss when my instance count is 7680 (6 vertices / 6 triangles per instance = 46,080 each in total)

When my instance count moves up to 8000 (so 48,000 verts/tris total), I immediately lose a handful of instances in basically random 'chunks' throughout all my meshes. The biggest missing chunk I noticed was perhaps 10 instances all in a row that weren't drawing. The smallest chunk I saw was a single missing instance. The specific instances that go missing seems to be random, and each time I lost between 20 and 50 instances.

When I reduce the instance count again, the missing instances do not return - they're gone for good.

Each time I add additional instances beyond that limit of 7680, I lose another handful of instances, but I seem to lose less and less each time - after adding hundreds of thousands of instances, I still was only missing perhaps 500 or so individual instances.

---

So yeah I have no idea what's going on - each individual prefab that I was spawning was a separate batch / separate compute buffer / separate call to DrawMeshInstancedIndirect. But they all seem to be affecting each other when it comes to passing over the limit of instances that's somewhere between 7680 and 8000 instances (or 46,080 and 48,000 instanced verts/tris).

The fact that they don't return when the count drops back down makes me think that my compute buffers are being messed with somehow. But their manipulation functions are analytical so there's no state that could be getting messed up...

 

One more update: I ran a final test, this time using a higher per-instance vertex count (to see if the problem was being caused when I passed a threshold on instance count, or on vertex count).

The conclusion: it's neither. I've officially realised that I shouldn't have posted in this thread as it's definitely a bug somewhere in the Unity -> Oculus -> Adreno pipeline.

Some of the strange effects that are happening:

• Instances are corrupted, not dropped - when an instance vanishes, I've now realised they are actually having some of their vertex positions multiplied by some large number (I can see them off in the distance)
• Instances are corrupted in chunks of 1-24 sequential instances. I can't discern any pattern to the clumping. Here, 20 instances are corrupted together, then there are thousands of instances being drawn correctly, and then a few more are corrupted together, then some correct ones, then a single corrupted one, etc.
• Instances seem to be corrupted probabilistically, and over time. The probability of an instance chunk being corrupted increases with the total vertex count. Once I pass the threshold for the current per-instance vertex count, chunks of instances begin to get corrupted, increasing to a maximum after 5-10 seconds. Every time I increase the number of instances being rendered each frame, a new period of instance-chunk-corruption begins.

Also, note that for 12-verts-per-instance (previously it was 6), the threshold of corruption is now 7,040 instances (80,640 instance-drawn vertices)

Note that the threshold isn't firm though - a few times (quite rarely during my testing) I've seen instance chunks getting corrupted at half or lower than the 'normal' threshold for the given per-instance-vertex-count.

So yeah - calling it there in terms of gathering more knowledge for myself. Workaroundable if I just stick to not rendering more than a few thousand instances, but still pretty disappointing given the whole point of instanced rendering is to be able to render lots and lots and lots...

@aleksandrk, let me know if you could use any more info etc, would be happy to help if it does turn out to be some bug on Unity's end. I might reach out to my contact at Oculus as well and see if their techs know anything about this...

 

I'll try to get on that but it might not be for a little while since I don't really have the couple-hours-to-spare it'll take to isolate the system without uploading a ton of code and IP under NDA for the client I'm working for...

 

Hi,

@aleksandrk we are experiencing the same issue as @mptp. I've made a little repro not relying on NDA.

Reproducible on:
Unity 2018.40f1, 2019.1.2f1, 2019.3.0b3
Mobile devices with Mali GPUs

In the example project I'm using DrawMeshInstancedIndirect to draw just grid of cubes. On Mali processors Mali-G71 and Mali-G72 we start missing instances and experience weird behavior when total instances per frame are over 252324. (When you increase count of instances, you must decrease number of draw calls and vice versa).

https://issuetracker.unity3d.com/is...69.1091740624.1569220857-946675193.1552998634 it seems that in 2017.3.0f3 it was working there

As @mptp pointed out, limit also depends on mesh that you are instancing.

Issue 1186522