[SOURCE INCLUDED] MeshInstanceRendererSystem with MaterialPropertyBlock support

I needed to use some per-instance customization on my sprites and started messing with it.

I've seen some solutions in the forum, but wanted to learn ECS internals so I did my own take.
Also this one keeps the same semantics as classic instanced shaders, no need to rewrite them.

So I thought I share it. All relevant data in the README.
https://bitbucket.org/2bam/meshinstancerendererwithmaterialpropertyblock




I've also sort of dissected the official MeshInstanceRendererSystem, I'll share it also because anyone trying to figure out what's going on might benefit from it.
(Disclaimer: It's not thorough and might contain some mistakes)

Best regards!


What does MeshInstanceRendererSystem actually does

VisibleLocalToWorld
ComponentData as transform matrix cache. If chunk version is not changed, uses this directly.

FrustumPlanes
Utility from camera to check if bounds in/out/partial

MeshInstanceRendererSystem ComponentSystem

• unsafe CopyTo Copy NativeSlice of VISIBLE float4x4 to managed "Matrix4x4[]" array with memcpy
  
  
• OnCreateManager&OnDestroyManager Add component dependencies and cache component groups (for chunk iteration). Split in groups for "frozen" (static) & dynamic meshes.
  
  
• UpdateFrozenChunkCache/UpdateDynamicChunkCache.
  Makes a big HashMultiMap (100k elems). This is a hard limit for chunks amount.
  Worst case 100k meshes (all w/diff SCD), best case ~100M meshes (all same SCD/archetype).
  Chains MapChunkRenderers, GatherSortedChunks and UpdateChunkBounds jobs (read below) and waits on them.
  
  
• UpdateFrozenInstanceRenderer/UpdateDynamicInstanceRenderer.
  Schedules chained CullLODToVisible and PackVisibleChunkIndices jobs (read below). Waits them.
  Then iterates the contiguous sorted array, storing in a batch and then flushes on "context" changes.
  By that I mean it batches by max-size (1023 limit), shared component uniqueness (pre-sorted) or flipped-winding.
  "batchCount" is the offset inside the batch, "activeCount" is the total matches in a chunk query.
  
  Possible bug: Chunk sizes always below 1023 assumed as no batching steps are taken to handle chunks bigger than this. "fullBatch" would detect it, but no code for slicing is present. Probably has to do with 16KiB limit on chunks (having float4x4 caps it at 1023 entities per chunk), but if chunk size changes it might become a problem.
  
  
• OnUpdate Calls above funcs.

UpdateChunkBounds Parallel Job
Processes chunks to get boundaries PER CHUNK. Expands instances from chunks to get every bounds component.
Updates spheric boundaries by all instances (meshes) [I think?] inside chunks.

MapChunkRenderers Parallel Job
Associates unique MeshInstanceRenderers' shared component data indices to a list of chunk indices that that exact shared data.

GatherSortedChunks Single Job
Translates the multi-map (from MapChunkRenderers) to a contiguous sorted array of chunks sharing same MeshInstanceRenderer data.
Brute forces all SharedComponentData indices because there is no way to enumerate keys in MultiHashMap.
This allows later functions to do a "run length encoding" to then run bigger batches of matrices for DrawMeshInstanced.

CullLODToVisible Parallel Job (HACKY!)
Checks if bounds inside frustum, LOD, and copies only visible meshes transforms to VisibleLocalToWorld component data.
If chunk bounds (i.e. everything inside) is either complelty inside or outside of the frustum, copy all matrices or set visible count to 0.
[If I understand correclty] for partially inside chunks, it does a shady HACK. Specifically inside VisiblePartial() function.
if(hasWorldMeshRenderBounds) then it "compacts" the matrices for each entity inside the frustum to be contiguous, but at OTHER ENTITIES VisibleLocalToWorld DATA!!!
This chunk's component data array can later be directly passed to DrawMeshInstanced as a matrix array.

PackVisibleChunkIndices Job
Pack chunks in array only if some entities are visible (detected by CullLODToVisible).

UpdateMissingVisibleLocalToWorld
Use shared component type "OrderVersion" to poll changes to component layout/structure (changes in SharedComponentData or Archetype) to update caches. (See https://forum.unity.com/threads/solved-conciling-procedural-generation-with-ecs.545352/)

 

Out of interest how does it compare/differ to this solution?
https://forum.unity.com/threads/mat...shinstancerenderersystem.562369/#post-3773455

 

I didn't look very much into it, but I can see it's better thought and more general solution.

Mine is pretty basic, you have to edit a couple of parts to use it and just has "one" format.
Could be enhanced to take a couple delegates for genericity. Even map each SCD to MPB+delegates to get the appropriate props.

As per the last message in the other thread, it seems to have an issue randomly changing color? I've just added a config to my test project to check sequentiality (incremental position and hue) and works ok with 50k ents, but haven't tested it very thoroughly.

I kind of lied though, I forgot to mention you do need to modify a bit your shader, because (EDIT: You don't need to change the shader, it gets auto-casted, just need to cast to float when updating the cache) MaterialPropertyBlock doesn't accept Int, Bool or Texture arrays. Just Float, Vector4 (float4) and Matrix4x4 (float4x4).
You can cast to float and back, but of course you'll loose int precision/range around it mantissa's 23bits.
I have to try and see if you can do a reinterpret cast using memcpy raw pointers (and if it works ok in the shader), maybe texture works with an int handle too...? (EDIT: No, see below)

 

So yeah, apparently Unity is casting ints from SetInt() to float anyway (maybe to keep OpenGL ES compatibility?), reinterpret casting isn't working, regular int to float works.

Code (CSharp):

1.  
2.         unsafe void CopyCustomDataToCache(CustomDataCache cache, ArchetypeChunk chunk, int offset, int count, ArchetypeChunkComponentType<Dat_CustomRenderData> customDataType) {
3.             var customData = chunk.GetNativeArray(customDataType);
4.             Dat_CustomRenderData* p = (Dat_CustomRenderData *)customData.GetUnsafeReadOnlyPtr();
5.             fixed (float* q = cache.testInt) {
6.                 for(int i = 0; i < count; i++) {
7.                     cache.color[i + offset] = customData[i].color;
8.                     //NO!: UnsafeUtility.MemCpy(q + i + offset, &p[i].testInt, UnsafeUtility.SizeOf<float>());
9.                     cache.testInt[i + offset] = (float)customData[i].testInt;
10.                 }
11.             }
12.         }
13.  
14.     struct Dat_CustomRenderData : IComponentData {
15.         public Vector4 color;
16.         public int testInt;
17.     }
18.  
19.     class CustomDataCache {
20.         public const int MaxLength = 1023;
21.         public Vector4[] color = new Vector4[MaxLength];
22.         public float[] testInt = new float[MaxLength];
23.     }
24.  

For textures you can work around doing SetTexture with a Texture2DArray (which extends Texture) apparently:

https://forum.unity.com/threads/dra...e-texture-via-material-property-block.557020/
https://docs.unity3d.com/ScriptReference/Texture2DArray.html

 

It's a confirmed bug in Graphics.DrawMeshInstanced where batches are merged incorrectly. Workaround to explicitly set the material's value for the property. I'm using CommandBuffer to draw, which doesn't have this issue.

 

That's not happening to me (2018.2.18f1). Maybe because I always set 1023-sized arrays to the MPB (even for 1 instance).

Link/source?

Here's 100k , 2500 and 100 cubes (ordered by position and hue), not showing any problems:

 

The random color changes only happened to me once in the demo and I haven't seen it since I implemented it into my own project. Not sure if it was fixed or I just haven't run into the circumstances that cause it.