One Batch: Combine hundreds of materials into one draw call

There's a set of amplify shader nodes included to decode the vertex data, so you basically get indexes and properties (tint, normal strength, etc) out from the nodes.

 

Afaik OneBatch is modifying the alpha channel of vert colors to distinguish the index of the arrays to use per object. The amplify shader function decodes it, yes, but I'd need the vert color info still. Unless I'm misunderstanding how the tool is working..?

Would be nice if there was just a button for writing that vertex data and generating the prop tex. Just decoupled from the texture array generation part, where the error is happening. Again, my main issue is the ASTC error, so if this is not a trivial feature to add, I'd be happy on any pointers on how to fix that error.

 

I mean you can change what FORCE_ETC2 maps to in the TextureArrayConfig to any format you want and figure out what your device really supports. I'll port over the code that loops through the formats and figures out what Android supports soon though.

 

It just stores two indexes in the vertex color- one into the array, another into the propData. The amplify node gives you the array index into the texture array along with the values decoded by reading the propData, so tint, normal strength, etc..

 

Just checked every compression option (Automatic, DXT, PVR, ETC2, ASTC, Crunch, and Uncompressed) . same exact error:


Beginning to think maybe this is related to a unity version problem? I'll try to see if this happens on an earlier version

 

Yeah, it's on Android (for Oculus Quest).

I will send you a link to my packaged scene - perhaps you could first send me your contact details to ciaran_at_maze-theory_dot_com.
Thanks

One other thing to note - The existence of my oneBatch config asset within the project structure causes the error (reported above). ie I am not required to run it for it to error - which seems odd.

 

@jbooth please could you give me your contact details to ciaran_at_maze-theory_dot_com - so I can get this scene over to you for diagnosis. Thanks.

 

Found a solution to my problem! Turns out I had in my build settings forced texture compression settings to ASTC (see screenshot), which was somehow interfering with the tool when creating texture arrays. I just switched it to the default 'dont override' and that fixed it the compiler error I was getting.

If anyone else was having a similar error..just changing the TextureArrayConfig didnt do the trick for any of the compression options, so I guess unity build settings can interfere if its set to a specific compression method..? Anyway, I'm up and running for now.

 

Well, Im back again already. Now that my compiler error is gone I believe I am having the same problem as @cd_mt :

Basically I run through everything and my scene has the material applied with the texture arrays slotted in, but all my objects look plain black.

Steps I'm taking:
1) I've written myself a tool that prefabs any unique meshes out and replaces non-unique ones with those prefabs, so I have a bunch of prefabs (nearly 100) that I'm feeding into the config. Each of those prefabs is just the mesh, with the mesh renderer (and some case colliders) and no children object.

2) I drag these into the prefab slot on a newly created config and hit extract.

3) one batch generates a bunch of entries**

4) I've tried both messing with the package settings and leaving defaults (with the same results) and hit combine.

5) after some time I see that texture arrays have been generated, im able to preview them in the inspector and their various indices, i see the mesh filters that have been created and populated into the prefabs, and I see that the onebatch generated material has been applied to all those prefabs.

Only thing I can think of is the vertex to uv (?) mapping process isnt happening properly...? For now I'm going to be putting a pin in this particular optimization and revisit later, but I'll keep a tab on this one..

**out of curiousity, is there some sort of optimization that happens when the texture arrays are created for prefabs that share the same material? I have a bunch of these sort of cases, and it felt wrong that there was a separate entry for each instance of a material on a mesh, rather than just for the material and the meshes share that index.

 

Maybe break it down to just 2 prefabs and see if there's something unique about one of them that's causing the issue. One Batch was written before nested prefabs, and there may be something about those changes causing an error? If you can then package up those assets I could take a look.

Things which share the same material are combined- however, things which share the same textures but not the same material are not- because the material properties might be different (though if the texture combination is the same, it will be shared).

 

So I pushed an update to the store today (maybe out tomorrow?) which aggressively tests texture formats and falls back through them on android. When set to auto, it will first test to see if the project is set to use ASTC and choose either 4x4, 8x8, or 12x12 based on compression settings, then if that isn't supported fallback to 4x4 if that is supported, then test/fall back to ETC2/1, PVRTC, DXT and finally uncompressed RGBA.

 

That's a very interesting system, but I have some questions:
1- I can't use static or dynamic batching. Will OneBatch give any benefits?

2- Does it work with Unity's texture streaming? Will it stream all the textures in the array at once or don't work at all?

3- Also, does it respect the original renderer sorting order?
It's important for transparency but also because I reorder renderers from front to back manually to aggressively avoid overdraw (yeah, the old fashioned way, but you would be amazed how well it works for mobile SoCs were the GPU is so bandwidth limited while the CPU is underused) (that's why I can't use batching).

Thanks!

 

1. Likely not - unless your system benefits from having things atlased in some other way.
2. No, a texture array is a single GPU resource, so will be loaded/unloaded as one resource.
3. No, because all objects (of a given type- opaque, transparent, etc) are a single material.

 

Hi, just bought and played with this for the first time. The tool looks promising but I noticed a couple of issues on Unity 2019.2.17f1 on Android:

- Combine generates a lot of warnings like this:

SendMessage cannot be called during Awake, CheckConsistency, or OnValidate

UnityEditor.AssetDatabase:SaveAssets()

JBooth.OneBatch.OneBatchConfigEditor:CompileConfig(OneBatchConfig, List`1, String, Boolean) (at Assets/GitExclude/OneBatch/Editor/OneBatchConfigEditor_Compiler.cs:937)

JBooth.OneBatch.OneBatchConfigEditor:CompileConfig(OneBatchConfig) (at Assets/GitExclude/OneBatch/Editor/OneBatchConfigEditor_Compiler.cs:955)

JBooth.OneBatch.OneBatchConfigEditor:DelayedCompileConfig() (at Assets/GitExclude/OneBatch/Editor/OneBatchConfigEditor_Compiler.cs:299)

UnityEditor.EditorApplication:Internal_CallDelayFunctions()

- The combined materials are dark. Increasing the Occlusion value from 1 to 10 for all combined materials fixes this (I'm not using Occlusion maps).

- Revert does revert the prefabs correctly but prefab instances in the scene don't get updated. A workaround is to select each prefab in project hierarchy after reverting (must have inspector unlocked). Upon selecting a prefab, its instances in the scene are also updated.

 

The warnings are odd, because it's just in a call to AssetDatabase.SaveAssets(), which is unity's internal function. (There are no send message calls in OneBatch, so it's being called from in there). Maybe something with the asset database v2 if you're using that? Interesting bug with the revert too- must be some kind of update glitch with the new prefab system. I'll see if I can repro and find some kind of hack to refresh them automatically, like forcing the inspector to select them all or something.

 

@jbooth any chance of adding compatibility to LWRP/URP/HDRP like you did for microsplat?

 

Not likely- it's a ton of work, and the way that the SRP shaders work means basically having to re-do that work every release because of changes unity makes, so the $100 this makes each month doesn't really justify it.

 

for some reason when I revert the prefabs used in scene revert but when you open the actual prefab the one batch array material and mesh haven't been reverted. anything specific I need to do to avoid this?

 

strange; I would expect the opposite. Maybe save project and swap to another app and back to see if it refreshes the data?

 

Hi @jbooth

Would you mind explaining in technical terms what you're doing to allow tiling on an atlas? Is it like MeshBaker? Couldn't find an answer to that in this thread or the documentation. Thank you.

 

Not using an atlas. I'm using texture arrays instead.

 

Just watched your video. Very interesting. So these arrays store different materials, tiling or not, and allows them to behave like individual materials that Unity sees as a single material?

 

No, arrays just store textures. You can have up to 1024 textures in an array, and they act as a single GPU resource. However, all textures in the array have to be the same size (this way accessing them by index is just a known jump in memory). So basically you end up with a 3 coordinate UV where the 3rd element is which texture you want to sample.

What OneBatch does is to put all your textures into these arrays, put all the properties on your material into a texture, burn the lookup indexes into those things into the colors of the mesh, then at runtime it uses that information to access the data in the arrays. It's smart about material/texture differences too so that two materials using the same textures but different material properties don't end up with 2 copies of the textures in the array, etc.

This basically allows everything to tile as normal, but obviously requires custom data on the meshes and a custom version of the shader which works with this data, but then everything is the same material as far as rendering is concerned.

 

Oh wow. That's pretty darn clever, sir. I do all my own atlases for meshes in 0,1 - but still have some tiling materials I'd like to take care of. Thank you for the explanation.

 

thanks yeah was something as simple as this, sorry to trouble you should have checked the obvious first

 

Can One Batch support the Universal Render Pipeline.

 

I have no plans for this- maintaining support for SRPs is a daily nightmare as the pipelines are constantly changing and breaking things, and no documentation is available for the changes.

 

Hi, it's been quite a while since I tried to optimize a couple of levels with this tool in Unity 2019.2. I like the idea how this is supposed to work and I was able to reduce batches to a small number indeed. Unfortunately the measured performance actually decreased slightly instead of improving. Decreases were apparent inside Unity Editor 2019.2, Windows Standalone Build and Android Build running on Oculus Quest.

I ended up optimizing materials by hand to get the actual performance benefits I was after. That was of course a lot of work not to mention I had to mess UVs of models that I combined together in a shared material.

Have you made measurements regarding actual performance gains? I'm still hopeful this tool could be used in the future.

 

Performance gains are entirely dependent on where you are bottlenecked. If you weren't bottlenecked on SetPass calls, then you're not going to see performance gains. On quest, you are usually bottlenecked on GPU fill rate, which has nothing to do with SetPass calls, so reducing them won't help if that was the case.

This is why I include an extensive optimization guide in the documentation, because what I find is that most people don't even know why their frame rate is slow, and often reach for optimization tools that optimize the wrong thing. This tool is all about reducing SetPass calls to save CPU time, but it won't do a thing to help with GPU fill rate, and actually will increase it very slightly.

 

Hi, thanks for your answer. You know better and I'll be looking into this more but note that I was able to achieve 72 fps on Oculus Quest by combining level materials manually. Also, the performance of original un-optimized scene was higher than on OneBatch'd scene in Unity Editor and Standalone Build (GTX 1070) -- not only Quest. That's why I was asking for evidence of some real world performance gains with this tool....

 

One Batch was designed originally for a product which was heavily draw/setpass call limited, and on those scenes it got us about 3ms back. If it helps or not is entirely dependent on your scene and bottlenecks.

 

Hi, I'm having an issue with objects getting non-uniformly lit. They are darker on one side and lighter on the other side of the meshes. It seems like a problem with the occlusion strength values. When I adjust them for individual materials from the OneBatchConfig it makes the darker areas light and it looks more like the original materials. Please watch the video to understand what I mean. I have also attached the link to the scene with the used assets.

Can you please help me with this?, I have a lot of assets and adjusting occlusion values for each one of them after combing materials is super tedious.

Thank you

Video: https://drive.google.com/file/d/1a1bCahZZVZrsxxvBjlUoEteh0d93sLF6/view?usp=sharing

Scene: https://drive.google.com/file/d/1LCqccvxEqels3Wm4sDmgTQ_uht3XAUCL/view?usp=sharing
​

 

I'll take a look. When you adjust the AO, is it always to the same value? Like is it just inverted or something simple like that?

 

Thanks,

It's usually off by 1, but not always. I suspect that the PropertiesTexture isn't parsed properly at the shader side. The combined material generated by OneBatch never matches exactly to the original materials(Even the simplest materials), no matter what settings you tweak in the OneBatch combined material.

I have also noticed another thing. If in the original (Uncombined) materials you check the Specular Highlights and Reflection boxes in the Forward Rendering Options and then you combine these materials using OneBatch and adjust the occlusion strength to be twice of the original, then the combined material matches exactly with the uncombined materials.

 

@jbooth Did you find any solution to my problem ?. I have stalled my work and am waiting for you. Please help me with this. My invoice number is : IN010200825777

Thank you

 

I haven't had much time to look at it yet- been finishing up a contract. I'll hopefully get to it this week though..

 

Alright, Thank you. I'll wait for your response this week.

 

Hi @jbooth

Do you have a workaround (aside from removing the affected prefabs) where the parent uses an opaque texture, but its child uses transparent? As a result of the auto-combining of submeshes, this forces the child to use opaque rather than transparent which is what it needs.

For these scenarios, could OneBatch just extract these submeshes instead of combining so the correct version of the OneBatch material can be applied?

The only workaround I know of is editing the mesh data in an external 3d modelling program to make 2 unique meshes, but depending on the amount of content requiring this kind of editing this could get tedious fast.

 

You mean submeshes, not child/parent game objects right? If you assign the transparent material does it work? Not sure if this is a bug or design limitation from your description..

 

Submeshes. None of the other OneBatch generated materials fixed it . However, it's really isn't a big deal for someone able to separate submeshes using mixed rendering mode materials in a 3D app.

What happened is OneBatch processed a mesh with 6 submeshes using 4 opaque materials and 1 CutOut material, but skipped the texture on the CutOut material and replaced it with the texture of a rock mesh. Maybe the rock texture was the next opaque texture on the array?

Anyway, not a big deal. Guess I brought it up more as a roundabout question if it a check for mixed rendering methods on meshes could be implemented as a safeguard, since even the best content from the asset store can be really messy.

The attached screenshots illustrate both the original mesh and the OneBatch generated version, and how the glass CutOut texture is omitted for an entirely different texture in the OneBatch Test material.

 

Seems like a bug- if you assign the cutout shader to it instead of the original, does it work?

 

No, but I also wouldn't expect it to since the rock texture has no alpha channel.

 

It's been 6 months since this was asked and URP/HDRP should be more stable now. Can you please make our One Batch materials upgradable to URP/HDRP?

 

No, and if you think URP/HDRP are stable you haven't been paying attention. Every release requires a complete rewrite- if you want One Batch for URP, you are free to port it yourself since you seem to think it's so easy to maintain.

 

Wow, that bad huh. You're right I haven't been paying attention. This makes our decision to move to HDRP now a little easier. Thanks.

 

i would like to leave a note that the Generated cutout shader did not cut out the Alpha , but the when i made a custome shader using ASE it worked just fine

the render option i chosed was Transparent Cutout
you can check the Shader below

 

Any Update coming for One Batch , there is still a problem in Alpha Cut and ASE nodes

 

I noticed that the Render Queue setting in the OneBatch Standard shader UI has no effect and gets reset when the shader UI is closed. Updating the code in OneBatchStandardShaderGUI.cs OnGUI function to:

Code (CSharp):

1. if (m_FirstTimeApply)
2.          {
3.             int renderQueue = material.renderQueue;
4.             MaterialChanged(material, m_WorkflowMode);
5.             material.renderQueue = renderQueue;
6.             m_FirstTimeApply = false;
7.          }

That seemed to fix it. Also in Unity 2019.4.x LTS Some of the UI in the OneBatch asset seems to be drawing twice. Doesn't seem to be causing any errors but it is a bit confusing.

 

Whats the best workflow for adding an additional mesh to an existing Config? Is it best to revert, then combine?

 

Yes..